Post #1362195

2026-04-09 03:15 UTC

[XCancel link](https://xcancel.com/HeidyKhlaaf/status/2041591737563394442) for those of us sick of being badgered to sign up/in On a more productive note, this feels likely to be tied in with the usual issues of AI sycophancy re: false positive rate. If you ask the model to tell you about security vulnerabilities, it's *never* going to tell you there aren't any, any more than existing scanners will. When I worked for F5 it was not uncommon to have to go down a list of vulnerabilities that someone's scanner turned out and figure out whether they were actually something that needed mitigation that could be applied on our box, something that needed to be configured somewhere else in the network (usually on their actual servers) or (most commonly) a false positive, e.g. "your software version would be vulnerable here, which is why it flagged, but you don't have the relevant module activated and if an attacker is able to modify your system to enable it you're already compromised to a far greater degree than this would allow." That was with existing tools that weren't trying to match a pattern and complete a prompt.* Given that we've seen the shitshow that is Claude Code I think it's pretty clear they're getting high on their own supply and this announcement ought be catnip for black hats.

Replies (2)

@wizardbeard@lemmy.dbzer0.com 2026-04-11 16:59

Wow, sounds like they just automated "shitty infosec teams that only forward scanner output without evaluating it" out of a job. Holy shit they were right that AI was coming for jobs!

Open ##1362196
@scruiser@awful.systems 2026-04-11 17:16

> On a more productive note, this feels likely to be tied in with the usual issues of AI sycophancy re: false positive rate. I suspect this is the real limit. Claude Mythos might find real vulnerabilities, but if they are buried among loads of false positives it won't be *that* useful to black or white hat hackers and the endless tide of slop PRs and bug reports will keep coming. I tried looking through [Anthropic's "preview"](https://red.anthropic.com/2026/mythos-preview/) for a description of the false positive rate... they sort of beat around the bush as to how many false positives they had to sort out to find the real vulnerabilities they reported (even obliquely addressing the issue was better than I expected but still well short of the standard for a good industry-standard security report from what I understand). They've got one class of bugs they can apparently verify efficiently? > Memory safety violations are particularly easy to verify. Tools like Address Sanitizer perfectly separate real bugs from hallucinations; as a result, when we tested Opus 4.6 and sent Firefox 112 bugs, every single one was confirmed to be a true positive. It's not clear from their preview if Claude was able to automatically use Address Sanitizer or not? Also not clear to me (I've programmed with Python for the past ten years and haven't touched C since my undergraduate days), maybe someone could explain, how likely is it that these bugs are actually exploitable and/or show up for users? Moving on... > This process means that we don’t flood maintainers with an unmanageable amount of new work—but the length of this process also means that fewer than 1% of the potential vulnerabilities we’ve discovered so far have been fully patched by their maintainers. So its good they aren't just flooding maintainers with slop (and it means if they do publicly release mythos maintainers will get flooded with slop bug fixes), but... this makes me expect they have a really high false positive rate (especially if you rule minor code issues that don't actually cause bugs or vulnerabilities as false positives).

Open ##1362197