Post #1379253

2026-04-15 15:05 UTC

Has nobody suggested that a lot of the supposed vulnerabilities this Mythos gizmo claims to have found are bullshit? Because that&#39;s what we&#39;ve seen with genAI bug reporting so far. Vast quantities of generated bug reports are bullshit. I don&#39;t understand why Mythos would be any different. Yet I just read a piece by somebody I thought was reputable enough to put in my feedreader that swallowed &quot;Mythos vulns are Big Scary News&quot; hook line and sinker.

Replies (7)

@bitsgalore@digipres.club 2026-04-15 15:13

@dsalo Here's some useful (I think) context by @davidgerard: https://pivot-to-ai.com/2026/04/09/claude-mythos-the-ai-hacking-model-too-good-to-release-allegedly/ This also links to this study which basically concludes Mythos' performance is similar to existing, smaller models: https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jagged-frontier

Open ##1455822
@mdekstrand@hci.social 2026-04-15 16:11

@dsalo my sense from discussions I’ve seen, e.g. https://lwn.net/Articles/1065620/ and the blog post it is replying to, and what Daniel Stenberg has been saying lately, is that even without Mythos there has been a massive step change in the last few months: going from “AI reports are almost all useless” to reliable, repeated AI-powered discovery of real vulnerabilities, or at least defects. That increases the odds I would place that Mythos is scary-good at this specific task.

Open ##1455824
@pauamma@mstdn.social 2026-04-15 22:35

@dsalo Have you read https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos-previews-cyber-capabilities ? (More on this upthread and downthread from https://bsky.app/profile/ciaranm.bsky.social/post/3mjicz7m2ws2u)

Open ##1455828
@Da_Gut@dice.camp 2026-04-17 12:51

@dsalo they still need to be investigated in full. Unfortunately, past performance does not indicate present performance if the cost of it being right is too high. You still got to spend the resources on a parallel route to establish that.

Open ##1455832
@sl1200@nixnet.social 2026-04-17 12:56

@dsalo Those ‘AI‑induced vulnerabilities’ from Mythos are just marketing bullshit to catch suckers who pour money in and feed the AI bubble.

Open ##1455843
@redsakana@infosec.exchange 2026-04-17 15:34

@dsalo Anthropic seems little different from a hustler hawking ivermectin to the manosphere/antivaxx crowd: they have found an audience that will believe (and buy) _absolutely anything_ as long as the speaker has impeccable credentials within their respective echo chamber, and zero qualms about using that influence. The main difference being that their audience has a lot more spending money.

Open ##1455844
@kevingranade@mastodon.gamedev.place 2026-04-17 22:55

@dsalo if you've ever worked with a static analysis tool on a large codebase, "reported vulnerabilities that have a chance of being significant" are a dime a dozen. The hard part is triaging the massive number of mostly not meaningful reports and finding the handful of ones worth addressing, and the extra extra hard problem is doing this in a way where you can keep running the tools without having to repeat the whole audit process every time. AFAIK Mythos doesn't address either of these points.

Open ##1455845