Post #3219998

2026-06-03 13:45 UTC

@whitequark@treehouse.systems@astolfo.social @lina@vt.social I can imagine it being useful as a generalization of fuzzers in some cases, but I think circulating the uncritical claims that LLMs don't work at all is dangerous even just because of the use cases where they can successfully be used for fucked up shit

Replies (1)

@whitequark@social.treehouse.systems 2026-06-03 13:52

@natty@astolfo.social @lina@vt.social I don't think I've ever said that; LLM-backed automated exploitation is one of the few applications which deliver some tangible results. in my evaluation (I've used V12), most of the bugs they find are not security relevant at all, or are DoS vectors that would not be practically useful, but there are definitely pretty major issues like RCE or full auth bypass too. denying that would be counterfactual. what is much less clear is whether the ROI is there considering the opportunity cost of using the resources you've funneled towards a company which explicitly aims to make you dependent on their service (and deploys literal gacha mechanics to that end too) instead towards improving your tooling, processes, and collaboration (I don't have access to Mythos myself but I've talked to several people with personal experience of being on the receiving side of that particular firehose and they seem to broadly agree, ± the exact place in the tradeoff spectrum) (edit: word order)

Open ##3219999