@adamshostack@infosec.exchange
Post #4148030
2026-07-27 21:07 UTC
Understanding the value of an improvement, rather than focusing on a single benchmark, is a crucial part of engineering. Engineering is always about tradeoffs, not “maxxing.”
(11/15)
Replies (1)
-
@adamshostack@infosec.exchange 2026-07-27 21:07
We see these issues: anthropomorphization, overreliance on benchmarks, and aspiration-driven thinking all the time as we help clients navigate how to use LLMs in threat modeling, but they are really about LLMs. They extend far beyond threat modeling. That's why we included two (anthropomorphization and over-reliance) in version 1 of the PHANTOM-B model (/resources/whitepapers). Who knows, maybe a future version will need benchmarkmaxxing as a threat? (12/15)