@adamshostack@infosec.exchange
Post #4147055
2026-07-27 21:07 UTC
Volume overwhelms everything, especially judgment. The team at OpenAI either can’t or won’t slow down to look at the output of these systems. Now, maybe, that’s the right call? Velocity is its own reward? That’s certainly possible - maybe small shifts in position on benchmarks herds customers onto those models, and so a win on the benchmark is a revenue driver. (If only the AI folks knew about the complexity of gradient-climbing as a strategy? 🤷) But more seriously: OpenAI gets business value out of using its technology, learning where it adds value, and trumpeting that as a new value prop for LLMs. Apparently that’s not... parsing the output of its tools.
(8/15)
Replies (1)
-
@adamshostack@infosec.exchange 2026-07-27 21:07
Benchmarks are useful when they measure what matters. As far back as 1975, Goodhart wrote “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes,” or as Marilyn Strathern memorably rephrased it, “When a measure becomes a target, it ceases to be a good measure.” The possibility of stealing test answers may be the ultimate expression of Goodhart’s law. (9/15)