Replies (1)

• @david_chisnall@infosec.exchange 2026-09-19 09:53

  @flyingpenguin@infosec.exchange I’m not sure I agree because it helps to clearly frame these things as tools that cannot be used safely in those contexts. A lot of ‘AI’ marketing is intentionally misleading. It portrays a probabilistic token predictor as an intelligent entity with a pool of knowledge and a reasoning framework on top that allows it to make intelligent decisions. And that leads to it being applied in places where that kind of tool would be useful. It is absolutely the vendor’s fault that it causes these problems. But a narrative that says ‘anyone who uses these machines in these stupid ways is to blame’ pushes the responsibility back to people who make the purchasing decisions to enable them. And that’s much more likely to result in getting them out of these applications than trying to force the companies that have an enormous commercial incentive to change their marketing. To me, it’s a question of which incentives are easier to change. As long as you can blame the tool for failure when used in a highly inappropriate (but vendor-advertised) way, there’s an incentive to keep deploying it and the incentives for the vendors to lie and for their customers to believe the lies are aligned. That makes it hard to make them stop lying because a bunch of the people who would be responsible for enforcing the regulations that would prevent them lying are financially incentivised to believe the lies. If you avoid the liability shift and place the blame on the users, then the users are financially disincentivised to believe the lies. And this means that they are incentivised to push back against the lies and to lobby for banning the lies. Now, rather than having LLM vendors and users pushing back against your goal (or, at least, my goal) you have LLM vendors pushing back but LLM users pushing forward.

  Open ##4760936