Post #4760936 · Elektrine

2026-09-19 09:53 UTC

@flyingpenguin@infosec.exchange I’m not sure I agree because it helps to clearly frame these things as tools that cannot be used safely in those contexts. A lot of ‘AI’ marketing is intentionally misleading. It portrays a probabilistic token predictor as an intelligent entity with a pool of knowledge and a reasoning framework on top that allows it to make intelligent decisions. And that leads to it being applied in places where that kind of tool would be useful. It is absolutely the vendor’s fault that it causes these problems. But a narrative that says ‘anyone who uses these machines in these stupid ways is to blame’ pushes the responsibility back to people who make the purchasing decisions to enable them. And that’s much more likely to result in getting them out of these applications than trying to force the companies that have an enormous commercial incentive to change their marketing. To me, it’s a question of which incentives are easier to change. As long as you can blame the tool for failure when used in a highly inappropriate (but vendor-advertised) way, there’s an incentive to keep deploying it and the incentives for the vendors to lie and for their customers to believe the lies are aligned. That makes it hard to make them stop lying because a bunch of the people who would be responsible for enforcing the regulations that would prevent them lying are financially incentivised to believe the lies. If you avoid the liability shift and place the blame on the users, then the users are financially disincentivised to believe the lies. And this means that they are incentivised to push back against the lies and to lobby for banning the lies. Now, rather than having LLM vendors and users pushing back against your goal (or, at least, my goal) you have LLM vendors pushing back but LLM users pushing forward.

Replies (1)

@flyingpenguin@infosec.exchange 2026-09-19 10:13

@david_chisnall@infosec.exchange blame flows to the least powerful, who knows every reason to stay silent. Aviation proved it clearly. "Pilot error" as a standard finding suppressed reporting until NASA's blame-free ASRS launched in 1976 and reports surged. That's because blame culture buries failures. I never, ever require a fix for a bug report, similarly. In 2017 I personally, at great cost, flipped MongoDB from a couple of CVE a year to tens of thousands of secret defects triaged and dozens of public CVE. Eliminated blame, created transparent CNA. You say tools that "cannot be used safely in those contexts." That says defect claim to me. A product that cannot be used safely as advertised is defective in its advertising, and saying so puts the vendor on the spot. It is hard and it is also the only thing that has ever worked. We don't go to the moon because it is easy. Placing accountability on the victim who cannot inspect the model makes the lie free to tell. That's the whole Elon Musk snake oil pyramid. His buyers are dispersed and cannot verify claims, which is the entire premise of false advertising law. Liability sits with the cheapest cost avoider (Calabresi, 1970). Assumption of risk was the tobacco defence too for forty years: smokers chose, smokers knew. Smoker liability never produced a smoker lobby. The 1998 settlement after 10s of millions died came from state attorneys general suing the manufacturers. Driver blame ran the auto industry from the 1920s until 1966. Drivers never organised against the design. The Motor Vehicle Safety Act came from putting the flawed designs on trial. A Miami jury just did what I have been talking about for over a decade. $243 million against Tesla in August 2025 for Autopilot design failure, $200 million of it punitive. It shouldn't have taken so long after 2015 immediately revealed the Tesla defects in AI.

Open ##4760934