Post #2854567

The list of valid use cases for AI is bound by “what is the worst possible consequence of a mistake done here”, because the statistical distribution of mistakes in terms of severity of consequences of things like Agentic AI is uniform (meaning, they’re just as likely to do the worst mistakes with the nastiest consequences as they are doing the smallest mistakes), which it is not the case with humans who make more of an effort and give more attention to avoiding catastrophic mistakes and also have a “this is stupid” (i.e. don’t put glue in pizza, don’t tell a suicidal person to kill themselves) recognition capability which also stops a lot of the nastiest mistakes. This is something which is not noticeable to most people because most people don’t have deep enough process experience in at least one expert domain and process analysis experience, to upfront recognized anything beyond the “in your face” elements of using AI (or using anything, really) in a process. Very few people would think “what’s the risk profile for this business of giving this thing these responsabilities”. So they seriously overestimate what are valid use cases for AI, something that the hype around it also pushes for: not a single AI vendor will ever mention just “error distribution” or anything close to it. Obviously, when the thing blows up catastrophically by doing something which for a human is “obviously a bad idea”, THEN people recognized that AI is unsuitable for that, but by then its often too late. (Easy example: lawyers using AI to make submissions to the Court and ending up disbarred because those submissions “quoted” invented case law). So I don’t expect Agentic AI to fuck society up by taking a large fraction of the jobs, I expect Agentic AI to fuck society up by an accumulation over time of random catastrophic mistakes that kill people and collapse otherwise stable companies, mistakes that humans in such positions would never do or at least be way less likely to do. It’s going to be akin to death by cummulative poisoning.