Post #4390381

Replies (1)

• @tbortels@infosec.exchange 2026-08-05 05:29

  @darkuncle@infosec.exchange I am a big fan, and it would harsh my mellow to bag on Alice - but the argument they are having with whoever is missing an important point. AI output is inherently untrustworthy. Ok! Lots of things are. Noise makes things untrustworthy. Bias does too. Humans are notoriously untrustworthy. That's why we have error detection and correction mechanisms - feedback loops so we don't accept erroneous data as valid. This has been studied for *decades*. And a lot of the mechanisms can and do apply to AI output. You don't use AI output directly - not if you're savvy. You feed it into an error detection/correction step. Sometimes that's easy - does the produced code compile and pass its unit tests? Strong indicator there you're ok. Don't have unit tests? Ah - I see the problem. Point is - you don't have the AI do data analysis. It's really bad at it. You have the AI write code to do data analysis. It's good at code. And you test it, exactly as you'd test your own code. We all test our code - right? It's harder if it's not code - but it's the same as having an intern. You gotta check the results. Hit the URLs. Spot check facts. Some of that in some cases can be automated, but when you can't check the output - you had no business asking the AI to begin with. If you ask the AI for advice about life-and-death - that's evolution in action. Now - there's plenty of excellent reasons to avoid AI that have nothing to do with unpredictable output. They aren't wrong. IP issues, market issues, expense and the bubble and data centers and layoffs - all big problems. But they's *human misuse* issues. We had similar ones with cybercoin and NFTs and data brokers and a lot of other techs - and it all boils down to human misuse. https://youtu.be/CD9YqdWwwdw It's worth noting in the clip above (spoilers): HAL was correct. It was human error. GIGO.

  Open ##4390380