Post #2795366

@canacar@ioc.exchange I mean, I do V&V on safety codes, both deterministic & probabilistic. We have documented use cases, requirements, tests & inspections, acceptance criteria, etc. You actually can test probabilistic code, you just need appropriate acceptance criteria. I cannot tell you what the acceptance criteria are for any of these agentic systems other than "line goes up". They are tuned to be obsequious and compliant and agreeable. I'm sure all their unit tests pass; you can write all the unit tests you want and still never deliver a system that's fit for purpose. Unless and until the companies paying for cjatbots and agentic systems step up and demand actual software QA with defined use cases and relevant, verifiable acceptance criteria, the AI tool vendors will keep shipping randomly destructive trash. If you're the one writing the checks for these systems, it's on you to hold your vendor to account and ensure you are getting what you intended to get. It sounds like a fantasy world but somewhere in all this is a Procurement and SQA bureaucracy specifically intended to make sure deliverables are specified and confirm what's received is as described and meets spec. If there isn't, if IT or Engineering is just writing checks and nobody is doing the slightest technical diligence, the organization is getting exactly what they deserve. It's a systemic organizational problem far larger than "admin didn't write a good enough chatbot prompt". Honestly, investors and insurers need to start holding CTOs and CEOs responsible for these sorts of failures. Show that the investment was used wisely, that appropriate technical diligence was exercised, the risks taken were known and reasonable. Workers will make mistakes, accidents will happen, but if you aren't working to understand and limit risk, you're not doing your job as an executive. And if the investors are the ones pushing for AI use, maybe it's time to find a better class of investor, one that isn't actively trying to burn the business to the ground.