Post #4502682
2026-08-10 23:39 UTC
But there's _zero_ movement on a pervasive sandbox. Everyone's blithely proceeding as if the OSS packages are still as thoroughly reviewed, despite the majority adopting "agentic" tools. Headline after headline about "rogue AIs[2]" are doing bad stuff, whether it'sd dropping the production database or hacking a competitor.
[2]: random outcome generators with outcomes insufficiently constrained by robust technical safeguards, where human & process-based safeguards have been implicitly eliminated
Replies (1)
-
@glyph@mastodon.social 2026-08-10 23:42
This is the logic of the O-ring disaster. Everybody just relies upon everybody else's safety margin — in the software community, that safety margin is built mostly out of earnestness and the willingness to sit with a problem and *really* understand it — and hollows out their own, assuming that the load will fall somewhere else. https://en.wikipedia.org/wiki/Space_Shuttle_Challenger_disaster#O-ring_concerns