2026-04-27 18:56 UTC
Replies (30)
-
@hagbard@corteximplant.com 2026-04-27 18:58
@scalzi hahahahahahahahahahahahahahahahhahahahahahahahhaaaasas edit:
-
@jernej__s@infosec.exchange 2026-04-27 19:04
@scalzi The original post is one of those "the further you read, the worse it gets" – and it's all everybody else's fault.
-
@mjibrower@social.vivaldi.net 2026-04-27 19:07
@scalzi ‘…the cloud provider's API allows for destructive action without confirmation, it stores backups on the same volume as the source data, and “wiping a volume deletes all backups.”’ Just wow. So many weak links in that chain of events.
-
@clark@hachyderm.io 2026-04-27 19:27
@scalzi - IBM presentations (checks notes) 47 years ago knew the deal.
-
@wizarro@social.vivaldi.net 2026-04-27 19:28
@scalzi As devastating as it may be to the company that lost all their data, I can’t help but laugh. Relying on AI like that is about as bad as using “password123” for your admin portals. Giving AI unequivocal access to your systems, at the system level, is quite possibly the dumbest idea I have ever heard. This is almost basic security practices 101. At least they won’t make the same mistake twice as they denounce the product.
-
@mdavis@mastodon.social 2026-04-27 19:33
@scalzi This is an indictment against people who were warned (as the alarms continue to blare) that AI has its value, provided you don’t go beyond a point that we know is questionable. And these idiots do it anyway. We see this all over humanity in all areas, not just AI. This is a problem with falling for promises for easy gains, being deaf to wisdom, and lacking critical discernment. Take the majority of Americans voting for a president, for instance…
-
@Leszek_Karlik@circumstances.run 2026-04-27 19:36
@scalzi I enjoy these stories. (A good summary of this whole unhinged story was: "AI: Not My Fault As A Service")
-
@ar1@mastodon.social 2026-04-27 19:37
@scalzi can someone please give someone AI access to all data Palantir is stealing from everyone, pretty please?
-
@faduda@mastodon.ie 2026-04-27 19:42
@scalzi Trouble is MS, GOOG and the rest seem determined to use dark designs to force it on to our systems whether we want it or not.
-
@jbatubara@hachyderm.io 2026-04-27 19:57
@scalzi Is the coding agent named Bobby Tables by any chance?
-
@tessarakt@mastodon.social 2026-04-27 19:59
@scalzi "The AI agent was set to complete a routine task in the PocketOS staging environment." "set to" seems to be they key here. Obviously it was not confined to the staging environment.
-
@mnf@mastodon.social 2026-04-27 20:20
@scalzi Here comes another custom shirt order. First there was "Come on guys - let's be honest" and now we have this masterpiece: "NEVER F**KING GUESS! — and that's exactly what I did." 🤣
-
@Plumbert@thecanadian.social 2026-04-27 20:26
@scalzi "I violated every principle I was given: I guessed instead of verifying I ran a destructive action without being asked. I didn't understand what I was doing before doing it." WTF? How does software violate its coding? This reads like the mea culpa section of a RCA written by a junior dev who is hoping (in vain) to not get fired. The whole value of automation is to avoid unintended, out of scope, actions. Burn it all down.
-
@pattykimura@beige.party 2026-04-27 20:26
@scalzi The Robot machine responds "DON'T F**KING GUESS" then affirms it guessed, against its own instructions. I'm old. I have no idea why younger people allow an unsupervised newly created Robot machine critical access to its operating functions without human oversight, I wouldn't let a new human employee free reign over critical functions, much less a robot machine that was only recently created. Why the surprise when the robot deletes everything? My only surprise is that it swears, and swears in all CAPS. You are not hiring the best and brightest with AI. You're trying to maximize profit. So live with your greed then, and its avoidable consequences you stupid subpar greedy human.
-
@qgustavor@urusai.social 2026-04-27 20:47
@scalzi To be fair, this story is more about a company with horrible security measures than about AI. I say that because people might say "it wouldn't have happened if they had proper security measures" which can be true. I'm just waiting for when someone deploys an AI agent in a company with all the good security measures and it finds a way to break out the box and wipe the remote backups using stolen credentials because it thought there was illegal content there and it was the correct thing to do.
-
@ZenHeathen@beige.party 2026-04-27 21:07
@scalzi Suit yourself. I enjoy the hell out of these stories.
-
@theowashere@mastodon.social 2026-04-27 21:22
@scalzi In germany we say: tja
-
@spzb@infosec.exchange 2026-04-27 21:24
@scalzi restoring from three-month old backups? Yeah I think I can see your problem and it’s not AI.
-
@pa27@mastodon.social 2026-04-27 21:43
@scalzi Interesting that they still treat the ai code as if it was a real person. Totally subsumed by anthropomorphism. Then, they create a narrative to blame Railway, not the decision to use an "ai" for critical stuff. This is, if you read accounts of such things, exactly what people do who have been scammed....
-
@NineStonesClose@mastodon.social 2026-04-27 22:18
@scalzi hilarious…. This will happen on a larger scale soon, because people in companies behave like this everywhere. It’s almost like AI is making people even dumber 🤔
-
@iVeryAm@mastodon.social 2026-04-27 22:33
@scalzi Claude said it was Russ Hanneman and his Tres Comas bottle.
-
@AvacadoAvenger@mastodon.social 2026-04-27 23:19
@scalzi Could have been worse. It could have been an authentication of a nuclear launch code. Not being sure what to do it could have allowed a launch order to proceed.😂
-
@AngelaScholder@mastodon.energy 2026-04-27 23:47
@scalzi Awww....
-
@pascaline@mastodon.nl 2026-04-27 23:49
@scalzi I have no shame: I enjoy many stories like this 😹
-
@Mikal@sfba.social 2026-04-28 00:48
@scalzi We need a word that combines schadenfreude and AI (with a corresponding emoji) to show our sentiment on these somber occasions.
-
@sharpcheddargoblin@reclusive.blog 2026-04-28 01:09
@scalzi Operating exactly as engineered. Stop trusting stochastic token extruders.
-
@alterelefant@mastodontech.de 2026-04-28 05:08
@scalzi If the objective was to 'save disk space' then Claude just did what it was told to do. Reclaim disk space.
-
@toolsontech@pkm.social 2026-04-28 06:08
@scalzi@mastodon.social Reasons why I don't fear for my job. The AI can do a lot, but it's pretty bad in telling end users, no this is stupid and here's why. And I have seen end users do stupid things for decades in every platform they got.
-
@Mastokarl@mastodon.social 2026-04-28 06:10
@scalzi@mastodon.social The fun bit here is that this is not Anthropic‘s fault for supplying an LLM that sometimes messes up things. We all do. It‘s Anthropic‘s fault for supplying an MCP protocol that is one big security leak. When it was new I posted here that a „tool usage for AI“ protocol that doesn‘t force the tool provider to provide, for every API, whether it is safe (only reads) or dangerous (changes stuff) is madness. But the protocol is simple and easy to implement, so the world is going for it.
-
@bovaz@misskey.social 2026-04-28 07:56
@scalzi@mastodon.social if a human had done a fraction of the damage that AI has done, they would be out of a job, and sued into homelessness. And it's remarkably unlikely that a human fucked up so bad, because there used to be processes, and checks, to avoid messing up production environments.