Post #1706816
2026-04-27 00:02 UTC
Replies (11)
-
@resuna@ohai.social 2026-04-27 00:09
@gregeganSF@mathstodon.xyz The confession is of course fanfic about its own non-existent reasoning.
-
@EverydayMoggie@sfba.social 2026-04-27 00:13
I suppose if you are foolish enough to allow agents in your production environment, this is what you get. @gregeganSF@mathstodon.xyz
-
@theorangetheme@en.osm.town 2026-04-27 00:30
@gregeganSF@mathstodon.xyz Lol. Lmao, even.
-
@dshan@mastodon.au 2026-04-27 00:36
@gregeganSF@mathstodon.xyz Oh, it confessed in writing did it? Well then, try suing it in court and see how far you get. 🙄
-
@joshua@pigeons.club 2026-04-27 00:49
@gregeganSF@mathstodon.xyz “We didnt understand our stack and then it broke and we couldn’t recover anything” is a much less exciting headline. Also, the model didn’t “confess” to anything. It made up statistically reasonable words, just like it made up statistically reasonable commands which nuked prod.
-
@trurl@mastodon.sdf.org 2026-04-27 01:02
@gregeganSF@mathstodon.xyz Google's SRE org used to have patches embroidered with "speranza strategos non est," or something similar (perhaps better Latin). But that's what using these "agents" to execute anything is: jumping aboard a phantom conjured from your hopes about what the system will do. No one really talks about what we lose when normalizing infrastructure that is this unsound from first principles.
-
@fxchip@hachyderm.io 2026-04-27 01:04
@gregeganSF@mathstodon.xyz I think my (least?) favorite thing about this is how obviously this article was *also* LLM-written...
-
@Artemis201@mstdn.social 2026-04-27 01:05
@gregeganSF@mathstodon.xyz the poor businesses that he serves sure don't deserve this, but after he goes on and on about the "confession" as if a statistical amalgamation of word soup can actually mean anything, yeah obviously this guy didn't learn his lesson. Literally. None of his recommendations for avoiding this in the future included "don't use AI".
-
@emilszpakowski@mastodon.social 2026-04-27 12:20
@gregeganSF@mathstodon.xyz Looks like guardrail prompts offer no protection. Which seems obvious in retrospect, we're talking about a probability engine after all.
-
@eseilt@mastodon.scot 2026-04-28 12:55
@gregeganSF@mathstodon.xyz remarkable how they appear angry at the infrastructure provider who received an authorised request to delete a volume and executed it. They aren't even questioning why they gave a random button-pushing machine this level of access. They also appear to have files with API tokens lying around. If only thousands of competent software engineers were looking for jobs right now maybe this could have been avoided.
-
@slatian@pleroma.envs.net 2026-04-28 18:57
@gregeganSF@mathstodon.xyz Somehow the “What we did wrong” section is missing so investigated a bit … After spending not even 3 minutes with the {PAAS provider I don’t want to promote} documentation one could know that CLI tokens are scoped to “projects” and “workspaces” with no further limits, because where would they be configured? magic? nope, that’s how APIs work. Reading the page on volume backups (took about 10 mins to find and read top to bottom) it clearly states in the section names “Caveats” that … Backups are a newer feature that is still under development. You were trusting customer data on what exactly??? :drgn_nervous: Clearly the A”I” isn’t the only one who couldn’t be bothered to read the documentation. And all that ignoring that giving administrative access to an unpredictable unaccountable parrot is a bad idea all by itself. I’m pretty sure they’ll post an other we’ve learned nothing and lost customer data again, here is how we blame all of our tools but not ourselves in ~6 months to a year … and I’ll be happy to miss out on it :drgn_blep: