Post #511308

2026-03-04 05:00 UTC

Man, AI agents are remarkably bad at “self-awareness” like this, I’ve used it to configure some networking on a Raspberry Pi, and found myself reminding it frequently, “hey buddy, maybe don’t lock us out of connecting to this thing over the network, I really don’t want to have to wipe the thing because it’s running a headless OS”. It’s a perfect example of the kind of thing that “walk or drive to wash your car?” captures. I need you to realize some non-explicit context and make some basic logical inferences before you can be even remotely trusted to do anything important without very close expert supervision, a degree of supervision that almost makes it totally worthless for that kind of task because the expert could just do it instead.

Replies (5)

@A_norny_mousse@piefed.zip 2026-03-04 11:59

AI agents are remarkably bad at “self-awareness” 🤔 what does it say when you tell it something like “look, this is wrong, and this is why, can you please fix that”? In a general sense, not going into technical aspects like what OOP is describing.

Open ##517054
@Earthman_Jim@lemmy.zip 2026-03-04 15:45

Hey, maybe if we’re lucky, Claude will accidentally lock the world out of using nukes forever.

Open ##520575
@Confused_Emus@lemmy.dbzer0.com 2026-03-04 19:34

AI agents are remarkably bad at “self-awareness” Because today’s “AIs” are glorified T9 predictive text machines. They don’t have “self-awareness.”

Open ##524129
@sudoer777@lemmy.ml 2026-03-05 00:27

For AI I think a lot of future improvements will be around making smaller more specialized models trained on datasets curated by people who actually know what their doing and have good practices as opposed to random garbage from GitHub (especially now with vibecoding being a thing, so training off of low quality programs that it created itself might make the model worse), considering that a lot of what it outputs is of similar garbage quality. And remote system configuration isn’t obscure so I do think this specific issue will be improved eventually. For truly obscure things LLMs will never be able to do that.

Open ##528198
@qjkxbmwvz@startrek.website 2026-03-04 19:24

“…I really don’t want to have to wipe the thing because it’s running a headless OS” I feel like logging in as root on a headless system and hoping you type the command(s) to restore functionality is a rite of passage.

Open ##1876714