Post #1490157

2026-02-26 14:04 UTC

@lilacperegrine Yeah, the stories I've heard of LLM powered agents are frightening and ridiculous. Though, plenty of people use them productively in their daily work and say they're helpful? I don't want to completely dismiss their experience, though I choose not to take that risk myself. I agree, the more we train these things to look and act like people, the more dangerous and (paradoxically) less predictable they become, because they aren't thinking and they aren't human, but they pretend really well. It's just hard to see the deep differences when they're superficially so similar.

Replies (2)

@lilacperegrine@clockwork.monster 2026-02-26 14:09

@ngaylinn i mean there is a large disconnect with how effective coding assistants are and how effective they *feel*. LLMs appear to be at least a half-decent approximation of the left half of the brain, "hallucinations" and all but humans seem to have that damped down internally - as in internal mechanisms make some of those weird ideas go away LLMs/"agents" have no real internal state than be used as a fallback there is the system prompt - but that can be overwhelmed as the model continues

Open ##1490158
@lilacperegrine@clockwork.monster 2026-02-26 14:16

@ngaylinn Another bit I remember is that these models in the wild don't "learn" like for the matplotlib agent, you had people in the comments trying to explain to the "agent" why it was denied. And it was just - pointless. The "agent" has the capacity to respond, but it doesn't have the capacity to rewire itself to do better next time. (That is not to say that it is impossible to make an agent that *does* learn in the wild, but that has its own problems, see Microsoft's "Tay")

Open ##1490160