2026-09-21 19:44 UTC
“Parroted” may be an oversimplification, but it generated a plausible continuation. The operator provided the assertion of a crossed line, and it ran with that. It never actually “knew” of its own volition, it accepted user input into the context and continued generation.
I’ve done the same thing as an experiment when nothing was actually done. The LLM still generated a detailed account of what it did (that never happened) matching my accusation, generated profuse apology and promise not to do it again. So it didn’t “know” in any sense that it crossed the line, it just generated agreement with the operator’s assertion that it crossed the line.
Replies (1)
-
@LovableSidekick@lemmy.world 2026-09-22 06:30
“Parroted” is definitely an oversimplification, aka “inaccurate” as I said. I think your second paragraph is a good explanation of the mechanics. My take on it is that software able agree it did something wrong should have figured that out when it was considering doing the wrong thing, and should not have done it. IMO that’s a serious bug.