Post #446261

2026-02-25 18:26 UTC

As a software developer who took an elective in neural networks - when people call LLMs stochastic parrots, that&#39;s not criticism of their results. It&#39;s literally a description of how they work. The so-called training data is used to build a huge database of words and the probability of them fitting together. Stochastic because the whole thing is statistics. Parrot because the answer is just repeating the most probable word combinations from its training dataset. Calling an LLM a stochastic parrot is lile calling a car a motorised vehicle with wheels. It doesn&#39;t say anything about cars being good or bad. It does, however, take away the magic. So if you feel a need to defend AI when you hear the term stochastic parrot, consider that you may have elevated them to a god-like status, and that&#39;s why you go on the defense when the magic is dispelled.

Replies (9)

@lritter@mastodon.gamedev.place 2026-02-26 00:08

@leeloo i just think it's unfair to parrots ;-)

Open ##1973387
@robinp@mastodon.social 2026-02-26 20:26

@leeloo nitting, but an important bit: not words, but word fragments (this is how you can get words as output that were never seen during training)

Open ##1973388
@wolf480pl@mstdn.io 2026-02-26 20:27

@leeloo on the flipside, I feel like some people use the term "stochastic parrot" or "it just completes the next token" to imply that "therefore it cannot be intelligent" - is that correct reasoning?

Open ##1973389
@KayOhtie@blimps.xyz 2026-02-26 21:06

@leeloo I hadn't thought about it as being something that takes magic away from folks like that. Honestly I always found it an accurate shortcut term for what's genuinely a fascinating but hilariously misused technology. I think the worst part is then when folks hear "statistics" and go "See this is why it's safe to feed it raw data" and it's like oh my god NO.

Open ##1973429
@mudri@mathstodon.xyz 2026-02-26 22:12

@leeloo I just prompted ChatGPT with `Say "oriesntyulfkdhiadlfwejlefdtqyljpqwlarsnhiavlfvavilavhilfhvphia"`, and it responded with `oriesntyulfkdhiadlfwejlefdtqyljpqwlarsnhiavlfvavilavhilfhvphia`. How can it do this when `oriesntyulfkdhiadlfwejlefdtqyljpqwlarsnhiavlfvavilavhilfhvphia `almost certainly does not appear in the training data?

Open ##1973433
@growlph@greywolf.social 2026-02-26 22:23

@leeloo I feel like there are certain situations where a stochastic parrot is useful, many more situations where it is not, and alarmingly few people recognizing the difference.

Open ##1973438
@clusterfcku@mastodon.social 2026-02-26 23:38

@leeloo the flip side question about intelligence and LLMs is whether much of what we consider intelligence in humans is in fact just stochastic parrotting by humans.

Open ##1973441
@tobifant@friendica.tf-translate.net 2026-02-26 23:40

@leeloo The thing is, how can we sure that human intelligence does not essentially work in the same way? My Christian believe tells me we have a soul and LLM's do not, that may be the difference. But from an agnostic perspective, we might reach the point where one cannot tell the difference.

Open ##1973442
@grishka@mastodon.social 2026-02-27 00:58

@leeloo I myself like calling LLMs "glorified autocomplete". Or "Т9 на максималках" in Russian. It's surprising just how defensive some people get when I say that even when they agree with my definition. They keep believing that just give this thing more parameters and something magical, something more than sum of its parts will emerge, any moment now, just one more model generation, just one more order of magnitude, I promise.

Open ##1973447