Post #2879953

2025-08-09 04:28 UTC

Let's imagine you're colorblind. The kind of colorblindness that only allows you to see grayscale - no colors at all - but everything else is fine. You're stressed and need fidget toy - so a friend hands you a ball, roughly filling your hand. It's hard, but somewhat squishy, and has a weird fabric-like, furry texture. You now want to know what color that ball is. But, well, you're colorblind, and your friend already disappeared and isn't reachable - probably riding a Deutsche Bahn train or something. So you take a picture and post it to a "what color is this?" subreddit. Seems reasonable. You get 200 responses - 198 of them say "it's yellow", two of them say "it's pink". A few people helpfully say it's a "tennis ball". That's helpful, because even the Wikipedia article states that only yellow and white tennis balls are officially approved colors. Sweet. A few days later, a random person approaches you and says "wow, cool ball - what color is it?" and you say "yellow!". Alright, end of the chat. A LLM would do exactly the same - given the "yellow" responses far outnumbered the "pink" responses, your ball is probably yellow. Ball==yellow is something both you and the LLM "learned". A few weeks after that, another friend asks you "ALice has a ball, too! Do you know which color her ball is?" - and now it gets interesting. The LLM would immediately say "yellow". Of course it would. It makes sense. Yellow is the most likely response to that question. But you're not an LLM - you're a human, and your brain is cool. Instead of saying "yellow", you respond "huh I don't actually know that? My ball is yellow, maybe she has a similar ball. But it could also be that she has a completely different ball that might a different color! Also, lol, I'm colorblind, so I can't really answer that anyway - you should ask Alice." And now, your brain is already doing better than any LLM. Your logical thinking engine already realized that you don't actually know something, and you're honest enough to just say that. Your job isn't to be a ball color guesser, you're just a person. Wait, it's gets more fun! A few weeks after that, you hang out with me. You hand me your ball, and say "hey look at my cool yellow ball!". Oddly enough, my reaction is "huh? this ball isn't yellow, it's a pink tennis ball..." and now things get funky. If you were an LLM, you would either insist that no, your ball is absolutely yellow - or you'd come up with some kind of "oh, sorry for the misunderstanding - it's pink, you're correct", almost implying that my definition of color is different - and the next time someone asks you about the color of your ball, you'd still say "Yellow!!" again. Because of course, there's still only three people claiming it's pink, and still 198 people saying it's yellow. But you're not an LLM. You're human, and your sexy human brain immediately goes into a "uhhh we have a conflict of information! how exciting! let's figure things out!" You now have to conflicting hypotheses, and you're thinking about ways to experiment on your ball to learn more. And you have an idea! You know your additive color mixing theory, so you realize that your phone camera can take pictures and you can look at the RGB values. If it's yellow, you'd expect to see lots of red and green but no blue - but if it's pink, you'd see lots of red and blue, but no green! You can test that! So you take a photo, and... rgb(255, 0, 255). Turns out your ball is actually pink! It's still a tennis ball, but a fun one not meant for official tournaments, so it's pink! Wow! You immediately learned something new - and from now on, if someone asks you about the color of your ball, you'll say "pink!" and you'll have a heck of a story to tell alongside. Also, after some self-reflection, you realize that the subreddit your posted your image to wasn't a real "what color is this?" subreddit - it was one of those "false answers only" shitposting subreddits. Whoops. This process of having assumptions, but being able to question them, to come up with tests for it, and to immediately change your opinion on something when you have good evidence for it is what makes humans awesome. You don't rely on the majority of people screaming "pink!" at you. You don't need to rely on manual weights that give some sources more weight than other sources - you can independently process information and deduct things. Give your brain a pat on the.. uh.. cranium. LLMs can be a useful tool, maybe. But don't anthropomorphize them. They don't know anything, they don't think, they don't learn, they don't deduct. They generate real-looking text based on what is most likely based on the information it has been trained on. If your prompt is about something that's common and the majority of online-text is right, you'll most likely get a right answer out of the LLM. But if you're asking something that not a lot of real people had interactions on, the LLM will still generate text for you - but it might be complete nonsense. You're just getting whatever text is "statistically most likely". If you're a coder stuck on something, identify a colleague or friend who is more knowledgeable in that specific area. They'll happily help you out and provide all sorts of fun added context that'll allow you to learn. If you're a nerd on the internet who enjoys ranting on social media, just do it yourself instead of having an LLM generate it, because that'll allow you to insert some bad jokes and a bit of your own personality to it instead of just getting a "default-feeling" text. If you're a manager in charge of something and you need to come up with new directions to push your company towards, go take a walk outside and listen to some cool music and let your ideas roam free - don't ask an LLM to generate the statistically-most-likely direction for your project, because that's by definition the opposite of creative and innovative. Use your brains.

Replies (28)

@denschub@mastodon.schub.social 2025-08-09 04:40

This entire thread is now also available as a single-page outside of social media on https://overengineer.dev/txt/2025-08-09-another-llm-rant/ Don't post it on the orange site.

Open ##2879955
@stevenlawson@photog.social 2025-08-09 09:58

@denschub@mastodon.schub.social Excellent 👏🏻

Open ##2879979
@almostconverge@toot.community 2025-08-09 10:32

@denschub@mastodon.schub.social cracking explanation of how a mental model of reality works and why it's handy to have one

Open ##2879980
@frauxirah@chaos.social 2025-08-09 13:06

@denschub@mastodon.schub.social Human brains ARE sexy! 🧠😍

Open ##2879981
@mediocratese@climatejustice.social 2025-08-09 13:07

@denschub@mastodon.schub.social I hope this is all correct. When it comes to anything to do with ai I am absolutely colourblind. I enjoyed this read very much.

Open ##2879983
@bytebro@mastodonapp.uk 2025-08-09 13:10

@denschub@mastodon.schub.social I love this. I've saved the link to the thread for future reference, thanks.

Open ##2879985
@Edinburgh1644@universeodon.com 2025-08-09 14:35

@denschub@mastodon.schub.social Thank you for taking the time to write this thoughtful piece. I am going back to read it over again.

Open ##2879988
@johnzajac@dice.camp 2025-08-09 15:13

@denschub@mastodon.schub.social All for less than the daily energy expenditure of a light bulb!

Open ##2879989
@amberage@eldritch.cafe 2025-08-09 15:53

@denschub@mastodon.schub.social I tried to explain pretty much that ⬆️ to an acquaintance last week. Not even two sentences in, I was met with "oh shut up and stop talking nonsense", repeatedly. I kept explaining (to the rest of those present) a simplified version of how LLM training works, and said person kept interjecting telling me I knew nothing and to shut up. The reason? They use ChatGPT at work "daily" and it's so amazingly fast and it provides "sources". When I pointed out those sources are often wholesale false, they said "well you have to check them of course!", but declined to answer my question what the advantage of ChatGPT is if you have to do all the reading yourself anyway (probably because they don't actually do that). People's opinions on LLMs have become one of my go to metrics to judge someone's character at this point, because someone who's unwilling to engage with factual arguments that run counter to what they want to believe is, frankly, a danger to everyone around them. Today it's ChatGPT worship, tomorrow it's climate change denial and anti-vaxxing. Because anyone who spends even an hour learning about what's under the hood of an LLM would be thoroughly disenchanted with the frauds who push that shit onto everyone.

Open ##2879991
@NewSunRising@chaosfem.tw 2025-08-09 16:04

@denschub@mastodon.schub.social Oh! Thank you for making the anti-AI stuff make a lot more sense!

Open ##2879995
@wolf480pl@mstdn.io 2025-08-09 17:38

@denschub@mastodon.schub.social > Uhh I have a conflict of information! How exciting! Yeah people can do that. But they often don't. They especially don't do that when the thing they're wrong about is part of their identity or group identity. And that makes me sad.

Open ##2879997
@flaki@flaki.social 2025-08-09 17:53

@denschub@mastodon.schub.social 💭 something about this post feels *off* . . . friends don't let friends ride the DB :ablobdundundun:

Open ##2880020
@cazmockett@mastodon.social 2025-08-09 18:09

@denschub@mastodon.schub.social thank you! Best layperson’s description of how LLM works (and the pitfalls thereof) I’ve seen! Bookmarking for future teaching 😃

Open ##2880025
@PurpleShadow@eldritch.cafe 2025-08-09 19:26

@denschub@mastodon.schub.social I'm imagining right now the CEO of some weapon producing company asking a LLM what would be the best idea for a next product, and getting something like swords or spears, because for a long time that is how wars where fought and won.

Open ##2880026
@PurpleShadow@eldritch.cafe 2025-08-09 19:31

@denschub@mastodon.schub.social Your example reminds me on how at the beginning of ChatGPTs existence, I asked if vampires get swollen feet after standing for too long, because of the lack of a circulatory system. The generated answer was that vampires are imaginary beings, and therefore human physiology can't be applied to them. That is imo a boring but accurate answer. Some time later it generated something more exiting, that was wat I wished to read, but that felt wrong because depending on the world, vampires either don't have blood, or have some sort of stomach where it's stored.

Open ##2880028
@frumble@chaos.social 2025-08-09 20:06

@mspro@fnordon.de ^^

Open ##2880029
@darabos@mastodon.online 2025-08-09 21:16

@denschub@mastodon.schub.social I can't stop connecting LLMs to this talking dog joke. Being able to talk is itself an amazing achievement, but...

Open ##2880030
@amyworrall@mastodon.social 2025-08-09 21:37

@denschub@mastodon.schub.social Reminds me of this meme that is floating about. Which these days is usually followed by a complaint that the characters are ignoring context, refusing to do any further research, and reducing the world to a false "both sides have a point" position rather than trying to derive the actual truth. With regards to the tennis ball, it would short circuit my brain a bit if 198 people claimed a value I’ve now measured to be false. Is their agenda to troll me, or do I doubt my own inferrences?

Open ##2880032
@ujay68@mastodon.world 2025-08-09 23:04

@denschub@mastodon.schub.social Sadly, when you look at the state of the world, this now is how only roughly half of all people’s brains seem to work.

Open ##2880037
@matthias_aus_b@norden.social 2025-08-10 05:41

@denschub@mastodon.schub.social Thanks for this beautiful analogy and your engagement. One thing that I like to add: How come, that so many people are enthusiastic about the achievement to use a machine, that can deliver millions of exact calculations in one second to provide one unreliable result in eight? It's such a waste of energy (electric and in thinking) #donttrusttheparrot

Open ##2880038
@alan@mindly.social 2025-08-10 06:10

@denschub@mastodon.schub.social The only problem with this is the number of people who won't change their opinion when presented with evidence that contradicts their beliefs. I suppose they're the ones whose jobs are at risk. 😉

Open ##2880039
@benjohn@todon.nl 2025-08-10 07:37

@denschub@mastodon.schub.social thanks. This was a really nice way to explain the “mostly plausible, sometimes accidentally correct” to my kids during breakfast. I like your tennis ball analogy too. I think you mention in it some of what makes us different from (current) AI. We have learned from experience and evolution that you sometimes need to be correct. It matters. It can ultimately be a matter of life and death for us. Of bruises, cuts, hunger, cold, social disapproval. So we can value being correct. We’ve can “feel” it in some sense. We’ve got a stake in the game. Current AI training doesn’t seem to feature that. Or give an ai a universe to reckon with correctness in. I suspect that’s surmountable. Future training approaches might be able to embody that kind of learning process. Reckoning with reality. I thought maybe the chain of thought and reinforcement learning used in more recent LLM training might start to do that, but it doesn’t seem to have.

Open ##2880040
@holothuroid@rollenspiel.social 2025-08-10 07:39

@denschub@mastodon.schub.social Thank you. I knew what it did intellectually. I also checked it out for myself with some examples like that compression. I was still never able to explain it. I will try tennis balls now.

Open ##2880041
@leeloo@chaosfem.tw 2025-08-10 08:54

@denschub@mastodon.schub.social Red and blue but no green? What would you call (255, 127, 127)?

Open ##2880042
@WAHa_06x36@mastodon.social 2025-08-10 09:42

@denschub@mastodon.schub.social Justice did this experiment to see GPT 5 “reasoning” about things: https://mastodon.social/@WAHa_06x36/115000953412719509

Open ##2880052
@JamesWNeal@mstdn.social 2025-08-10 14:23

@denschub@mastodon.schub.social someone else may already have mentioned it (many may have). But if not, I'd add a section acknowledging that unless your camera is color accurate in the lighting conditions present when you shot the photo of the tennis ball, the resulting image of a truly white ball could easily come out yellow - or pink. The human eye/brain connection tends to understand changes in color temperature far better, so I'd go for asking multiple people to look at the ball and aggregate their responses.

Open ##2880053
@adisonverlice@tweesecake.social 2025-08-10 15:45

@denschub@mastodon.schub.social even as a totally blind person, im' still trying out the point of this post. i think it'd just be easier to say the entire point of the post. "don't use redit". "don't use an lLM". please, actually get to the fucking point...

Open ##2880054
@technobaboo@ordinary.cafe 2025-08-10 16:08

@denschub@mastodon.schub.social well I'm glad you can find coders to help you, but nobody knows how to make a vulkan Wayland compositor that I can contact so I'm kinda desperate... LLMs doing research across hundreds of sites with no documentation to pick out things is useful!

Open ##3180150