Post #1494434

2026-01-21 16:39 UTC

@lucydev @KatS Autocorrect on steroids is basically GPT-3 tech. There's a lot more that goes into modern LLMs. A lot of the improvements are due to reinforcement learning, where LLMs learn to predict tokens that actually achieve some outcome, E.G. code that passes tests, answer that is judged "good" by a domain expert. There's still token prediction involved of course, but it somehow turns out that token prediction can get better scores than any human at (unseen) math olympiad questions. And people still say it's not in any way intelligent...

Replies (2)

@KatS@chaosfem.tw 2026-01-21 16:43

@miki @lucydev The last thing I think I can usefully add to this thread is that you sound very much like the kind of person Michael Crichton wrote about. I recommend watching Westworld some time - the movie, that is. I've never seen the series based on it.

Open ##1494435
@lucydev@wetdry.world 2026-01-21 16:54

@miki @KatS if i memorize every possible answer to a specific test, i can pass too. doesn't mean i know shit about fuck. There's no actual thinking or reasoning involved (and no, reasoning models don't actually "reason"), so yeah, an LLM isn't actually intelligent, it just shows how flawed our tests for intelligence are. To get some actual intelligence, thinking or reasoning involved, I'd reckon we'd have to fundamentally change something in the architecture of LLMs, and use a fuckton more computing resources for a single model, and considering how much energy the current tech already wastes, and the whole shtick that made LLMs (and more broadly generative AI) work in the first place is "we discovered that there comes a point where the output gets better when we throw rediculous amounts of compute resources on the problem", and it's already getting super difficult to run and maintain. Honestly, either you're unreasonably optimistic, or you've never taken a look at how things actually work under the hood, but I really recommend you to take a closer look at the technology you praise so much. A couple things you could take a look at (without an AI summarizer, otherwise you'd learn jack shit): Attention is all you need, which is the paper that sparked all that AI craze and the development of GPT models and The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity , which takes a closer look and tests reasoning models to infer strengths and weaknesses of reasoning models with all sorts of levels in problem complexity. Honestly, before you make any claims about where the tech could be and what it could do, you should have a look at how things actually work under the hood and have a rough idea of how things work, otherwise, no offense, you're just talking out of your arse.

Open ##1494436