@mit_scharf@social.anoxinon.de @iain@kolektiva.social Yeah, LLMs don't work in words. They work in Tokens, which are substrings of text with a fairly consistent length. Which means that on top of not knowing facts, they don't even know how to put words together in sentences. They just know how to assemble tokens into things that look like sentences! The basic problem is that there's an infinite number of word-like things, but a finite number of tokens. So it can assign numbers to tokens and do the calculations with those.