Elektrine lite

← Feed

@NathanHarvey@kolektiva.social

2024-05-21 16:53 UTC

@emilymbender Dr. Bender, I fear you have been spreading a lot of misinformation about these models and what they do, and your views are showing their errors here. LLMs don’t model languages. They aren’t string manipulators in the sense you imply. LLMs model the “language producing processes” they are trained on. Training is semantic. Every researcher making claims about the meaning produced by LLMs should be very clear about why these models are able to produce output with such semantic stability, and how their responses are semantically relevant to the input. The reason is because the training process constrains the models semantically, not just syntactically. A stochastic parrot producing autocomplete through statistical properties of language never stays semantically stable. We’ve known that for years, and it’s why syntactic techniques like n-point moment analysis is not pursued for LLMs. The “language producing process” is a mental one, which is why we also see other parts of the mental process in the outputs of these models. Sometimes they are silly, sometimes sad, and their words and phrasing change in the common ways we know from human interaction. There is a growing industry to get these to model other states, like flirtatious for lonely people, or caring and therapeutic for psychotherapy. They are not silly or sad or flirtatious, but they model those states. And doing this kind of testing is a way to measure how well these models are capturing the mental states we see in human communication. We are seeing greater ability to model a lot of subtleties in the language producing process, and will likely be seeing growing use of communicative exchanges (psych tests and such) looking to understand the strength of these models in various areas moving forward. Note also, this is the same as when we use these large token-string models for other data interpretations, like time series analysis. For example, when trained on weather data sequence data, these models get better and better at modeling the “weather producing process”. It’s easy to see that training is semantic, as it is trained on “true data” and selected when it produces “correct” results (with truth being semantic, of course). We don’t have as much aversion to thinking of those as actual “weather models” because the processes are external and less emotion-laden than the mental models of the LLMs.

Replies (0)

No replies.