Elektrine lite

← Feed

@NathanHarvey@kolektiva.social

2024-05-21 18:50 UTC

@walruslifestyle@octodon.social @dpiponi @gregeganSF @BoydStephenSmithJr Although I stopped following Wolfram long ago due to issues I saw in his mathematical and popular work, I will accept he may have described AI like you state. It is wrong, though. What you describe is an n-point autocomplete. That process is not a trained model process, but a simple statistical analysis on large text collections. It only loses information by summarizing statistical moments of data and used as a statistical generation process word for word. There is no training on the process of generation, just the initial distribution analysis, and generation is known to lose semantic reference quickly. Often, syntactically correct sentences may be formed that are purely Chomskian (lack any semantic reference). The randomness of modern LLMs is very different. Modern transformer architectures generate the response deterministically and the entire response is determined by the input. However, to add variability and explore potential responses, a part of the input to the models is trained to be a random indicator of specific language-producing states of variation. This will vary depending on the training, but for instance, a model trained to explore multiple emotional states may use this random part of the input to indicate emotional state. Models trained to deliver “technical” or “objective voice” responses may train different styles of explanation into this random input. Typically, there are many dimensions of randomness expressed in the random input section, and the models use it to identify aspects of variability that do not tend to affect training selection. It is because the model output is generated entirely from the input, from first token to terminator, that allows the LLMs to have a semantic stability far beyond anything possible with an autocomplete statistical approach.

Replies (0)

No replies.