Post #3147651

2026-04-07 00:58 UTC

To be more detailed: Words are converted into tokens, which are just numbers representing either a single word or part of a word. So like "posted" would get split into a token for "post" and a token for "ed" Then these are converted into word embeddings, and then that conversion is somewhat reversed by running it back through the embedding. This outputs a provability distribution basically saying "hey for that given set of nearby words, here's the likely starting words for that", which can possibly tie synonyms to each other instead of having them act entirely separate I think Then that gets pumped through a whole bunch of encoder-decoder layers, which are partially just "here's a normal neural network layer" and partially attention layers, which are somehow trained (no one ever mentions the technique used here hmmMMMM) to output a weighted set of numbers basically representing the relative "importance" of words in the prompt like for "the red apple" the word "apple" is probably most important I guess? It's far and away the most vague and handwavey part of the whole setup and seems to just boil down to "we use a bunch of neural network layers to overcomplicate things enough til the monkeys on typewriters give us good output" anyway then you stack a bunch of encoder-decoder layers on top of each other, which by the way the encoder part here is just a separate stack of layers that gets its output pumped into the input of every decoder layer to try to mitigate problems I think??? "It just works" is very much the running theme when it comes to asking why anyone is doing it this way instead of another way so get used to not having answers Anyway after we've fed our word embedding distribution salad of nonsense through enough monkeys on typewriters, we end up with our final product: a list of tokens and their probabilities This is then selected from completely at random (with a bias towards high probability results) to output the next token Then we rerun the entire thing again but with the new token as part of the input and keep doing this for each and every token until we get a specially designated stopper token as output, or we output the preset maximum number of tokens for a single input

Replies (2)

@norikawa@efdn.club 2026-04-07 01:08

you may notice several problems with this entire structure. don't worry, nobody who likes ai cares about any of them! the word embeddings are a major source of bias. basically, the entire math that they're based on presumes that the set of text data used for creating the embedding has an overall 100% perfectly balanced bias, that is, the data set has no pull or push towards or away certain viewpoints, writing styles.etc. also, the impact of a given bias scales with the amount of text containing it, and is entirely unrelated to the actual importance or authority of the source. so like, if say Alex Jones talks about something more than anyone, that topic in the embeddings will ties words together in a way disproportionately like him than anyone else. there's zero way to have any human in the loop on any of this at all. the attention mechanism is also rather odd. neat idea in theory maybe but given the size of the typical token window (we're talking highly advanced models don't go much over a dozen or so tokens!), it's entirely inadequate for any discussion longer than a single sentence or two. I also imagine it's even more susceptible to bias than the embeddings, as the only reasonable training methods I can conceive of are either human guided (which usually involves just hiring a company to do it for you, whose employees are all likely culturally similar!) or trying to do some manner of automated training likely using the embeddings to begin with (so therefore the attention mech will just exaggerate it bias even more!) the fact that each decoder layer has to have a correction applied via the encoder stacks final output seems like a good indication that something's wrong here, but don't put it past AI bros to not give a shit at all costs!! also the final output token being just randomly selected is just hilarious, since that token then feeds into the system for the next tokens. so you better hope to god it never randomly picks a bad one even once the whole time!

Open ##3147652
@alas@artemisia.systems 2026-04-07 01:00

@norikawa@efdn.club not to quibble inordinately but the openai special sauce was slicing off the encoder half and going "oh hey it still works"

Open ##3147660