How a sentence becomes an answer: an LLM end to end

You can read a hundred explanations of how large language models work and still not understand how they work. Each one takes a single piece, attention, tokens, embeddings, temperature, and explains it well, in isolation, and then stops. What almost nobody does is follow one input all the way through: from the moment you press enter to the moment the first word comes back, in order, with each stage handing off to the next. That end-to-end path is where understanding actually lives, because the pieces only make sense as a chain.

So this is the whole machine, walked start to finish, on one concrete example. You type "The capital of France is" and hit enter. Here is everything that happens between that keystroke and the word "Paris" appearing, every stage, in sequence, no piece left floating.

Stage 1: your sentence is broken into tokens

The model does not see your sentence as words, or as letters. Before anything else, the text is chopped into tokens, the chunks the model actually operates on, drawn from a fixed vocabulary of tens of thousands of pieces.

For our sentence, the split might be: The · capital · of · France · is. Five tokens, roughly one per word here, though common words are usually a single token while rarer ones fracture into pieces ("tokenization" becomes "token" + "ization"), and the leading spaces are part of the tokens themselves. Each token is then looked up in the vocabulary and replaced by its ID number, an integer. After this stage your simple sentence is a short list of integers, something like [464, 3139, 286, 6890, 318]. That is what the model receives. Everything downstream operates on numbers; the words are already gone.

This is why models measure length in tokens rather than words, why unusual text costs more of the context window, and why a model can stumble on spelling, it never saw the letters, only the token.

Stage 2: each token becomes a vector of meaning

An integer ID carries no meaning, token 6890 isn't "more" than token 286. So the next stage converts each token ID into an embedding: a long list of numbers (a vector, often thousands of values) that represents the token's meaning as a position in space.

This is the step where meaning enters. Embeddings are arranged so that related tokens sit near each other, "France" lands near "Spain" and "Italy," far from "banana", and the geometry encodes relationships the model learned during training. Our five token IDs become five vectors, each a point in a high-dimensional meaning-space. The sentence is now a sequence of five meaning-rich vectors instead of five bare integers.

But there's a problem the embeddings alone don't solve: order. "France is the capital of" uses the same tokens as our sentence but means something broken. So alongside each embedding the model adds a positional encoding, a signal marking where in the sequence each token sits, so position 1 is distinguishable from position 4. Now each vector carries both what the token means and where it is. The input is finally ready for the engine.

Stage 3: attention lets every token read every other

Here is the heart of the machine, the mechanism that made modern language models possible: attention. Up to now each token's vector was independent. Attention is where they talk to each other.

At each attention step, every token looks at every other token in the sequence and decides how much each one matters for understanding it, then pulls in information from the ones that matter most. When the model processes "is," attention lets it look back at "capital" and "France" and recognise that this "is" is about to state a fact linking a country to a city. The token "France" enriches the representation of the whole sentence, because attention carried its meaning to where it was needed.

This all-to-all reading is what lets a model handle long-range dependencies, connecting a pronoun to a noun twenty words earlier, or holding a thread across a paragraph. attention does this in parallel across all tokens at once, rather than reading left-to-right one word at a time. That parallelism is why these models could be trained at the scale that unlocked their abilities; it's the reason the architecture is called a transformer, and the reason the 2017 paper that introduced it was titled "Attention Is All You Need."

Stage 4: the stack refines the meaning, layer by layer

Attention isn't done once. It's stacked, dozens of layers, each containing an attention step and a small feed-forward neural network, each taking the previous layer's output and refining it further.

The useful way to picture the stack is as escalating abstraction. Early layers capture surface patterns, grammar, which words go together. Middle layers assemble phrases and local meaning. Later layers work with something closer to concepts and intent: by the top of the stack, the representation of our sentence "knows" it is a geography statement awaiting a capital city. Each layer hands its refined vectors up to the next, and after passing through the whole stack, the representation at the final position, sitting under "is", has been transformed from a bare token into a rich encoding of everything the model needs to predict what comes next. Nothing new has entered from outside; the model has simply thought harder, layer by layer, about the input it was given.

Stage 5: the model produces a probability for every possible next token

At the top of the stack, the model takes that final refined vector and does one specific thing: it produces a score for every single token in its vocabulary, tens of thousands of numbers, one per possible next token. These raw scores are called logits.

A high logit means "this token is a likely continuation." For our sentence, the token Paris gets a very high score; London and Madrid get moderate scores (they're capitals, plausible but wrong); banana gets a very low one. These raw scores are then passed through a function (softmax) that squashes them into clean probabilities that sum to 1, turning "Paris scored highest" into "Paris: 91%, London: 3%, Madrid: 2%, …". The model has now expressed, as a full probability distribution, its belief about what comes next. This is the one and only thing a language model fundamentally does: predict a probability distribution over the next token. Everything else, the tokenizing, the embeddings, the attention, the stack, exists to make this one prediction good.

Stage 6: one token is chosen, and this is where you have control

The model has a probability distribution. Now a single token must be picked from it, and how it's picked is sampling, the stage where the settings you may have heard of actually act.

If the model always took the highest-probability token. It would be deterministic and often flat and repetitive. So instead it samples from the distribution, and temperature controls how boldly. Low temperature sharpens the distribution toward the top choice: for our factual sentence, that's what you want, so "Paris" wins nearly every time. High temperature flattens the distribution so lower-ranked tokens get a real chance: useful for creative writing, dangerous for facts. This is the knob behind "make it more creative", it's not a mood, it's a reshaping of this probability distribution before the pick. For "The capital of France is," any sensible setting picks Paris.

One token is now chosen. Just one.

To make that concrete, picture the actual distribution the model produced for "The capital of France is," as rough probabilities: Paris 91%, London 3%, Madrid 2%, the 1%, and a long tail of thousands of tokens splitting the remaining 3% into slivers. At a normal temperature, sampling from this picks "Paris" almost every time, the distribution is so peaked that even a random draw lands there. Now raise the temperature: the distribution flattens toward Paris 60%, London 12%, Madrid 9%, and the tail fattens, suddenly a wrong-but-plausible capital has a real chance of being drawn, which is why high temperature is wrong for facts. Lower the temperature toward zero and it sharpens to Paris 99.9%, effectively deterministic. Same model, same forward pass, same logits, the only thing that moved was how the distribution was shaped before the draw. That is the entire mechanism behind "make it more precise" versus "make it more creative."

Stage 7: the whole thing repeats, one token at a time

The part that surprises people. The model did all of that, tokenize, embed, attention through the whole stack, predict, sample, to produce a single token. To generate a full answer. It does the entire thing again, and again, once per token.

The chosen token "Paris" is appended to the input, and the new, longer sequence ("The capital of France is Paris") is fed back through the model to predict the next token, maybe a period, maybe "," if a longer sentence is forming. Then that token is appended and the whole sequence runs through again. This is autoregression: generation is a loop, each pass producing one token that becomes part of the input for the next pass. A hundred-word answer is a hundred trips through the stack.

This is why longer responses take longer in a way you can watch, the words appear one at a time because they're computed one at a time. It's also where an optimization called the KV cache earns its keep: rather than recompute attention for the whole growing sequence every pass, the model caches the earlier tokens' attention data and only computes the new one, which is what makes token-by-token generation fast enough to feel like typing rather than a slideshow.

The loop stops when the model samples a special end-of-sequence token, or hits a length limit. Then the tokens produced are converted back into text, and that text is what appears on your screen.

Where the simple picture gets more complicated

The seven-stage chain is true and complete for a basic model generating text, but three things complicate it in the systems you actually use, and knowing where the simple picture ends is part of understanding it.

The model was shaped before you ever typed. The pipeline above describes a trained model in action, but the behaviour you experience, helpfulness, refusals, tone, was installed in a stage that happens long before your prompt: after the base model learns next-token prediction on a vast corpus, it's further tuned on curated examples and human feedback (RLHF and its relatives) to prefer responses people want. So when the model assigns a high probability to a helpful, well-formatted answer rather than a plausible-but-unhelpful continuation, that's the fingerprint of training shaping the very probability distribution in stage 5. The pipeline is the same; the values baked into the weights are what post-training set.

"Thinking" models loop before they answer. The autoregressive loop in stage 7 is exactly how reasoning models work too, but they're trained to first generate a long stretch of intermediate "thinking" tokens, working through a problem step by step, before generating the final answer. Mechanically it's the same next-token loop; the difference is that the model learned to spend tokens reasoning out loud before committing to a response. When you see a model "think" before answering, it's running stages 1–7 to produce that reasoning, then continuing the same loop to produce the answer that follows from it.

Nothing here retrieves facts. The pipeline shows the model predicting "Paris" because that continuation was overwhelmingly represented in training, the fact lives, fuzzily, in the weights. This is exactly why models hallucinate: asked something the weights don't reliably encode, the machine still produces a confident probability distribution and still samples a fluent token, because predicting a plausible next token is the only thing it does. It has no separate "do I actually know this?" check. That limitation is the entire reason for retrieval systems, which insert real documents into the input at stage 1 so the model predicts from supplied text rather than fuzzy memory, but that's a system built around the pipeline, not a change to it.

The short version

Step back and the chain is clean. Your sentence is tokenized into integer IDs; each ID becomes a meaning-rich embedding with a positional marker; attention, stacked across layers, lets every token read every other and refines the meaning from grammar up to intent; the top of the stack emits a probability over every possible next token; sampling (shaped by temperature) picks one; and then the whole loop repeats, one token at a time, until the answer is complete.

Every isolated explanation you've read is a single link in that chain. Attention is stage 3. Temperature is stage 6. Tokens are stage 1. They confuse people in isolation because a link only makes sense as part of the chain, attention is pointless until you know it's refining embeddings on their way to a next-token prediction; temperature is meaningless until you know there's a probability distribution for it to reshape. Seen end to end, the mystery dissolves into something almost mechanical: a machine that turns text into numbers, thinks about those numbers in parallel through a deep stack, predicts one next token, and loops.

That "loops one token at a time" ending is worth sitting with, because it reframes what these systems are. A large language model is not retrieving answers, and it is not reasoning in some hidden place before speaking. It is, at its base, an extraordinarily sophisticated next-token predictor, run in a loop, and the fact that this alone produces fluent, useful, often startling responses is the surprising thing about the whole enterprise. Understanding the pipeline doesn't make it less remarkable. It makes it remarkable in the right place.

Common questions

What actually happens when you send a prompt to an LLM? Your text is broken into tokens (small chunks turned into ID numbers), each token becomes an embedding (a vector of meaning) with a position marker, and the sequence passes through a stack of attention layers where every token reads every other and the meaning is refined. The model then produces a probability for every possible next token, samples one, appends it, and repeats the whole process token by token until the answer is complete.

Do large language models generate a whole answer at once? No. They generate one token at a time. Each token is predicted, chosen, and appended to the input, and then the entire longer sequence is run through the model again to predict the next token. This loop (autoregression) is why longer answers take longer and why you can watch responses appear word by word.

What is a token in an LLM? A token is the unit a language model actually processes, a chunk of text from a fixed vocabulary, often a whole common word or a piece of a rarer one. Your text is split into tokens and each is converted to an ID number before the model does anything else. Models measure length and cost in tokens, not words or characters.

What does attention do, in plain terms? Attention lets every token in the sequence look at every other token and pull in the information that matters for understanding it. It's how the model connects related words across a sentence or paragraph, linking a pronoun to the noun it refers to, or "is" to the "France" it's about to describe, and doing this in parallel is what made modern language models trainable at scale.

Where does temperature fit in? Right at the end, at the sampling stage. After the model produces a probability distribution over possible next tokens, temperature reshapes that distribution before one is picked: low temperature sharpens it toward the single most likely token (good for facts), high temperature flattens it so less likely tokens get a chance (good for creativity). It's not a mood setting, it's a mathematical reshaping of the next-token probabilities.

Is an LLM just predicting the next word? At its core, yes, a language model fundamentally predicts a probability distribution over the next token, then samples one, in a loop. Everything else (tokenization, embeddings, attention, the deep stack) exists to make that single prediction good. The surprising part is that next-token prediction, done well enough and run in a loop, produces the fluent and useful behaviour we see.

How does an LLM know when to stop generating? The model has no fixed answer length; it keeps generating one token at a time until it produces a special end-of-sequence token that signals the response is complete, which it learned to emit during training at natural stopping points. The system also enforces limits: a maximum token count caps the length regardless, and stop sequences can tell it to halt when certain text appears. So stopping is partly the model's learned judgment about where an answer naturally ends and partly external constraints imposed by the application, working together.