The output token

The id chosen by the sampler — emitted as the next character and fed back as the next step's input.

Once the sampler has a probability distribution over the 27 vocabulary tokens, it picks one. The result is a single integer in the range 0–26: the output token id. That id is immediately converted back to a character by the tokenizer's inverse map (id → char) and appended to the visible output sequence. It is also fed back as the next input to the model — the whole forward pass (embedding lookup → attention → feed-forward → logits → sample) repeats, this time reading a sequence that is one token longer than before.

How the sampler chooses depends on the strategy in use. Greedy always picks the id with the highest logit, so the same prompt always produces the same output. Temperature sampling draws randomly from the softmax distribution, scaled by a temperature T; lower T concentrates probability on the top token, higher T flattens it. Top-k restricts the draw to the k highest-probability tokens; top-p restricts it to the smallest set whose cumulative probability exceeds p. All four strategies read the same 27 logits; only the selection rule differs.

The sample-pick figure shows the probability bars and highlights the chosen token. At near-LOD you can see the individual probability assigned to each character. When the same prompt is run twice with temperature > 0, a different token can be picked from the same bars — the model's output is the same, but the stochastic draw is different. This is the source of variation in language model outputs. The chosen token only becomes the final output after passing through the sampler; the logits themselves never directly name the result.

Related: Logits · Temperature · The generation loop