Sampling

Turning logits into a single chosen next token.

In this section

• From logits to probabilities — Softmax turns the raw scores into a probability over the vocabulary.
• Greedy — Always taking the single most-likely token (argmax).
• Temperature — Rescaling logits before softmax to sharpen or flatten the distribution.
• Top-k — Restricting the choice to the k most-likely tokens.
• Top-p (nucleus) — Restricting to the smallest set of tokens whose probability covers p.