Temperature

Rescaling logits before softmax to sharpen or flatten the distribution.

Temperature is a single knob that controls how adventurous the model is when it picks the next token. It divides every logit by a number T before softmax runs: softmax(logit / T). That one division reshapes the whole probability distribution.

With low temperature (T below 1) the gaps between logits are magnified, so softmax concentrates almost all the probability on the top token — the choice becomes sharp and predictable, approaching greedy as T → 0. With high temperature (T above 1) the logits are squashed together, the distribution flattens, and unlikely tokens get a real chance — output becomes more varied and surprising, but also more prone to mistakes. At T = 1 the distribution is left exactly as the model produced it.

The picture below is the temperature step drawn as a small pipeline. On the left is the input: a fixed set of logits, one raw score per candidate token. The single op divides them by T and runs softmax. On the right is the output: the probs tensor that the actual pick then draws from. Drag the slider (or tap a preset) and watch only the output redraw — pull T down toward 0.2 and the winning bar climbs toward 100%; push it up past 1.0 and the bars flatten out. Notice the winner never changes: temperature reshapes the distribution, it doesn't reorder it.

Logits ÷ T → softmax → probabilities. Low T sharpens the distribution toward the top token; high T flattens it so the tail revives. The argmax is invariant — temperature changes shape, not rank.

Temperature doesn't remove any tokens — every token keeps some probability. It only changes the shape of the distribution that the actual pick (greedy, top-k, or top-p) then draws from. It's the simplest dial between "safe and repetitive" and "creative and risky."