From logits to probabilities

Softmax turns the raw scores into a probability over the vocabulary.

The output projection hands you one logit per vocabulary token — a raw, unbounded score where higher means "more likely," but the numbers themselves have no fixed scale. Before anything can sample from them, they need to become a proper probability distribution: all non-negative, all summing to 1. That conversion is softmax.

Softmax exponentiates each logit and divides by the total, so p(token) = e^{logitᵢ} / Σⱼ e^{logitⱼ}. Exponentiating keeps everything positive and exaggerates gaps — a token a few points ahead in logit space ends up with a disproportionately larger share of the probability. The result is a distribution you can read directly as the model's confidence in each next token.

Only after this step do the decoding strategies apply: greedy takes the largest probability, while temperature, top-k and top-p reshape or trim the distribution first. Softmax is the shared bridge from "scores" to "choosable probabilities."

Related: Logits · Greedy · Temperature