The token

A discrete unit — a snippet of text encoded as a single integer.

A token is the currency of the model. Text is not fed into the network directly; instead the tokenizer breaks text into small, overlapping pieces (words, subwords, characters — the strategy depends on the tokenizer), assigns each one a unique integer from 0 to the size of the vocabulary minus 1, and passes those ids into the model. Everything downstream — embedding, attention, generation — operates on these integers, never on text. One token at a time flows through the pipeline, gets projected to a probability distribution over all possible next tokens, and the sampler picks one. That becomes the next token fed in.

Why tokens instead of characters? Because a model that reads at the character level would need to burn layers learning that 'a', 'c', 'a', 't' spell "cat", whereas a tokenizer that recognizes "cat" as a single unit lets the model start with that knowledge already baked in. Tokens are a compression: they group meaningful substrings so the model sees longer context per layer and uses its parameters efficiently. They also have a cost — a token boundary in the middle of a word (e.g., "un" + "expected") means the model must learn to stitch pieces back together, and some token splits are less natural than others.

In the playground, each token is shown as a chip (or position in the residual rail) at the base of the diagram. As you press Generate, new token chips appear on the right, each one flowing up through embedding, attention, and feed-forward, and the sampler picks the next one based on the logits at the top. The sequence grows left to right as tokens accumulate; the model reads all of them to predict what comes next.

Related: The prompt · Tokenization · Embedding