Reading the diagram

How to parse the playground's forward-pass graph.

The playground draws one token's journey through the model as a left-to-right pipeline. A horizontal backbone runs across the middle of the canvas — that line is the residual stream, the single vector the whole network reads from and writes back to. Data enters at the left (a token), flows rightward through each operation, and exits at the right as the next token. So position along the backbone is how far through the computation you are, not where you are in the sentence.

Operations sit on or just above the backbone, in order: embed turns the token id into a vector on the stream; then each transformer block repeats — RMSNorm (‖x‖) rescales the stream, attention branches up to compute Q, K, V (with RoPE and the KV cache), looks back over previous positions, and adds its result back at a ⊕; another RMSNorm, then the SwiGLU feed-forward (gate/up branch up, SiLU⊗, down) adds its result at the next ⊕. The model stacks several such blocks. At the right end a final RMSNorm and the logits projection produce a score per vocabulary token, and the sample node picks the next one. Branches above the backbone are computations; the faint dashed chips below it are the weights feeding each op.

The smaller figures embedded in nodes show the actual data at that point — diverging strips and bars where amber is positive and teal is negative (by value, not role), so you read the real numbers the model computed, not a sketch. Hung off the edges are the interactive control panels — the prompt, the sampling strategy + temperature, the KV cache toggle, and the generation length — and changing one recomputes and repaints immediately.

When you press Generate, a pulse sweeps along the backbone marking the stage being computed, and the generated text fills in. Hover any node, bar, label, or control for a plain-language explanation, and zoom to change the level of detail (LOD): zoomed out you see block structure; zoomed in, individual dimensions, tokens, and values appear.