The residual stream

The running vector the whole network keeps refining as it flows right.

In this section

• The vector — What the [dim] vector holds, and how to read the teal/amber feature bars.
• The logit lens — The 'best guess from here' readout, and why it sharpens toward the answer rightward.
• Residual add (⊕) — The skip connection x = x + sublayer(norm(x)) that lets gradients flow.