The KV cache

Storing each position's keys and values so attention isn't recomputed every step.

In this section

• Why cache — The redundant recomputation a KV cache removes.
• Correctness — Why greedy output is provably identical with the cache on or off.
• Reading occupancy — Reading the KV lane: stored positions, the current token, and attention shading.