-
1The Forward Pass, End to EndFollow a single token through a decoder-only transformer: from embedding, through a stack of identical blocks, to a probability distribution over the vocabulary.Advanced
2Multi-Head Attention: Shapes, Q/K/V, and the ScaleOpen up the attention sublayer: the query, key, and value projections, the scaled dot-product, the head split, and the causal mask — with every tensor shape.Advanced
3The Arithmetic of One TokenCount the cost of a single forward pass: why it's roughly two FLOPs per parameter, where the attention term bites, and why decoding is bound by memory, not maths.Advanced
4Positional Information and the Residual StreamAttention is blind to order, so position must be injected — and everything a layer computes travels on a shared residual bus. Two structural ideas that make transformers work.Advanced
5From Attention to the KV CacheThe KV cache isn't a trick bolted on — it falls straight out of the causal-attention equations. Derive it, read the decode loop, and see the complexity shift it buys.Advanced