Home

Inside the Transformer: Attention & the Forward Pass

Five expert explainers on how a transformer actually computes — the forward pass end to end, multi-head attention shapes and the square-root-d scaling, the arithmetic of one generated token, positional information and the residual stream, and how attention leads directly to the KV cache. The mechanics beneath the optimization.