Research seriesL2algorithms
The algorithms that make a large language model answer on demand — attention and the KV cache, speculative decoding, quantization error, and vector-index recall — derived properly, with the complexity and the point where each one breaks. Grounded in the primary papers.
Why serving a language model is a memory problem, derived from the attention equation to the exact bytes a running conversation holds.
How a cheap draft model and a rejection-sampling rule cut latency without changing a single output token's distribution.
The arithmetic of rounding a model's weights to a coarse grid, why every bit buys about six decibels, and where the error finally wins.
Why approximate nearest-neighbour search is a dial between how many true neighbours you find and how long you wait, and how HNSW and IVF set that dial.