Watch · narrated walkthroughs

Inference-Infrastructure Side Channels: Leaks in Shared Serving

Multi-tenant, batched, KV-cached, speculatively-decoded serving is efficient — and leaky. Timing and cache behavior turn a shared endpoint into a side channel. This threat lab formalizes the shared-serving threat model, prompt-cache membership oracles, decoding-timing leaks measured in bits per query, denial-of-wallet resource amplification, and the isolation trade-offs that mitigate them — each paired with a hardening. Grounded in the primary systems and side-channel literature.