Research seriesL3offensive security
Multi-tenant, batched, KV-cached, speculatively-decoded serving is efficient — and leaky. Timing and cache behavior turn a shared endpoint into a side channel. This threat lab formalizes the shared-serving threat model, prompt-cache membership oracles, decoding-timing leaks measured in bits per query, denial-of-wallet resource amplification, and the isolation trade-offs that mitigate them — each paired with a hardening. Grounded in the primary systems and side-channel literature.
The optimizations that make model serving cheap — shared caches, dynamic batching, speculative decoding — also give a co-tenant a window into other users' traffic.
A cache hit is faster than a miss, and in shared serving that speed difference tells an attacker whether someone else recently sent a given prompt.
Speculative decoding makes generation faster when the text is predictable — which means the response time itself encodes how predictable your output was.
You do not need to crash a model to take it down — you can make each request so expensive that serving it, or paying for it, becomes the attack.
Every serving side channel closes the same way — remove, mask, or meter the shared state — and every fix costs throughput, so the real work is choosing the trade deliberately.