Research seriesL3offensive security
A model exposed only through an API still leaks its parameters, its architecture, and its training data — and the leakage is quantifiable. This threat lab treats extraction as a measurement problem with information-theoretic limits: the query-access threat model, recovering a production model's last layer from logits, training-data memorization and its extraction rate, membership inference as a privacy metric, and the utility cost of every defense. Grounded in the primary extraction literature.
A model reachable only through an API is not a black box in the way defenders hope — what it returns per query sets a precise, quantifiable ceiling on what can be stolen.
The map from a model's hidden state to its output is a low-rank linear layer — and low-rank linear maps are exactly what a few thousand API responses can solve for.
Large models reproduce fragments of their training data verbatim, and the fraction that can be pulled back out is a measurable quantity that grows with scale, duplication, and context.
Deciding whether one record was in the training set is the canonical privacy leak — and its true severity lives in the high-confidence, low-false-positive corner, not the headline accuracy.
Every extraction defense buys confidentiality with a currency — output precision, accuracy, throughput, or a formal privacy budget — and choosing well means pricing each honestly.