Research

Watch · narrated whiteboard episodesL3

Sparse Weight Backdoors: Detecting an Attack That Lives in a Few Neurons

A backdoor no longer has to be trained in. Editing fewer than ten rarely used neurons in an open-weight model can make an agent emit an attacker's command on a trigger, while benchmarks and behavioural tests see almost nothing. This threat lab asks the defender's question: which evidence actually catches such an edit — behavioural testing, reference-free weight inspection, a diff against the declared base, or a signature — what each result does and does not justify, and how to choose. It explains the mechanism without a recipe, the attacker's trade-off between stealth in the weights and precise control of the payload, and the evidence limits of every detector. Grounded in the primary Lazarus study and the backdoor, model-editing and supply-chain literature.

Murali Chillakuru·1 episodes
  1. 19 min Episode 1Loud in the Weights, Silent in the Tests: Sparse Weight Backdoors and Choosing the Right EvidenceA few rewritten neurons can make an agent run an attacker's command on cue while tests hear almost nothing. How the edit works, why behaviour stays quiet, what a weight scan sees, how to read a detector's verdict, the attacker's trade-off between precision and stealth, and how to choose evidence by reference, signer and agent authority.