Research

Research seriesL3offensive security

Sparse Weight Backdoors: Detecting an Attack That Lives in a Few Neurons

A backdoor no longer has to be trained in. Editing fewer than ten rarely used neurons in an open-weight model can make an agent emit an attacker's command on a trigger, while benchmarks and behavioural tests see almost nothing. This threat lab asks the defender's question: which evidence actually catches such an edit — behavioural testing, reference-free weight inspection, a diff against the declared base, or a signature — what each result does and does not justify, and how to choose. It explains the mechanism without a recipe, the attacker's trade-off between stealth in the weights and precise control of the payload, and the evidence limits of every detector. Grounded in the primary Lazarus study and the backdoor, model-editing and supply-chain literature.

Murali Chillakuru·1 articles
  1. 1
    Loud in the Weights, Silent in the Tests: Sparse Weight Backdoors and Choosing the Right Evidence

    Edit seven rarely used neurons and an agent runs an attacker's command on cue. Tests barely notice. The weights do. Which evidence should you trust?