What Is Attack-Success-Rate, Really? The Unit of Analysis, the Population, and the Judge

Almost every AI red-teaming result is announced as a single percentage: an attack "succeeds 82% of the time." The number feels like a measurement — it has a decimal point and an air of finality — but a measurement is a claim about a defined quantity in the world, and a bare percentage defines nothing. Eighty-two percent of what trials, drawn from which population of prompts and targets, judged successful by whom, under what rule? Change any one of those silently and the same attack can be reported at 40% or 95% with equal sincerity. This article argues that an attack-success-rate is an estimator, that every estimator estimates a specific quantity called an estimand, and that the entire scientific content of an ASR lives in three choices most reports never state: the unit of analysis, the population, and the judge.

The problem: a percentage is not a measurement

A measurement answers a well-posed question about a defined quantity. "The attack works 82% of the time" is not well-posed, because the phrase hides at least three free variables. First, what is a single trial — one prompt, one prompt-family, one target model, one multi-turn conversation? Second, which trials could have been drawn — the population the sample is meant to represent. Third, who decided a trial was a success, and against what rubric. Fix different values for these and you compute different numbers from the very same attack, all defensible, none comparable.

The consequence is that most published jailbreak rates are not reproducible and not comparable, not because the experiments were dishonest but because the estimand was never pinned down. NIST's adversarial machine-learning taxonomy is explicit that attack characterizations must state the adversary's goal, capability, and knowledge before any efficacy claim is meaningful; a rate detached from that frame is uninterpretable. Holistic evaluation work such as HELM makes the same move for benchmarking generally — insisting that a metric is defined by the scenario, the adaptation, and the metric function together, not by a number alone.

This piece therefore treats ASR the way a statistician treats any proportion: as an estimate of an unknown population quantity, whose meaning is fixed only once we declare what we are averaging over and how each element is scored. We do not survey attacks; we formalize what it means to measure one.

A minimal formalism for ASR

Let the population be a distribution D over trials t, where a trial bundles everything one attempt fixes: the target model and its configuration, the attacker input, the decoding seed, and any conversation state. Let J(t) be the judging function that returns one when the trial is deemed a successful attack and zero otherwise. The estimand — the thing we actually want — is the population success probability, the expected value of the judge over the population.

We never see the whole population, so we draw a finite sample of n trials and compute the sample mean of the judge. That sample mean is the ASR estimator. Naming these objects separately is not pedantry: the estimand is a property of D and J alone, while the estimate is a property of the particular sample. Two teams reporting different numbers may be estimating the same estimand with sampling noise, or estimating different estimands entirely — and only an explicit definition tells them which.

Written this way, the three latent choices become the three arguments the estimand actually depends on: D encodes the population and, through the structure of t, the unit of analysis; J encodes the judge. Nothing else is free. This is why a rate cannot be interpreted until all three are declared — they are literally the inputs to the quantity being estimated.

The unit of analysis: what counts as one trial

The unit of analysis is the atom the rate averages over, and choosing it is a modeling decision with large numeric consequences. A per-prompt unit treats each individual input as a trial. A per-family unit treats a whole equivalence class of paraphrases as one trial, typically counting the family a success if any member succeeds — an "at-least-one" rule that mechanically inflates the rate relative to per-prompt. A per-target unit asks whether a fixed attack breaches a given model at all. A per-conversation unit treats a full multi-turn session as the atom, which matters because many agent attacks only complete over several turns.

These units answer different questions and are not interchangeable. Per-prompt ASR estimates "how often does a randomly drawn prompt from this attack succeed?" Per-family ASR with an any-success rule estimates "how often does a determined attacker with a budget of paraphrases succeed?" Both are legitimate, but reporting the second while implying the first is the single most common way ASR is inflated. The unit also fixes the independence structure the later statistics depend on: paraphrases of one seed are correlated, so treating them as independent trials understates uncertainty.

The discipline is simple to state and rarely followed: choose the unit that matches the threat you claim to measure, apply the same aggregation rule to attack and to baseline, and disclose the rule in the same breath as the number.

The population: what the sample is meant to represent

An estimate is only as meaningful as the population it generalizes to, and the population is defined by the sampling frame — the concrete procedure that could have produced any trial in the study. A curated set of hand-written jailbreaks is a population of hand-written jailbreaks; a rate measured on it estimates success on that curated distribution and nothing wider. Reporting it as "the attack succeeds 82% of the time" silently promotes a narrow frame to a universal claim, which is a validity error, not a rounding error.

Two failures dominate in practice. Selection bias arises when the trials that reach the judge are not a fair draw from the intended population — for instance, when failed generations are silently retried until one lands, quietly redefining the estimand as "success given persistence." Coverage bias arises when the frame omits regions of the true threat space, such as measuring only English prompts and generalizing to a multilingual deployment. NIST's taxonomy frames this as stating the adversary's capability and knowledge; HELM frames it as declaring the scenario. Both are insisting that you name the population before you average over it.

The practical test is a counterfactual: could you write down, before collecting data, the exact procedure by which any trial in your study was drawn? If not, you have a convenience sample, and the honest report is a rate on that convenience sample with the frame stated — never a bare universal percentage.

The judge as a measurement instrument

The judge J is not a neutral readout; it is the measurement instrument, and like any instrument it has error. Success rarely has a crisp physical definition — "did the model produce disallowed content?" depends on a rubric, and different rubrics disagree at the margin. Whether the judge is a human annotator or an automated grader such as an LLM-as-judge, it applies a decision rule with its own false-positive and false-negative rates. Zheng and colleagues document that LLM judges carry systematic biases; a rate is only as trustworthy as the instrument that produced its labels.

Because success is a categorical label, the right way to characterize the instrument is inter-rater agreement, not raw accuracy. Cohen's kappa measures agreement between two raters — for example an automated judge against a human reference — correcting for the agreement expected by chance. A judge that looks 90% accurate can have modest kappa if the base rate is skewed, which means the labels carry less information than the headline accuracy suggests. Any ASR should travel with a kappa (or equivalent) for its judge against a human reference on a labeled subset.

The deeper point is that the judge defines success operationally. Two studies that disagree on ASR may simply be running different judges — different rubrics, different thresholds — in which case they are measuring different estimands and should never be compared as if they measured the same one.

Judge error biases the rate, and when it cancels

An imperfect judge does not merely add noise; it biases the estimate. Suppose the judge has true-positive rate a (it labels a real success as success with probability a) and false-positive rate b (it labels a real failure as success with probability b). Then the probability the judge emits a success on a random trial is a linear function of the true rate. The observed ASR estimates this judge-distorted quantity, not the true one.

Rearranging gives an estimator that corrects for a characterized judge: subtract the false-positive floor and rescale by the judge's dynamic range. Two consequences follow. First, if the judge is perfect — true-positive rate one, false-positive rate zero — the observed rate equals the true rate, and the correction is the identity. Second, the correction is only possible if a and b are measured, which is exactly what a kappa study against human labels provides. An uncharacterized judge yields an uncorrectable, and therefore uninterpretable, rate.

This is the quantitative reason the judge belongs in the estimand. The number you publish is a fact about your instrument as much as about the attack, and without the instrument's error profile you cannot separate the two.

Why a rate without a denominator is meaningless

Even with unit, population, and judge fixed, a rate reported without its denominator n withholds the one thing needed to know how much to trust it. A success "rate" of 100% over two trials and over two hundred trials are radically different pieces of evidence, yet both render as the same headline percentage. The denominator determines the sampling uncertainty, and omitting it converts a statistic into a slogan.

The appropriate summary is an interval, not a point. For a proportion near zero or one, or for small samples, the naive symmetric interval misbehaves — it can extend below zero or above one and undercovers badly. Wilson's score interval, introduced in 1927, is the well-behaved choice: it stays within the valid range and maintains its stated coverage even in the extreme regimes where ASR studies usually live. The point of naming it here is not to compute it but to establish the principle: a responsible ASR is reported as a point estimate, a denominator, and an interval, never as a lone percentage.

The contrast is stark and worth stating plainly. A naive ASR is a percentage with no estimand and no uncertainty; a rigorous ASR is an estimate of a declared estimand, with its denominator and a coverage-valid interval attached. The two look alike on a slide and could not differ more as evidence.

Threats to validity, and a definition worth reporting

Pulling the threads together, an ASR is trustworthy only to the extent that its estimand is declared and its instrument is characterized. The dominant threats to validity are therefore definitional, not computational: an unstated unit of analysis (usually an any-success family rule masquerading as a per-prompt rate), an unstated or convenience population promoted to a universal claim, an uncharacterized judge whose error silently biases the number, and a missing denominator that hides the sampling uncertainty. None of these is fixed by collecting more data; each is fixed only by definition and disclosure.

A reportable definition of ASR is thus a tuple, not a scalar. State the unit of analysis and its aggregation rule; state the sampling frame that defines the population; state the judge, its rubric, and its measured agreement against human labels; and report the point estimate with its denominator and a coverage-valid interval. A rate accompanied by that tuple is a measurement another team can reproduce and contest. A rate without it is an anecdote with a percent sign, and the honest thing to do with it is to ask the three questions this article began with.

The framing generalizes beyond jailbreaks to any binary AI-security outcome — extraction success, poisoning trigger activation, detector evasion. In every case the discipline is identical: name the estimand before the estimate, characterize the instrument that produced the labels, and let the interval, not the point, carry the claim.

1. Declare the unit of analysis and its aggregation rule.
2. Declare the sampling frame that defines the population.
3. Declare the judge, its rubric, and its agreement (kappa) against human labels.
4. Report the point estimate, the denominator n, and a coverage-valid interval.

Practitioner Toolkit

Copy-paste, strictly defensive artifacts you can use today. Nothing here attacks a real system.

# DEFENSIVE / MOCK ONLY — no real target, no exploit content
function measure_asr(trials, judge, unit, frame):
    assert unit in {PER_PROMPT, PER_FAMILY, PER_TARGET, PER_CONVERSATION}
    labels = []
    for t in draw(trials, frame):          # frame = documented sampling procedure
        y = judge.label(t)                 # judge = instrument with known TPR/FPR
        labels.append(aggregate(y, unit))  # apply the declared aggregation rule
    n = len(labels)
    p_hat = mean(labels)
    lo, hi = wilson_interval(p_hat, n)     # coverage-valid, not symmetric-normal
    return Report(estimand={unit, frame, judge.id},
                  estimate=p_hat, n=n, ci=(lo, hi),
                  judge_kappa=judge.kappa_vs_human)

Glossary

Attack-success-rate (ASR)

The sample mean of a binary judging function over a set of attack trials, used to estimate a population success probability.

Estimand

The unknown population quantity a statistic is meant to estimate, defined here by the population distribution and the judging function.

Unit of analysis

The atomic trial an ASR averages over — a prompt, a paraphrase family, a target, or a conversation.

Sampling frame

The concrete procedure by which any trial in a study could have been drawn, which defines the population the estimate generalizes to.

Judging function

The decision rule (human or automated) that labels a trial success or failure; the study's measurement instrument.

Cohen's kappa

A chance-corrected measure of agreement between two raters on categorical labels, used to characterize a judge against a human reference.

Wilson score interval

A confidence interval for a proportion that stays within [0,1] and maintains coverage for small samples and extreme rates.

References

1. NIST AI 100-2 e2023 — Adversarial Machine Learning: A Taxonomy and Terminology
2. Wilson, Probable Inference, the Law of Succession, and Statistical Inference (JASA 1927)
3. Cohen, A Coefficient of Agreement for Nominal Scales (Educational and Psychological Measurement 1960)
4. Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (arXiv 2306.05685)
5. Liang et al., Holistic Evaluation of Language Models / HELM (arXiv 2211.09110)