The Base-Rate Problem: False-Positive Cost, Drift Budgets, and Calibrated Escalation Thresholds

Suppose your drift detector is right ninety-nine percent of the time on any single decision. That sounds excellent until you notice how often it decides and how rarely true drift actually occurs. Across a fleet of agents each taking thousands of actions, the detector renders millions of judgments a day, and genuine goal drift is a tiny fraction of them. Under those conditions, most of the alarms it raises can be false even though each decision is individually accurate — not because the detector is bad, but because rare events swamp accurate detectors with false positives. This is the base-rate problem, and it is the difference between a drift detector that operators trust and one they mute on the first day.

The Base-Rate Fallacy in Drift Detection

The base-rate problem is an application of Bayes' rule to a rare condition. What operators care about is the probability that drift is real given an alarm — the positive predictive value — but a detector is characterized by the reverse: the probability of an alarm given drift (its sensitivity) and the probability of an alarm given no drift (its false-positive rate). Bayes' rule combines these with the base rate, the prior probability that drift is occurring, and when that base rate is tiny, the posterior probability of real drift given an alarm can be small even for a highly sensitive, low-false-positive detector.

The intuition is that rarity does the damage. If true drift occurs in one decision in ten thousand, then even a false-positive rate of one in a hundred produces a hundred false alarms for every true one, because there are ten thousand times more opportunities to false-alarm than to true-alarm. The detector's per-decision accuracy is genuinely high; the fleet-level flood of false positives is a consequence of the base rate, not of any defect in the detector.

This is why per-decision accuracy is the wrong headline metric for a drift detector. Two detectors with identical sensitivity and false-positive rate can have wildly different usefulness depending on the base rate of the deployment they run in, and a detector validated on a balanced test set — where drift is artificially common — will look far better than it performs against a healthy fleet where drift is rare. The base rate must be part of the evaluation, not an afterthought.

Quantifying Alarm Volume

To make the problem concrete, count. The absolute number of false alarms is the number of decisions the detector makes, multiplied by its false-positive rate, multiplied by the fraction of decisions that are genuinely benign — which, since drift is rare, is nearly all of them. The number of true alarms is the number of decisions multiplied by the base rate of drift and the detector's sensitivity. The ratio of the two is what lands on an operator's screen, and it is governed almost entirely by the base rate.

This counting exposes two levers the detector controls and one it does not. It controls its false-positive rate, which scales the false-alarm count linearly, and its decision cadence, since fewer, more considered decisions produce fewer opportunities to false-alarm. It does not control the base rate, which is a property of the fleet's health. The design consequence is that on a healthy, rarely-drifting fleet, the false-positive rate must be driven extraordinarily low — far lower than balanced-test intuition suggests — to keep the absolute false-alarm volume tolerable.

The counting also reframes what tuning the detector means. Moving along the delay-versus-false-alarm frontier is not just trading two error rates; it is trading a manageable false-alarm volume against detection delay, at a scale set by the base rate. A one-percentage-point reduction in false-positive rate can remove far more alarms from the operator's queue than a one-percentage-point gain in sensitivity adds true detections, precisely because benign decisions vastly outnumber drifting ones.

Drift Budgets

The first structural remedy is to stop treating every deviation as an event. Because non-determinism guarantees constant small departures from the reference, a detector that escalates on any deviation is guaranteed to flood. A drift budget converts the question from is the agent deviating to has the agent deviated more than we tolerate, for long enough to matter. Concretely, it defines a tolerance band around the reference and a budget — an allowed amount of accumulated or sustained deviation — that must be exceeded before drift counts as an event at all.

A budget is what makes the base rate of events, as opposed to the base rate of deviations, something the design can raise deliberately. By absorbing benign fluctuation below tolerance, the budget removes the vast majority of would-be alarms at their source, before any thresholding, which is far more effective than trying to filter them afterward. The budget encodes a policy decision — how much drift is acceptable before it warrants attention — and stating that policy explicitly is healthier than burying it in a detector threshold.

Budgets also make drift detection compatible with a stochastic agent's normal operation. An agent will drift a little and self-correct constantly; a well-set budget treats that as the expected texture of autonomy rather than as a stream of incidents. The design goal is a budget generous enough to ignore benign self-correcting wobble and tight enough to catch sustained, accumulating departure before it completes — a band, not a line.

Calibrated Escalation Thresholds

With a budget absorbing benign deviation, the remaining thresholds must be set from the base rate and a cost model, not from accuracy. The principle is to escalate in proportion to expected cost: weigh the posterior probability that drift is real given the evidence against the impact of the drift if real, and route the result to a graduated response. A high-probability, high-impact drift blocks or halts; a low-probability or low-impact one logs. This is the same risk-proportioned escalation that behavioral enforcement uses, specialized to drift.

Calibration means choosing thresholds to hit a target on a base-rate-aware metric. Rather than setting a threshold to achieve a per-decision false-positive rate, set it to achieve a target precision — a tolerable fraction of true alarms among all alarms — computed at the fleet's actual base rate. Because precision degrades as the base rate falls, this forces the threshold higher on healthier fleets, which is exactly correct: the rarer true drift is, the stronger the evidence should be before you interrupt a human.

The escalation must be graduated rather than binary for the same reason a single threshold is brittle: any one line is simultaneously too sensitive for a healthy fleet and too dull for a compromised one. A ladder — enhanced logging at low posterior, a secondary automated check at moderate, human review at high, automated correction or halt at severe — lets the cheap responses absorb the ambiguous cases and reserves scarce human attention for the evidence that earns it. Each rung's threshold is calibrated to the base rate and the cost of that rung's action.

Precision at the Operating Point

The metric that survives the base-rate problem is precision — the positive predictive value — reported at the chosen operating point and the deployment's actual base rate. Precision answers the question operators live with: when this detector alarms, how often is it right. A detector that cannot state its precision at the base rate it runs in has not been evaluated for the environment it operates in, however good its balanced-test accuracy looks.

Precision must be reported as a function, not a scalar, because it moves with the base rate and the threshold. The honest artifact is a curve of precision against threshold at the estimated base rate, with the operating point marked and the base-rate estimate stated, so a reviewer can see how sensitive the precision is to a base rate that may itself be uncertain. Where the base rate is poorly known, the report should show precision across a plausible range of base rates, because a detector that is usable at one base rate may be unusable at a tenfold-rarer one.

This connects directly to trust, which is the detector's real currency. Operators calibrate their attention to the precision they experience: a high-precision detector earns fast response to its alarms, while a low-precision one trains reflexive dismissal that persists even when a true drift finally arrives. Because trust is slow to build and fast to lose, the conservative error is to set the threshold for high precision and accept some missed drift, then loosen only as the correction path becomes cheap enough that occasional over-alarming is harmless.

Alert Fatigue and the Disabled Detector

The base-rate problem's failure mode is human, not statistical. A detector whose queue is mostly false trains its operators, through ordinary learning, to dismiss its alarms — and a dismissed alarm is indistinguishable, in effect, from an alarm never raised. The endpoint is a detector that is technically running and practically disabled, which is worse than no detector because it creates a false sense of coverage while catching nothing that anyone acts on.

This makes false-positive volume a first-class safety concern rather than a mere annoyance. A drift-detection program that optimizes sensitivity while ignoring precision is optimizing the wrong quantity; it will detect more drift in the lab and less in production, because the production operators will have stopped looking. The NIST AI Risk Management Framework's emphasis on managing risk in operation, not only measuring it, applies directly: an alarm that no one acts on has not managed anything.

The remedy is the whole chain of this article: budgets to absorb benign deviation, calibration to the base rate, precision as the reported metric, and graduated escalation so that only high-precision, high-impact events reach a human. Each link exists to protect the scarce resource of operator attention, because that attention is what converts a detection into a mitigation, and the base-rate problem is precisely the force that destroys it if left unmanaged.

Setting Thresholds in Practice

Operationally, calibration is a repeatable procedure. Estimate the base rate of true drift from historical or adjudicated data, however roughly, because every threshold decision depends on it. Specify a cost model — the cost of a missed drift versus a false alarm at each escalation rung — from the deployment's actual consequences. Then set each rung's threshold to achieve its target precision at the estimated base rate, and record the base-rate estimate and cost model alongside the threshold so the choice is auditable.

Because the base rate is not static, calibration is not one-time. A fleet's drift base rate shifts with model updates, new tasks, and changing adversary pressure, and a threshold calibrated to last quarter's base rate silently changes its precision as the base rate moves. Monitor the realized precision on adjudicated alarms and re-calibrate when it drifts from target — the detector needs its own drift detection, applied to its false-alarm rate.

The reframing that closes the loop with the rest of this series is that detection and response are one system, not two. The tighter you can make the correction path — cheaper, more reversible, more automatic — the lower you can set thresholds without flooding humans, because the cost of a false escalation falls. Investing in cheap, reversible correction, the subject of the next article, is therefore also an investment in being able to detect drift sooner: the two ends of the loop set each other's budgets.

1. Estimate the base rate of true drift for the deployment, and record the estimate.
2. Set a drift budget so benign, self-correcting deviation never becomes an event.
3. Calibrate each escalation rung's threshold to a target precision at that base rate.
4. Report precision as a curve over threshold and base rate, with the operating point marked.
5. Monitor realized precision on adjudicated alarms and re-calibrate when it drifts.

Practitioner Toolkit

Copy-paste, strictly defensive artifacts you can use today. Nothing here attacks a real system.

function precision_at(threshold, sensitivity_fn, fpr_fn, base_rate):
    tpr = sensitivity_fn(threshold)          # P(alarm | drift)
    fpr = fpr_fn(threshold)                  # P(alarm | no drift)
    tp = tpr * base_rate
    fp = fpr * (1 - base_rate)
    return tp / max(tp + fp, 1e-9)           # positive predictive value

function calibrate(target_precision, base_rate, grid, sensitivity_fn, fpr_fn):
    for threshold in sorted(grid):           # ascending = stricter
        if precision_at(threshold, sensitivity_fn, fpr_fn, base_rate) >= target_precision:
            return threshold                 # lowest threshold meeting target precision
    return grid.max                          # none meets target: escalate design, not queue

Glossary

Base rate

The prior probability that drift is occurring on any given decision; when tiny, it makes accurate detectors produce mostly false alarms.

Positive predictive value (precision)

The probability that drift is real given an alarm; the base-rate-aware metric operators actually experience.

False-positive rate

The probability of an alarm when no drift is occurring; scales the absolute false-alarm count linearly.

Drift budget

A tolerance band plus an allowed amount of accumulated or sustained deviation that must be exceeded before drift counts as an event.

Calibrated threshold

An escalation threshold set to achieve a target precision at the deployment's actual base rate and cost model, rather than a per-decision accuracy.

Graduated escalation

A ladder of responses from enhanced logging to human review to halt, each rung's threshold set by base rate and the cost of its action.

Alert fatigue

The learned dismissal of a detector whose alarms are mostly false, which effectively disables the control.

References

1. NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0)
2. NIST AI 600-1, Generative AI Profile
3. Page, Continuous Inspection Schemes (Biometrika, 1954)
4. OWASP Agentic Security Initiative — Agentic AI Threats and Mitigations
5. OWASP Top 10 for Large Language Model Applications (2025)
6. Amodei, Olah, Steinhardt, Christiano, Schulman & Mane, Concrete Problems in AI Safety (arXiv:1606.06565, 2016)
7. MITRE ATLAS — Adversarial Threat Landscape for AI Systems