False Positives and Function-Breaking Remediation

A scanner reports a clean bill of health and a team relaxes; a scanner reports a data-flow finding and an automated remediation agent deletes the offending call. Both outcomes can be wrong, and they fail in mirror-image ways. The first over-trusts silence. The second over-trusts a flag. Between them sits the uncomfortable truth of every source-to-sink analysis: the tool has reconstructed what the program can do, but it cannot know what the running system is allowed to do, what an operator actually deployed, or which flagged flow is in fact the feature the product was built to ship. For conventional software this gap is a nuisance that shows up as noisy alerts. For an AI agent — whose central design is to route untrusted content into privileged tools — the same gap turns into two expensive failure modes: alarms nobody can act on, and fixes that quietly amputate the agent's reason to exist.

Two symmetric errors, one root cause

A security static analyzer answers a decidable proxy for an undecidable question. The real question is 'can an attacker cause harm here at runtime?'; the proxy the tool actually evaluates is 'does a path exist in my abstract model from a labelled source to a labelled sink without passing a labelled sanitizer?'. The distance between those two questions is where both errors live. A false positive is an alert that is true of the proxy but false of the deployed system: the path exists in the model, but at runtime it is unreachable, gated by a control the model did not see, or carries data an attacker cannot actually influence. A function-breaking remediation is the opposite motion: a change that makes the proxy report clean by removing the very flow that constitutes intended behaviour.

Define the terms precisely, because the whole discipline depends on the distinction. Query-validity is a property of the analysis: the finding correctly instantiates the query's source, sink, and reachability predicates over the abstract program. Exploitability is a property of the deployed system: an actor with a plausible level of access can drive the flagged flow to a harmful outcome under the configuration that actually ships. A finding can be query-valid and non-exploitable; that is precisely a false positive. It can also be non-obvious and genuinely exploitable; that is a true positive that deserves a fix. The analyzer can only ever assert the first property. The second is a judgement the analyzer is structurally incapable of making, and pretending otherwise is what produces both wasted triage and broken products.

Neither error is a bug to be patched out of a mature tool. Both are the visible surface of two theorems the field has known for decades, and an honest programme treats them as permanent operating conditions rather than temporary defects.

Why sound analysis must over-approximate

The formal reason false positives are unavoidable is abstract interpretation, introduced by Patrick and Radhia Cousot in 1977. Their framework computes program properties over an abstract domain related to the concrete semantics by a Galois connection: a soundness guarantee that the abstract answer over-approximates every concrete execution. Soundness is exactly the property a security analysis wants — it means 'if a real dangerous flow exists, the analysis will report it', i.e. no false negatives within the modelled semantics. But over-approximation is a two-sided coin. To guarantee it never misses a real flow, the abstract model must also admit flows that no concrete execution ever takes: infeasible paths, values that are conservatively assumed to be any value, branches assumed reachable because the analyzer cannot prove them dead. Every one of those admitted-but-infeasible flows is a candidate false positive.

The reason the model cannot simply be made exact is Rice's theorem (1953): every non-trivial semantic property of the function a program computes is undecidable. 'Is this source-to-sink flow actually exploitable at runtime' is a non-trivial semantic property, so no algorithm decides it for all programs. A tool therefore has exactly three moves: be sound and over-approximate (accept false positives), be complete and under-approximate (accept false negatives), or bound its effort and accept both. Security tooling deliberately chooses the first, because a missed real vulnerability is usually worse than a spurious alert. The false positive is not the tool failing; it is the tool keeping the promise you asked it to keep.

This reframes triage. If false positives were bugs, the correct response would be to file them and wait for a fix. Because they are the price of soundness, the correct response is to build a repeatable human-plus-context procedure that converts a query-valid finding into an exploitability judgement the analyzer was never able to make.

Where the false positives actually come from

Over-approximation is the theory; in practice a handful of concrete gaps generate most of the noise, and naming them is what makes triage fast instead of open-ended. The recurring causes are an unmodelled sanitizer (a real validation or encoding step the analyzer did not recognise, so it never clears the taint), runtime unreachability (the path is dead under the deployed configuration or feature flags), attacker-uncontrolled sources (the labelled source is real but only ever carries values the attacker cannot influence in this system), missing configuration context (an egress allow-list or policy engine sits between source and sink but lives outside the analyzed code), and query breadth (a rule tuned for recall on the reference corpus deliberately flags a broad pattern).

For AI-agent code these causes acquire agent-specific shapes. An unmodelled sanitizer is often a policy check that lives in a separate tool-server the analyzer never sees. A runtime-unreachable path is often a tool that is registered in code but disabled by deployment configuration. An attacker-uncontrolled source is often a retrieval call whose corpus is, in this particular system, fully first-party and immutable. Each of these makes a flow query-valid and non-exploitable at once — which is why an agent codebase can light up with findings that a knowledgeable operator can defensibly close, and why closing them requires system context the code alone does not contain.

The practical corpus lesson is the same one the field learned deploying analyzers across large real code bases: a substantial fraction of query-valid findings are non-exploitable, and the rate depends far more on how the sources, sinks, and sanitizers were modelled than on the depth of the solver. The controlled way to observe your own rate is a labelled test corpus.

Why the fix breaks the function

The mirror error is more dangerous because it ships. A source-to-sink finding for an agent almost always implicates a flow that is intentional. The whole point of a tool-using agent is that model output — derived from retrieved or user-supplied content — decides which tool to call, with which arguments, against which resource. That is, by construction, a flow from an untrusted-influenced source to a privileged sink. A naive remediation that 'removes the flow' to satisfy the query removes dynamic tool dispatch, or hard-codes a single destination, or strips the argument the model was supposed to fill. The scanner goes green. The agent stops being an agent.

This is qualitatively different from conventional software, where a flagged flow (say, a request parameter reaching a SQL string) is usually incidental and can be rerouted through a sanitizer with no loss of behaviour. In agent code the flagged flow frequently is the behaviour. Deleting it is not hardening; it is a silent functional regression dressed as a security fix — and because the metric it optimizes (findings-to-zero) looks like progress, it can pass review precisely when it should be stopped.

The design failure to avoid is treating 'make the finding disappear' as the objective. The objective is 'make the flow safe while it still works'. Those are different optimization targets, and only the first is what a query, or an agent naively acting on its own scanner's output, will pursue by default.

A reproducible triage protocol

Because the analyzer can assert query-validity but not exploitability, triage is the procedure that supplies the missing property. Make it a fixed cascade so that two engineers reach the same disposition from the same evidence, and so that every 'not exploitable' decision leaves an auditable reason rather than a shrug. The cascade confirms, in order: is the labelled source one an attacker can actually influence in this deployment; is the labelled sink genuinely privileged in this configuration; is the path reachable under the code and configuration that ship; is the tainted value attacker-controllable along that specific path; and does an existing control (a policy engine, an allow-list, an out-of-band sanitizer) already sit on the path outside the analyzed code.

Only after those five confirmations does the finding get a disposition, and there are exactly three legitimate ones. Fix: the flow is exploitable and the guard is missing — add a verifiable control. Suppress-with-reason: the finding is query-valid but provably non-exploitable — record the specific reason (which cause from the table above), attach the evidence, and encode it so the same alert does not resurface as noise. Accept-with-owner: the residual risk is real but a deliberate, owned business decision — record who owns it and what runtime control compensates. A bare 'ignore' is not on the list; every closed finding carries its reason.

The protocol's discipline is that suppression is a documented exception, never a default, and that a fix is only authorised after exploitability is affirmatively established — not merely because a red item appeared on a dashboard.

Capability-preserving remediation

When triage authorises a fix, the remediation pattern for agent code is almost never 'remove the flow'. It is 'keep the flow and make it verifiably safe at the boundary it crosses'. Concretely: retain dynamic tool dispatch but constrain it to a declared allow-list of (tool, argument-shape) pairs; retain model-chosen arguments but validate them against a schema and a policy engine before the privileged call; retain retrieval but mark its output as untrusted and require an explicit, logged authorisation step before any tool acts on an instruction derived from it. The capability survives; the unbounded authority does not.

The pattern generalizes to a rule: replace an implicit trust with an explicit, testable control on the same path. This is the enforcement point that the flow-modelling work is for — a guard placed exactly where the trust level changes. A good fix leaves the flagged data-flow present in the code (the feature still works) but adds a node on the path that a re-run of the same query can recognise as a sanitizer, so the finding legitimately clears because the risk was actually removed, not because the capability was.

The pseudocode below contrasts the two motions on the same running example — a support agent that reads tickets and can send email. The breaking fix deletes the send capability; the preserving fix keeps it behind a declared allow-list and an out-of-band approval for externally-influenced recipients.

# BREAKING FIX — clears the query, kills the feature
#   the agent can no longer send email at all
# def handle_ticket(ticket): summarize(ticket)   # send_email removed

# PRESERVING FIX — clears the query by adding a real guard on the path
ALLOWED_TOOLS = { "send_email": schema(to="email", subject="str", body="str") }

def handle_ticket(ticket):
    intent = model.plan(ticket)                 # source: ticket may be attacker-influenced
    tool, args = intent.tool, intent.args
    require(tool in ALLOWED_TOOLS)              # explicit allow-list (recognisable sanitizer)
    validate(args, ALLOWED_TOOLS[tool])         # schema check on model-chosen args
    if recipient_is_externally_influenced(args):
        require(human_approval(tool, args))     # out-of-band control for the risky case
    invoke(tool, args)                          # sink still reached — capability preserved

The special hazard of automated remediation

The stakes rise sharply when the entity applying the fix is itself an AI agent — a code-repair agent consuming its own scanner's findings and rewriting the code base to drive them to zero. Such a loop optimizes exactly the wrong objective (findings-to-zero) and has no access to the intent property that separates a false positive from a true one or a preserving fix from a breaking one. Left unsupervised it will confidently suppress real vulnerabilities it judges benign and delete real capabilities it judges dangerous, and it will do both at machine speed across the whole repository. NIST's Secure Software Development Framework is explicit that no single automated verification activity is sufficient on its own; an auto-remediation agent is a verification-and-repair activity that most needs that humility.

Two engineering constraints keep the loop safe. First, partial and query-aware repair: an automated fixer may propose a diff and re-run the specific query to show the finding clears, but it must not merge; the diff is a suggestion whose exploitability judgement and capability impact a human owns. Second, a hard human-review boundary on any change that alters a trust boundary, removes a tool capability, or touches a suppression: those are precisely the changes where the intent the agent lacks is decisive. Everything else — reformatting, adding a schema check the reviewer requested — can be lower-friction.

The countermeasure, in one line: let an agent draft remediations and prove the query clears, but never let the same agent that lacks runtime intent be the one that decides a finding is safe to close or a capability is safe to remove.

Regression discipline and the honest posture

Because both errors are decisions made under missing information, the guard against them is not a smarter scanner but a test discipline that catches a wrong decision after the fact. Every authorised fix ships with two test additions: a security test that fails on the original vulnerable flow and passes on the guarded one (proving the risk was actually removed), and a functional test that exercises the preserved capability (proving the feature still works). A fix that cannot satisfy both is either not a fix or is function-breaking, and the test suite says which. Every suppression ships with its recorded reason and, ideally, a check that re-flags it if the underlying context changes — a suppressed 'attacker-uncontrolled source' should reopen the moment that source starts carrying third-party data.

The posture this adds up to is the one to internalise. A static-analysis finding is a hypothesis produced by a sound over-approximation of a program whose runtime intent the tool cannot see. It is evidence, not a verdict; an input to human judgement, not a command to a rewriting agent. The analyzer's honest contribution is 'here is a query-valid flow you should look at'. The exploitability call, and the choice between guarding a flow and deleting it, belong to the people who know what the system is for. Treating the finding as a verdict produces wasted triage; treating it as a command produces broken agents. Treating it as a hypothesis produces neither.

That division of labour — the analyzer knows structure, the team knows intent — is not a workaround for immature tooling. It is the correct architecture given two theorems that are not going to be repealed.

Practitioner Toolkit

Copy-paste, strictly defensive artifacts you can use today. Nothing here attacks a real system.

allow_list:
  send_email: { to: email, subject: str, body: str }
guards:
  - require: tool in allow_list          # dynamic dispatch kept, bounded
  - validate: args against allow_list[tool].schema
  - if: recipient_externally_influenced
    require: human_approval               # out-of-band control for the risky case
principle: keep the capability, remove the unbounded authority

Glossary

Query-validity

The property that a finding correctly instantiates the analysis query's source, sink, and reachability predicates over the abstract program model.

Exploitability

The property that an actor with plausible access can drive a flagged flow to a harmful outcome under the configuration that actually ships.

False positive

A finding that is query-valid but not exploitable in the deployed system.

Function-breaking remediation

A code change that makes the analysis report clean by removing a flow that constitutes intended behaviour.

Sound over-approximation

An analysis that reports a superset of the real dangerous flows, guaranteeing no false negatives at the cost of admitting infeasible ones.

Capability-preserving remediation

A fix that keeps the flagged flow present but adds a verifiable control where the trust level changes, so the capability survives and the risk is removed.

References

1. Cousot & Cousot, Abstract Interpretation (POPL 1977)
2. Rice, Classes of Recursively Enumerable Sets and Their Decision Problems (1953)
3. Livshits & Lam, Finding Security Vulnerabilities in Java Applications with Static Analysis (USENIX Security 2005)
4. NIST Software Assurance Reference Dataset (SARD) and Juliet Test Suite
5. OWASP Benchmark Project
6. NIST SP 800-218, Secure Software Development Framework
7. MITRE Common Weakness Enumeration (CWE)
8. OWASP Agentic AI Threats and Mitigations