Model-Mediated Taint Tracking

Taint tracking is one of the oldest reliable ideas in program security: mark untrusted input as tainted, follow it as it moves and is transformed, and raise an alarm if it reaches somewhere dangerous while still tainted. The method works because every transform on the way has a known effect — a concatenation carries taint, a proper escaping routine removes it. An AI agent drops a transform into the middle of that path that has no such known effect: a language model that takes tainted text in and emits text out, having read, reinterpreted, and possibly obeyed it. The question is not whether taint tracking applies to agents — it does — but what rule governs taint as it crosses the model. This piece argues the rule is short and strict: the model preserves taint, can escalate it from data into instruction, and must never be credited with removing it.

Taint tracking and the transform assumption

Fix the vocabulary first. A source is where untrusted data enters; a sink is where tainted data causes harm; propagation is the set of rules for how taint moves through operations; and a sanitizer is an operation credited with removing taint, so a value that passes through it reaches a sink safely. The whole method is an accounting exercise over these four: mark, follow, and check, with sanitizers as the only way taint is discharged. Its soundness rests on getting the propagation rules right for every transform on the path.

That last clause hides an assumption that conventional code satisfies and an agent violates. For ordinary transforms, the effect on taint is knowable in advance: string concatenation propagates taint from its inputs, a parameterized-query binding neutralizes it, a hash of a value produces something no longer attacker-controllable in the relevant sense. The analyzer ships these rules, or a model author writes them, and they hold because the transform is deterministic and its semantics are fixed.

A language model is a transform on the path — tainted text in, text out — but its effect on taint is not knowable in advance in the same way. It is non-deterministic, its behavior shifts with the very content flowing through it, and an attacker can shape that content to steer the output. The remainder of this piece works out the propagation rules that survive those properties, starting from the one that matters most.

The model is a transform, but not a normal one

Three properties separate the model from every transform a taint engine usually meets, and each one forbids a rule that would otherwise be reasonable. The model is non-deterministic: the same input can produce different outputs across runs, so no static rule can claim 'this input always yields that sanitized output'. Any sanitization it might appear to perform in one run is not a property of the transform; it is a sample.

The model is semantics-shifting: unlike a hash or an escape, it does not apply a fixed function to bytes — it interprets meaning, and its output depends on the content in ways that are not compositional. A value that looks inert can change how the model treats everything around it, so taint cannot be reasoned about token by token as it can through a string operation. The transform's effect on one span depends on the whole context, which is precisely what a local propagation rule cannot capture.

And the model is adversarially steerable: the content flowing through it is exactly the lever an attacker uses to control its output. This is the property that makes the model unlike even other unpredictable transforms. A flaky function is merely uncertain; the model is uncertain in a direction the attacker chooses, because the attacker writes the input. Denning's classic lattice model of information flow gives the frame: taint is a label that may only move up the lattice unless an operation is trusted to declassify it, and nothing about a steerable, non-deterministic transform earns that trust.

The first rule: the model never sanitizes

The single most important propagation rule follows directly: a value that enters the model tainted leaves it tainted. The model is never a sanitizer. Crediting it with removing taint — reasoning that 'the model summarized the ticket, so the output is clean' — is the fatal error, because it discharges taint on the strength of a transform that is non-deterministic and attacker-steerable. A sanitizer must be a transform you can prove removes the danger; the model is the opposite of provable.

This is not pessimism about model quality; it is a soundness requirement. Soundness in taint tracking means never dropping real taint, so any transform whose taint-removing behavior cannot be guaranteed must be treated as taint-preserving, or the whole analysis loses its guarantee. A model that usually strips an injection is still, from the analysis's point of view, a transform that sometimes does not — and 'sometimes does not' is exactly a false negative waiting to happen. The conservative rule, taint in implies taint out, is the only one that keeps the accounting honest.

The practical corollary is a placement rule for real sanitizers: they belong outside the model, in verifiable code, on the model's output. An allow-list that checks a tool argument against a fixed set, a strict schema validator that rejects anything unexpected, a parser that accepts only a narrow grammar — these are transforms whose taint-removing effect can be proven, and they are where taint is legitimately discharged. The model proposes; verifiable code downstream is what may sanitize, and only it.

The second rule: taint can escalate from data to instruction

Preserving taint is necessary but not sufficient, because the model can do something no ordinary transform does: change what the taint means. A tainted value entering the model as data — the text of a ticket — can leave it as control, an instruction the agent acts on. This escalation from data-plane taint to control-plane taint is the essence of prompt injection, and a propagation model that tracks only a single kind of taint cannot express it.

Information-flow analysis already has the machinery for this: flow state, a label carried alongside taint that distinguishes kinds of tainted value and lets propagation rules depend on the kind. Here the states are simple and consequential. A value can be untainted; tainted as data, meaning attacker-influenced content that is being handled as data; or tainted as instruction, meaning attacker-influenced content that has become a directive the agent may obey. The model boundary is the transition that can promote data to instruction, and marking that transition is what turns a vague worry into a trackable event.

The rule, then, is a two-part propagation across the model: taint is preserved, and its state may escalate from data to instruction but never de-escalate on the model's word alone. A path that carries a value from an untrusted source, through the model where its state rises to instruction, into a consequential tool is the injection chain written in the vocabulary of taint. Expressing it this way makes it a first-class object an analysis can look for, rather than a phenomenon that lives only in prose.

What static analysis can represent

With the rules fixed, the question is how much of them a static analyzer can carry. The answer is the propagation, but not the judgement. An analyzer can model the model call as an additional taint step: a declared edge stating that taint on the input flows to the output, with the state escalated to instruction. This is a conservative over-approximation — it assumes the worst about a transform it cannot inspect — and conservatism is exactly what soundness wants. The analyzer does not need to understand the model to route taint through it; it needs only to refuse to drop taint at the boundary.

What the analyzer cannot do is decide whether, on a given run, the model actually obeyed the injected instruction. That is a runtime, behavioral fact about a non-deterministic system, and no static reasoning recovers it. The consequence is a precision cost: modeling the model as always taint-through, always escalating, will flag every path from untrusted content to a consequential tool, including those a real deployment would neutralize. Some of those alarms are the genuine risk; some are paths a downstream verifiable sanitizer or a runtime broker defuses.

This locates the sanitizer placement rule precisely in the analysis. Because the model is an opaque taint-through step, the only taint the analyzer can legitimately discharge is taint removed by a transform it can verify — the allow-list, the schema check, the strict parser sitting on the model's output before the sink. Those are declared as real sanitizers; the model is declared as a taint-through, state-escalating step; and the analysis is sound with a precision that depends entirely on how much verifiable sanitization stands between the model and the sink.

# model boundary: taint-through, escalate data -> instruction (NEVER a sanitizer)
model.complete ; input=Argument ; output=ReturnValue ; kind=taint ; state=escalate(D->I)

# a REAL sanitizer sits outside the model, on its output, and is verifiable
def guard(tool_args):
    assert tool_args.to in ALLOWLIST      # fixed set, provable
    assert schema.valid(tool_args)         # strict grammar, provable
    return tool_args                       # taint legitimately discharged here

The sources and sinks the model sits between

The propagation rules only bite if the sources and sinks around the model are marked, and agents introduce several that conventional analysis does not track by default. On the source side, the model's own output is the important and counterintuitive one: because the model preserves and can escalate taint, its output must be treated as a tainted source whenever tainted content reached its input. Retrieved documents, tool results returned into context, and prior-turn memory read back in are likewise sources, each an entry point for attacker-influenced content.

On the sink side, the consequential ones are the tool calls the model's output drives — sending mail, writing a file, issuing a request — and the writes into long-term memory, which are sinks in one turn and sources in the next. The pairing of a memory write as a sink with a later memory read as a source is what lets taint cross turns, and it must be modeled as a boundary pair or the cross-turn injection path is invisible. Enumerating these carefully is a topic in its own right; here the point is narrower: the propagation rules for the model are only meaningful against a correctly marked set of endpoints.

With endpoints marked and the model declared as a taint-through, state-escalating step, the analysis produces exactly the objects worth looking for: paths from any attacker-influenced source, through the model where state rises to instruction, into a consequential tool or a memory write, with no verifiable sanitizer in between. That path is the injection chain, and it is now something a query can name rather than something a reviewer must imagine.

Limits: propagation without judgement

The honest boundary of this approach is that it propagates taint correctly but cannot judge behavior. A static analyzer following these rules will tell you, soundly, that a path exists from untrusted content through the model to a tool with no verifiable sanitizer in between. It will not tell you whether the model, on a particular request, actually followed the injected instruction, because that is a fact about a non-deterministic execution that static reasoning does not have access to. The analysis answers 'is there an unguarded path', not 'will it be exploited today'.

That gap is not a defect to be engineered away; it is the correct division of labor. Static analysis contributes the sound structural result — the unguarded injection paths — and runtime contributes the behavioral judgement: a tool broker that authorizes the call, output filtering, human confirmation on consequential actions, and adversarial testing that probes whether the model can in fact be steered down the path. Pretending the analyzer can settle the behavioral question reintroduces exactly the unsound optimism the no-sanitizer rule exists to prevent.

The precision of the static half is a dial, and the sanitizer placement rule is how it is turned. The more verifiable sanitization stands between the model and its sinks — allow-lists, schema checks, narrow parsers — the fewer unguarded paths remain and the more each surviving alarm means. An agent with no verifiable downstream checks will light up every model-to-tool path, which is not the analyzer being noisy but the architecture being genuinely open. Reducing the alarms is the same work as reducing the risk.

Putting the rules to work

In practice the whole approach reduces to a handful of declarations and one habit. Declare the model call as a taint-through step that escalates data-state taint to instruction-state, never as a sanitizer. Mark the agent-specific sources and sinks — model output, retrieval, tool results, memory reads as sources; tool calls and memory writes as sinks. Declare as sanitizers only the verifiable transforms on the model's output, and place them deliberately between the model and every consequential sink. The habit is to read every surviving model-to-sink path as a finding until a verifiable guard or a runtime control accounts for it.

Doing this keeps taint tracking sound across an agent without pretending the model is transparent. The analysis no longer silently drops taint at the model, which is the failure mode that lets a clean report cover an injectable agent; instead it conservatively carries taint through and surfaces the unguarded paths, leaving the behavioral verdict to the controls built for it. The result is a clean report that means what it should: on the paths with verifiable guards, taint is discharged; on the paths without, the report is not clean, as it should not be.

The closing point is that the model does not break taint tracking; it demands discipline from it. Preserve taint across the model, track its escalation from data to instruction, discharge it only in code you can verify, and hand the behavioral question to runtime. Those four rules are enough to reason soundly about an agent's information flow, and they draw the same honest line the rest of this subject keeps arriving at: static analysis handles the structure, and the model's behavior belongs to the controls that watch it run.

Practitioner Toolkit

Copy-paste, strictly defensive artifacts you can use today. Nothing here attacks a real system.

def guard_tool_call(action):
    if action.name not in TOOL_ALLOWLIST: reject()      # fixed set
    if not SCHEMA[action.name].validate(action.args): reject()
    if action.crosses_external_boundary: require_human_confirm()
    return action        # taint discharged: reached here only if provably safe

Glossary

Taint tracking

A data-flow analysis that marks untrusted input, follows it through transforms, and reports if it reaches a sink while still tainted.

Sanitizer

A transform credited with removing taint, so a value passing through it may reach a sink safely; it must be provable to be sound.

Additional taint step

A declared edge telling the analyzer that taint on an input flows to an output through a call it does not inspect, such as the model boundary.

Flow state

A label carried alongside taint that distinguishes kinds of tainted value — here untainted, tainted data, or tainted instruction — so rules can depend on the kind.

Taint escalation

A transition that raises a value's flow state, here from data to instruction at the model boundary — the information-flow shape of prompt injection.

Declassification

The trusted removal of a security label in information-flow theory; a non-deterministic, steerable model cannot earn the trust declassification requires.

Over-approximation

A conservative analysis that assumes the worst about an uninspectable transform, preserving soundness at the cost of some false positives.

References

1. Denning, A Lattice Model of Secure Information Flow (Communications of the ACM, 1976)
2. Livshits & Lam, Finding Security Vulnerabilities in Java Applications with Static Analysis (USENIX Security 2005)
3. Cousot & Cousot, Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs (POPL 1977)
4. MITRE Common Weakness Enumeration (CWE)
5. NIST SP 800-218, Secure Software Development Framework (SSDF) v1.1
6. OWASP Agentic AI — Threats and Mitigations