Memory as an Attack Surface: Poisoning, Staleness, and Leakage

Memory is usually described as a feature that makes agents better. It is also, unavoidably, an attack surface. The moment an agent writes things down and reads them back later, everything it stored becomes a potential vector: content an attacker planted, facts that quietly went stale, data that leaks across users, or observations that got promoted to trusted knowledge without ever being checked. A memory system that is not designed defensively does not just fail to help. Over time, it makes the agent more dangerous.

Memory is not automatically good

A store that faithfully remembers everything will also faithfully remember the wrong things: a poisoned note, an outdated policy, another user's data. And because the agent trusts its own memory, those errors are recalled with confidence.

So memory has to be treated like any other untrusted input path, with the same suspicion you would apply to a web page or a user message, not as a private, trustworthy scratchpad.

The poisoning path

The classic attack is memory poisoning. Untrusted content is written into memory during one run, sits quietly, and is later retrieved and treated as a trusted instruction that steers a consequential action. Nothing looked wrong at the moment of writing.

Seeing the path makes the defense obvious: the danger is not the write itself but the later, unlabeled recall. Break the link between untrusted origin and trusted reuse and the attack collapses.

Untrusted content written to memory is later recalled as if trusted and steers an action.

Six ways memory betrays you

Poisoning is the headline, but memory fails in several related ways, and a defensive design has to account for all of them.

Each is a distinct failure mode with its own mitigation, and each is easy to overlook if you think of memory as simply storage.

Poisoning: untrusted content stored, then recalled as trusted instruction.
Staleness: old policy or structure overrides current truth.
Cross-user leakage: one user's context or results surface for another.
Over-promotion: raw observations become durable facts without validation.
Under-attribution: a claim is remembered but its source and confidence are lost.
Procedural drift: an old method is followed after tools or policies changed.

How much to trust a memory

Not every memory deserves equal belief, and the fix is to make trust explicit rather than assumed. A useful scale runs from a raw observation, to an attributed fact with a known source, to a validated fact confirmed by a check, to trusted knowledge safe to act on.

An item should only move up the scale by earning it. Retrieval can then respect the level: raw observations inform but never trigger action, while only validated knowledge is allowed to.

A memory item should climb from raw observation to trusted knowledge only by earning each step.

Stale and poisoned look like truth

The deepest problem is that the model cannot tell a poisoned or stale memory from a good one. What it sees is a confident, plausible recollection; whether that matches reality is something only the runtime can know, by tracking source and freshness.

So the gap between what the model sees and what is actually true is invisible to the model itself. Closing it is the runtime's job, not the model's.

A recalled memory looks equally confident whether it is accurate, stale, or poisoned.

Label everything you remember

The unifying defense is to attach a label to every memory: where it came from, how much it can be trusted, when it was written, and who it belongs to. Then a deterministic rule can decide what any given memory is allowed to influence, rather than leaving it to the model's judgment.

This is information-flow control applied to recall. It is what turns memory from an open attack surface into a governed one, where an item's origin travels with it and constrains what it can do.

⚠️

Recall is an input path. Treat anything read back from memory as untrusted until its label proves otherwise; a confident recollection is not proof.

What a practitioner decides

Defending memory means deciding, deliberately, that recall is an untrusted input path, that every item carries source, trust, freshness, and ownership, and that those labels govern what a memory may influence. It is more work than a simple key-value store, and it is what keeps an agent from getting more dangerous as it remembers more.

The question for any recalled item is not do we remember this. It is: where did it come from, how fresh is it, whose is it, and what is it allowed to do?

Key takeaways

Memory is an attack surface: stored content can become tomorrow's trusted instruction.
Memory poisoning works through unlabeled recall, not the write itself.
Account for staleness, cross-user leakage, over-promotion, under-attribution, and procedural drift.
Make trust explicit: raw observations inform, only validated knowledge acts.
The model cannot tell a poisoned or stale memory from a good one; the runtime must.
Label every memory with source, trust, freshness, and owner, and let labels govern influence.

Glossary

Memory poisoning: Storing untrusted content that is later retrieved and treated as trusted instruction.
Staleness: Remembered information that is out of date and no longer matches reality.
Cross-user leakage: Memory from one user, tenant, or role surfacing improperly for another.
Over-promotion: Treating a raw observation as a durable fact without validating it.
Under-attribution: Remembering a claim while losing its source, timestamp, or confidence.
Procedural drift: Following a stored method after the tools or policies it assumed have changed.
Information-flow control: Labeling data and letting the labels govern what it is allowed to influence.