An Evidence Budget for Agents

Every time an agent acts, someone eventually has to answer a question: how do we know it was allowed to do that? The honest answer is not a feeling of trust but a trail of evidence — signatures, attestations, provenance, approvals — that lets a person or another system verify the action after the fact. But evidence is not free. Gathering and checking it costs latency, complexity, and human attention. So the real design problem is proportion: spend almost no evidence on trivial reads, spend a lot on irreversible actions, and treat that trade-off as a budget you allocate on purpose rather than an afterthought you bolt on.

Proof should cost what the action is worth

Imagine demanding a notarized, independently witnessed record for every time an agent reads a public help article. The system would grind to a halt, and no one would be safer. Now imagine accepting the agent's own word that it was fine to wire a large payment. The system would be fast and catastrophically exposed. Neither extreme is defensible, which is the whole point: the right amount of evidence depends on what the action can do.

The guiding principle is proportionality. As the risk and irreversibility of an action rise, the strength of evidence required to justify it should rise with it. A read that changes nothing needs almost nothing; a config change needs a verifiable record; moving money or altering permissions demands the strongest proof you have. Drawn as a curve, evidence tracks risk — and any action where the two diverge is either a bottleneck or a hole.

What counts as evidence

Before you can budget evidence, you need to know what forms it takes. The useful ones share a property: they let a third party verify a claim without trusting the claimant. A cryptographic signature proves a message came from a specific key and was not altered. A provenance record traces where data or a decision came from. An attestation is a statement from an independent party vouching for a fact. And a human approval is a person deliberately accepting responsibility for a consequential step.

What does not count is the agent's unverifiable say-so. An agent asserting I checked and it was fine is not evidence; it is a claim awaiting evidence. The distinction matters because agents are fluent and confident, and confident language is easy to mistake for proof. Evidence is precisely the part of an action that survives the agent being wrong or compromised.

A ladder of proof strength

Not all evidence is equally convincing, and it helps to rank it. At the base sits self-assertion — the agent's own report, cheap and weak, fine for low-stakes actions. Above it sits signed provenance — a tamper-evident trail that ties an action to keys and prior state, harder to fake. At the top sits independent attestation — a separate party or system vouching for the fact, the strongest and most expensive because it does not depend on the actor at all.

The ladder is useful because it turns a vague call this needs more proof into a concrete choice of rung. A risky action does not just need more evidence; it needs stronger evidence — the kind that holds even if the agent is fully compromised. Climbing the ladder trades cost for independence, and the budget decides how high each action must climb.

Corroboration beats assertion

A single source, however trusted, is a single point of failure. The cheapest way to strengthen evidence without reaching for the most expensive rung is corroboration: two or more independent sources that agree. A tool's own log says the transfer happened; an independent monitor confirms the same amount moved; a human sign-off records that someone approved it. Any one of them can be wrong or forged; their overlap is hard to fake.

Independence is the load-bearing word. Two logs written by the same compromised component are not corroboration — they are one source counted twice. Real corroboration comes from sources that would have to be compromised separately to agree on a lie. Where an action's risk is high but full attestation is impractical, overlapping independent evidence is often the pragmatic middle of the budget.

From raw signals to admissible evidence

Systems produce an enormous amount of raw signal — logs, traces, metrics, messages. Very little of it is actually usable as evidence. Turning the flood into something you can rely on is a narrowing process: of all raw signals, only some are verifiable rather than mutable; of those, only some are relevant to the action in question; and of those, only some are admissible — captured in a form that survives scrutiny and cannot be quietly edited.

Recognizing this funnel changes what you build. It is not enough to log a lot; you have to ensure that the small fraction which matters is verifiable, relevant, and tamper-evident by design. An evidence budget spent on volume produces noise; the same budget spent on making the critical few signals admissible produces proof.

Spending the budget wisely

With a ladder, corroboration, and a funnel in hand, the budget becomes a set of explicit rules: cheap self-assertion for reversible, low-impact actions; signed provenance for anything that changes state; corroboration or attestation for the irreversible and the sensitive; and a human on the loop where a machine should not be the last word. The aim is not maximum evidence everywhere — that is a tax on every action — but matched evidence, where cost tracks consequence.

This is also where the topic connects to decisions leaders actually make. Over-proving frustrates users and invites people to route around the controls; under-proving leaves you unable to answer the only question that matters after an incident. A well-set evidence budget is what lets an organization say, precisely and defensibly, that a given agent action was authorized — without making every action expensive.

Where evidence budgets fail

The failure modes are distinctive. The first is proof theater — collecting impressive-looking logs that no one can actually verify, so the evidence evaporates under scrutiny. The second is self-signed everything — treating the agent's own attestations as independent, which counts one source many times. The third is a budget that never rises: applying the same light evidence to a money transfer as to a document read, so the riskiest actions are the least justified.

The tie to agent risk is direct. A compromised or manipulated agent will produce confident, plausible claims about what it did and why it was allowed. An evidence budget is what refuses to accept those claims at face value for consequential actions, demanding proof that holds even if the agent is lying. Treat unverifiable evidence and flat budgets as defects — the point of evidence is to be worth something exactly when the agent cannot be trusted.

What builders and leaders decide

For engineers, an evidence budget turns into a policy: classify actions by risk and irreversibility, attach a required rung of proof to each class, prefer independent corroboration where full attestation is costly, and make the critical signals tamper-evident and admissible by design. It composes with the rest of this series — intent-bound grants say what was permitted, delegation records say who acted for whom, and the evidence budget says how strongly each of those claims must be proven.

For leaders, the payoff is accountability that scales. Every consequential action carries proof proportional to its stakes, so incidents can be reconstructed and responsibility assigned without drowning routine work in ceremony. That balance — defensible when it matters, invisible when it does not — is what makes an agent system trustworthy to run at scale.

• Classify actions by risk and irreversibility, and attach a required strength of evidence to each class.
• Prefer independent corroboration when full attestation is too costly — and check the sources are truly independent.
• Make the critical few signals verifiable, relevant, and tamper-evident by design, not just voluminous.
• Reject the agent's unverifiable say-so as evidence for consequential actions; require proof that holds if it is compromised.

Glossary

Evidence budget

A deliberate allocation of proof strength across actions, so cost tracks the consequence of each action.

Attestation

A statement from an independent party or system vouching for a fact, not dependent on the actor.

Provenance

A record of where data or a decision came from, used to trace and verify how an action arose.

Non-repudiation

A property, usually from signatures, that stops an actor from plausibly denying it took an action.

Corroboration

Agreement between two or more genuinely independent sources, making a shared claim hard to fake.

Admissibility

Whether a piece of evidence is captured in a form that survives scrutiny and cannot be quietly altered.

Self-assertion

An agent's own report of what it did — cheap and weak evidence, acceptable only for low-stakes actions.

Proof theater

Impressive-looking evidence that cannot actually be verified, giving false assurance under scrutiny.