Revocation Drills: Measuring When Every Verifier Says No

Ask most teams how quickly they can cut off a compromised agent and you will get a number. Ask how they know, and the answer is usually a design document: the deny-list refreshes every minute, tokens last ten minutes, so revocation takes at most ten. That is a claim about intended behaviour, and the services that actually decide whether a request goes through do not read design documents. Some cache an earlier decision. Some validate a signed token locally and never ask anyone whether it was revoked. Some belong to a partner whose caching policy you have never seen. The only way to know when a revoked agent stops being trusted is to revoke one on purpose and watch. This article treats that as a measurement method in its own right: the revocation drill.

Why a revocation design is a claim, not evidence

Revocation is the act of causing a credential that has not yet expired to be rejected. The rejection is done by verifiers: every gateway, application programming interface, resource server or tool service that checks a credential before acting on it. The time from the decision to revoke until the last relevant verifier refuses is the revocation latency, and the interval during which a revoked agent can still act is the residual window. A system's guarantee is set by its slowest verifier, not its fastest.

Revocation designs usually specify that latency from the components a team controls: the refresh interval of a deny-list, the lifetime of short-lived tokens, the polling cadence of a gateway. Each is a reasonable input, but the sum is a prediction. It assumes every verifier consults fresh state, that no verifier caches an earlier allow decision longer than intended, that delegated credentials die with their parent, and that services outside the organisation behave as the design imagines. None of those assumptions is visible from the design itself.

Measurement closes the gap between prediction and behaviour. Security programmes already distinguish discussion from demonstration: NIST SP 800-84 separates tabletop exercises, in which people talk through a plan, from functional exercises, in which personnel validate plans and operational readiness in a simulated operational environment. A revocation drill is a functional exercise for one specific promise, that a decision to revoke an agent takes effect everywhere within a stated time. Its output is not a pass or a fail but a set of timings, one per verifier, that can be compared with the promise.

Agents make the measurement more urgent than it is for human sign-ins. An agent acts at machine speed for the whole residual window, often holds credentials for several services at once, and may have delegated narrower credentials to sub-agents. A residual window that would cost a human attacker a few manual actions can cost an agent hundreds of tool calls.

Where a revoked credential lingers

Before a drill can measure anything, it needs a map of where an agent's authority can outlive the decision to remove it. Each layer below can hold a credential open independently, so a drill that probes only one of them measures only that layer.

The first layer is the issuer. If the agent's credentials are short-lived and renewed continually, stopping renewal ends authority within one credential lifetime, but not sooner. The second is the verifier's own validation path. A verifier that validates a self-contained signed token, such as a JSON Web Token defined in RFC 7519, checks the signature and expiry locally and never contacts the issuer, so it will accept the token until it expires unless it also consults a revocation list. A verifier that uses OAuth token introspection, defined in RFC 7662, asks the authorization server whether a token is active, but the standard explicitly allows the answer to be cached by the protected resource at the cost of liveness of the information.

The third layer is the partner. When an agent calls a service run by another organisation, that service's validation and caching policy is set by the partner. It may cache introspection results for longer than your budget, validate tokens locally, or hold a session that outlives the token that created it. RFC 7009 gives clients a standard way to tell an authorization server that a token is no longer needed, but whether a resource server notices depends entirely on how that resource server validates tokens.

The fourth layer is delegation. Agents that mint narrower credentials for sub-agents or tool servers create children whose validity may not be tied to the parent. Unless child validity is derived from the parent, revoking the parent leaves the children working until they expire. The fifth layer is the agent itself. A running agent may hold a token in memory, retry a refused call with backoff so that a refusal looks like a transient error, or fall back to a second credential it was given for resilience. None of these show up in a design diagram, and all of them extend the residual window in practice.

The table summarises the layers, what holds each one open, and what typically bounds it. The bounding mechanism is what a drill verifies; the holding behaviour is what it is designed to catch.

The drill method

A revocation drill revokes a dedicated test agent while it is running and measures when each verifier it depends on starts refusing it. Five design choices make the result trustworthy and safe.

First, use a dedicated test identity. The drill agent should be provisioned through the same issuing path, with the same credential types, lifetimes and delegation pattern as production agents of the class being measured, but it must not be a production agent and must hold only harmless permissions. Measuring a different kind of identity measures a different system. Revoking a real agent to measure it turns a drill into an outage.

Second, define one probe per verifier. A probe is a harmless request the test agent makes to a verifier on a fixed short interval, such as a read of a dedicated test object, chosen so that an accept and a refusal are unambiguous in the response. Probes run before the revocation to establish a baseline of successful calls, so that a later refusal can be attributed to the revocation rather than to an unrelated failure. Every verifier in the map should have a probe, including tool services and partner endpoints the agent normally reaches.

Third, timestamp the decision precisely. The drill records the moment the revocation is issued, using the same mechanism an incident responder would use, whether that is a deny-list entry, halted renewal, key invalidation or removal of the identity from its trust domain. Using a special drill-only mechanism measures a path no responder will ever take. Clocks on the probe host and the controller must be synchronised, or the timings are meaningless.

Fourth, record each verifier's first refusal and keep probing. The quantity of interest is time to first refusal per verifier. Probing continues after the first refusal, because a verifier made of several instances behind a load balancer can refuse on one request and accept on the next while its caches converge. Fifth, set a stop rule in advance: the drill ends when every verifier has refused consistently for a set period, or when a ceiling is reached, typically the longest credential lifetime on the path plus a margin. A verifier still accepting at the ceiling is a finding, not a reason to wait longer.

The drill also observes the agent itself. After the decision, does the test agent stop, or does it retry refused calls indefinitely, switch to a fallback credential, or continue using child credentials it minted earlier? Those behaviours are part of the residual window even when every verifier behaves correctly.

verifiers = load map of every verifier the test agent depends on
start probes(test_agent, verifiers, every 2 seconds)
wait until every verifier shows a stable baseline of accepts
T0 = synchronised_clock_now()
revoke(test_agent, mechanism = the one incident responders use)
record(decision_time = T0, mechanism)
while not stop_rule_met(ceiling = longest lifetime on path + margin):
    for each probe result r:
        record(verifier = r.verifier, time = r.time - T0, outcome = r.accepted)
    record(agent_behaviour = retries, fallback credentials, child credential use)
for each verifier v:
    first_refusal[v] = earliest refused time after T0
    stable[v] = refused on every later probe until the stop rule
report(first_refusal, stable, slowest verifier, comparison with budget)

Reading the result: four outcome classes

Raw timings become findings when each verifier is placed in one of four outcome classes. The classes matter more than the exact seconds, because they point to different fixes.

A verifier that refuses promptly and consistently, within the budget, is behaving as designed. A verifier that refuses only at expiry has no working revocation path: it ignores the decision and is bounded solely by the credential lifetime. That may be acceptable if the lifetime fits the budget, but it should be recorded honestly as expiry, not revocation. A verifier that is still accepting when the stop rule is reached is the most serious finding, because its acceptance is bounded by something the drill did not observe, such as a long-lived session or a cache with no ceiling. A verifier that refuses and then accepts again is flapping, the signature of inconsistent instances or caches converging slowly, and its effective latency is its last accept, not its first refusal.

An illustration, not a measurement, makes the reading concrete. Suppose a drill revokes a test agent with a ten-minute budget. The organisation's own gateway refuses after eight seconds. An internal tool service that caches introspection results refuses after four minutes. A partner analytics service refuses only when the agent's sixty-minute token expires. A sub-agent's child credential keeps working until its own expiry because it was not tied to its parent. The design document would have promised one minute. The measured revocation latency is sixty minutes, set by the partner and the child credential, and the drill has found both places to fix.

The slowest verifier sets the latency the organisation can honestly promise. A team can then decide, per verifier, whether to shorten it, contain it, or accept it with the reason recorded. What the team cannot do is report the fastest component's interval as the system's revocation time.

Verifiers you don't control

Partner services are a common place for a revoked credential to linger, and a drill is how a team finds out whether that applies to its own partners. A partner's cache cannot be configured from outside, but the length of time it can honour a revoked credential can be shaped.

The simplest lever is lifetime. Tokens issued for a partner's audience can be given a shorter lifetime than internal tokens, so that a partner that validates locally is bounded by a shorter expiry. The cost is more frequent issuance and more dependency on the issuer's availability. A second lever is binding the token to a key the agent holds, known as sender-constrained tokens, which the OAuth 2.0 Security Best Current Practice, RFC 9700, recommends. If the partner verifies that the caller possesses the key, invalidating that key on the agent side makes the token unusable even while the partner still considers it valid. This only helps if the partner actually performs the proof-of-possession check, which is itself something to verify.

A third lever is a signal. The OpenID Foundation approved the Shared Signals Framework and the Continuous Access Evaluation Profile, known as CAEP, as final specifications in September 2025. They let a transmitter send security event tokens, defined in RFC 8417, to receivers by push or by poll, using RFC 8935 and RFC 8936. CAEP defines events such as session revoked, token claims change and credential change. A partner that receives a session revoked event can drop its cached sessions for that subject. The specifications define the message, not the receiver's reaction, so a signal shortens the window only if the partner acts on it promptly, and a drill is the way to find out whether it does.

Vendors publish their own bounds, which are useful inputs but still claims. Microsoft's documentation for continuous access evaluation in Microsoft Entra, for instance, describes near real-time response to critical events while noting that latency of up to fifteen minutes might be observed because of event propagation. A drill that includes such a service measures the bound that applies to that organisation's configuration on that day.

Finally, partners can be included in drills with their agreement, against sandbox or test tenants where they exist, and expected revocation times can be written into agreements. Where a partner cannot be drilled, the honest record says so, and the residual window for that partner is the token lifetime the organisation issued.

• Shorter lifetimes for partner-audience tokens bound local validation.
• Sender-constrained tokens let key invalidation end use, if the partner checks possession.
• Shared Signals and CAEP events let partners drop sessions, if they act on them.
• Agreed revocation times and joint drills turn partner behaviour into evidence.

Drilling agents, not just tokens

Credential revocation drills exist for human sign-ins, but agents add behaviours that only appear when an identity is cut off in the middle of work. The drill should treat the test agent as a subject of measurement, not just a probe launcher.

Delegation is the first. If the test agent normally mints narrower credentials for sub-agents or tool servers, the drill should create those children before the decision and probe with them afterwards. A child that keeps working after its parent is revoked shows that child validity is not derived from the parent, the gap decommissioning is meant to close. The second is in-memory state. An agent that fetched a token at the start of a long task may keep presenting it for the token's whole lifetime, regardless of what the issuer now says, so the drill should run long enough to cover a realistic task length.

The third is error handling. Agent frameworks often retry failed calls with backoff, and some treat an authorisation failure as transient. A revoked agent that retries quietly for an hour produces no visible incident, and if one verifier later accepts a retry, the agent resumes work. The drill should record how the agent classifies the first refusal and whether it stops, escalates or keeps trying. The fourth is fallback. Agents built for resilience may hold a second credential, a different identity for a different tool, or a cached result they can act on. A drill that revokes one identity and watches the agent continue through another has found an authority path the revocation plan did not cover.

These observations turn the drill from a token test into an agent test. The question is not only whether every verifier refuses, but whether the agent's authority, by every path it has, actually ends.

From drills to audit evidence

A drill produces evidence only if its result is recorded in a form someone else can check. The record should capture what was revoked, how, and when; which verifiers were probed and which were not; each verifier's outcome class and timings; what the agent did; how the result compares with the budget; and what was changed as a result. Unprobed verifiers belong in the record explicitly, because silence about a verifier reads as coverage.

Cadence has two parts. Continuous checks, the kind that confirm every decommissioned identity has no surviving credentials, should run all the time, because invariants can break at any moment. Drills cannot run continuously, because each one revokes something, so they run on a schedule and on change: a regular interval such as each quarter, plus whenever a new verifier or partner is added, a revocation mechanism changes, or token lifetimes are adjusted. A timing measured before a configuration change says little about the system after it.

Kept over time, the records show a trend: the slowest verifier, the number of verifiers in each outcome class, and how long findings stayed open. That trend is what lets a security team tell an auditor something stronger than "we have a revocation process". It can say that on given dates, for a listed set of verifiers, revocation took effect within measured times, that findings were fixed, and that the next drill confirmed it.

An auditor can rely on that evidence only within its scope. It establishes the measured behaviour of probed verifiers under drill conditions on the drill dates. It does not establish behaviour for verifiers that were not probed, for configurations changed since, or for credential types the test agent did not use. Stating those limits in the record is what keeps the evidence honest.

Limitations and threats to validity

This is a methods article grounded in identity standards and exercise guidance; it reports no measured drill results, and the timings in its example are illustrative. Its central limitation is scope. A drill measures the verifiers it probes, under the load and configuration present on the day, using one test identity. Verifiers that were missed from the map, regions or instances the probes did not reach, and credential types the test agent did not use are outside what it shows.

Drills also carry their own risks and biases. A drill identity that differs from production agents, in lifetime, delegation pattern or issuing path, measures a different system. Probes that are too infrequent blur the timings, and unsynchronised clocks corrupt them. Partners may behave differently in a sandbox than in production, and a partner that knows a drill is running may not represent normal operation. Running drills against production verifiers needs care to avoid side effects, which is why probes must be harmless reads of dedicated test objects.

Finally, a drill measures time to refusal; it does not undo anything a revoked agent did during the residual window. Revocation ends authority but does not reverse consequences, so drills complement, rather than replace, limits on what an agent can do while its authority is in doubt.

Practitioner Toolkit

Copy-paste, strictly defensive artifacts you can use today. Nothing here attacks a real system.

drill:
  date: 2026-11-15
  test_identity: agent-drill-01   # dedicated, harmless permissions
  agent_class: procurement-agent
  credential_types: [access-token, child-token]
  mechanism: deny-list           # the path responders use
  decision_time_utc: '10:00:00'
  budget_seconds: 600
verifiers:
  - {name: api-gateway, outcome: refused-promptly, first_refusal_s: 8}
  - {name: tool-service, outcome: refused-promptly, first_refusal_s: 240}
  - {name: partner-analytics, outcome: refused-at-expiry, first_refusal_s: 3600}
  - {name: child-token-consumer, outcome: refused-at-expiry, first_refusal_s: 3600}
not_probed: [reporting-partner]   # state scope honestly
agent_behaviour: {retried_after_refusal: true, used_fallback_credential: false}
slowest_verifier: partner-analytics
within_budget: false
actions: [shorten partner token lifetime, derive child validity from parent]
next_drill: 2027-02-15

Glossary

Verifier

Any service that checks a credential before acting on a request, such as a gateway, application programming interface or tool service.

Revocation latency

The time from the decision to revoke a credential until the last relevant verifier refuses it.

Residual window

The interval after a revocation decision during which a revoked agent can still act.

Revocation drill

A planned exercise that revokes a dedicated test identity while it runs and measures when each verifier starts refusing it.

Probe

A harmless, repeated request from the test agent to one verifier whose response shows unambiguously whether the credential was accepted.

Time to first refusal

For one verifier, the time from the revocation decision until that verifier first rejects the test agent.

Flapping

A verifier that refuses a revoked credential and later accepts it again, usually because instances or caches are inconsistent.

Token introspection

A resource server asking the authorization server whether a token is still active, standardised in RFC 7662.

Self-contained token

A signed token, such as a JSON Web Token, that a verifier can validate locally without contacting the issuer.

Sender-constrained token

A token bound to a key the caller must prove it holds, so a copied token cannot be used without that key.

Security event token

A signed message describing a security event, standardised in RFC 8417 and used by the Shared Signals Framework.

CAEP

The OpenID Continuous Access Evaluation Profile, which defines events such as session revoked that one party can send another.

Functional exercise

An exercise that validates plans and operational readiness in a simulated operational environment, as described in NIST SP 800-84.

References

1. IETF RFC 7519, JSON Web Token (JWT)
2. IETF RFC 7009, OAuth 2.0 Token Revocation
3. IETF RFC 7662, OAuth 2.0 Token Introspection
4. IETF RFC 9700, Best Current Practice for OAuth 2.0 Security
5. IETF RFC 8417, Security Event Token (SET)
6. IETF RFC 8935, Push-Based Security Event Token (SET) Delivery Using HTTP
7. IETF RFC 8936, Poll-Based Security Event Token (SET) Delivery Using HTTP
8. NIST SP 800-84, Guide to Test, Training, and Exercise Programs for IT Plans and Capabilities
9. NIST SP 800-207, Zero Trust Architecture
10. OpenID Shared Signals Framework Specification 1.0 (Final)
11. OpenID Continuous Access Evaluation Profile 1.0 (Final)
12. Microsoft Learn, Continuous access evaluation in Microsoft Entra