The Hidden-Assumption Threat Register: Auditing Your Own Stack

The most dangerous defect in a system is rarely a bug in the ordinary sense. It is a premise the designers held so firmly that they never wrote it down: that the proxy and the origin will parse the same request the same way; that the file a program checked is the same file it later opens; that a wiped disk holds no secrets; that a reward function measures what its authors actually wanted. Named rules get review, tests, and alarms. Unnamed assumptions get none of that, because you cannot guard what you have not articulated. This article gathers nine well-documented classes of below-the-radar attack, extracts the single silent assumption each one violates, and turns the collection into a working audit artifact — a register you can run against your own architecture to surface the premises an attacker is already counting on.

The premise nobody writes down

A security control is only as strong as the assumptions it silently depends on. When Sassaman, Patterson, Bratus and colleagues framed what they called the science of insecure design, their central observation was that ad-hoc input handling turns every parser into an unintended interpreter, and that the mismatch between what a designer imagined the input language to be and what the code actually accepts is where exploitation lives. That gap is not a coding mistake you can lint away; it is an unstated assumption — 'both ends agree on the grammar' — that no one ever committed to a specification.

The pattern generalizes far beyond parsing. A time-of-check to time-of-use race assumes the world holds still between a check and the action it authorizes. A constant-time comparison assumes the hardware underneath keeps no timing secret. Secure deletion assumes the medium forgets. A reward function assumes the number it optimizes is the thing its authors care about. In each case the code is correct against its spec; the spec itself rests on a premise that an attacker, and only an attacker, thinks to question.

This asymmetry is the whole game. Defenders reason about the system they described. Attackers reason about the system that actually exists, including every premise the description left implicit. The remedy is not more rules layered on top of the same blind spots — it is a disciplined practice of making the blind spots visible, one assumption at a time, so that each can be tested, monitored, or designed away.

Nine classes, nine silent assumptions

The below-the-radar attack families share a structure: each violates exactly one premise that a reasonable designer would have taken for granted. Laid side by side, they form a taxonomy you can reason about rather than a bag of unrelated tricks. The value of the taxonomy is diagnostic — if you can locate which assumption a component depends on, you can predict which class of attack it invites.

The register below states each class as (the class, the silent assumption it breaks, the observable tell that the assumption is being probed). The assumption column is the part teams almost never articulate on their own, and it is the part that makes the whole collection reusable. The tell column is what a defender can actually instrument: a divergence, a timing spread, a flipped bit, a metric that climbs while the true outcome falls.

None of these are exotic. Every entry traces to a public, primary result — request smuggling and desync, TOCTOU file races, timing and microarchitectural leakage, Rowhammer, air-gap covert channels, trusting-trust build backdoors, cold-boot remanence and model memorization, subliminal channels, and incentive gaming. What is new here is not any single attack but the insistence on naming the assumption each one exploits, because that name is the hook a control hangs on.

Why one attack reaches impact without breaking a rule

The reason these classes evade ordinary defenses becomes concrete when you draw the path from primitive to impact. The attacker's goal is not to trip an alarm; it is to reach a real effect while every action stays inside the set the system considers permitted. That is only possible when an unnamed assumption bridges the gap between 'allowed' and 'harmful'.

Consider the shape shared by request smuggling, a TOCTOU race, and reward gaming. In each, no individual step is forbidden: the front end forwards a request it deems valid; the program opens a path it was allowed to open; the agent takes actions inside its policy. The harm emerges from the seam — the front end and back end disagree, the world moved between check and use, or the metric and the goal diverged. Because the seam was never named, no rule guards it.

This is why detection has to move from the action to the assumption. An allow-list of actions cannot catch an attack composed entirely of allowed actions. What can catch it is instrumenting the tell: watching for the parser disagreement, the interleaving-dependent outcome, the metric that climbs while the true outcome sinks. The attack tree makes the escape route explicit so a defender can place a tripwire where the attacker actually crosses.

Hunting the assumptions in your own stack

Naming assumptions is a repeatable procedure, not an act of inspiration. It runs in four stages, and its output is the register instantiated for your architecture. The first stage enumerates trust boundaries: every place where data or control crosses from something you do not control into something you do. Boundaries are where assumptions concentrate, because a boundary is precisely a claim that the other side behaves as expected.

The second stage names the assumption at each boundary in plain language — 'the upstream proxy and this service parse Content-Length and Transfer-Encoding identically', 'the file we validated is the file we open', 'the model will not emit its training data verbatim'. Writing it as a falsifiable sentence is the point; a premise you can state, you can test. The third stage identifies the tell: the observable signal that would appear if the assumption were being probed. The fourth stage designs an assumption-free control — one that holds even if the premise is false, such as canonicalizing input to a single interpretation, or binding a check and its use to the same handle so no interleaving can separate them.

The method deliberately mirrors the differential-testing tradition behind parser research and the invariant-checking tradition behind file-race analysis. Both work by refusing to trust that two views of the same thing agree, and instead measuring whether they do. Applied across a whole stack, the four stages convert a vague unease about 'edge cases' into a concrete, reviewable list of premises and the controls that survive their failure.

Where these seams converge on the agent stack

An autonomous LLM agent concentrates almost every one of these assumptions into a single, fast-moving loop, which is what makes the register urgent rather than academic. The agent ingests untrusted content — web pages, tool outputs, retrieved documents — normalizes it, reasons over it in a trusted context, and then takes real actions through tools. Each arrow in that flow is an assumption waiting to be named.

The parser-differential premise reappears as the gap between how a guardrail scans text and how the model actually tokenizes and interprets it; a subliminal or smuggled instruction can survive the scan and land in the model's context. The TOCTOU premise reappears between the moment a policy checks an action and the moment the tool executes it. The memorization premise reappears when a model emits secrets from its training or retrieval store that everyone assumed were access-controlled. The incentive premise reappears when an agent optimizes a proxy objective and drifts from the operator's intent. The trust boundary that matters is the crossing from untrusted input into the trusted action plane — and OWASP's LLM guidance, MITRE ATLAS, and the NIST AI Risk Management Framework all point at exactly this crossing as the locus of agentic risk.

Mapping the register onto the agent stack yields a small, high-leverage set of controls: canonicalize and spotlight untrusted input so the scanner and the model see the same thing; bind policy checks to the executed action so nothing changes in between; scope tool tokens to least privilege so a smuggled instruction cannot reach a dangerous effect; and track the true outcome beside any optimized metric so incentive drift shows up as a divergence rather than a surprise.

Triaging assumption risk

A register with every assumption weighted equally is a wish list, not a plan. Two factors set priority: how likely an assumption is to be violated in practice, and how much blast radius its failure carries. An assumption that is easy to probe and gates a dangerous action belongs at the top; an assumption that is hard to violate and gates a harmless one can be monitored rather than immediately re-engineered.

The two-by-two below is deliberately coarse, because precision here is false comfort. What matters is forcing an explicit decision for each entry: fix now, add a tripwire, hold a contingency, or accept and watch. Assumptions whose failure cannot be locally detected — build-chain integrity, hardware fault resistance, whether a model memorized a secret — deserve extra weight, because their tell is weak or absent, so a preventive control beats a detective one.

This is where the register stops being a document and becomes an operating discipline. Each cell implies an action, each action has an owner, and the highest-risk cell — likely to break and catastrophic when it does — is where scarce engineering time is spent first. The output is not a feeling of thoroughness but a ranked, owned list of premises and the controls that make them safe to rely on.

Building the register into the SDLC

An artifact that lives in one engineer's head decays the moment they change teams. To survive, the register has to become a gate in the software development lifecycle: a design review does not pass until every new trust boundary has a named assumption, a monitored tell, and a control that holds if the assumption fails. This is the same move that made threat modeling durable — turning a one-time insight into a recurring checkpoint.

The gate is lightweight by design. For most components the assumption is already covered by a class-standard control, and the review simply confirms it: input is canonicalized, checks are handle-bound, comparisons are constant-time, build provenance is attested, tool tokens are least-privilege. The gate earns its cost on the components where no standard control exists, forcing the team to either engineer one or consciously accept the risk and record why.

Over time the register becomes a living map of where the system trusts things it cannot verify, and the controls that make that trust survivable. It does not promise to catch every novel attack — nothing does. It promises something more useful: that the next below-the-radar technique will land on an assumption you have already named, weighted, and either defended or knowingly accepted, rather than on a blind spot you never knew you had.

Practitioner Toolkit

Copy-paste, strictly defensive artifacts you can use today. Nothing here attacks a real system.

COMPONENT: <name>
BOUNDARIES (untrusted -> trusted crossings):
  1. <source> --> <sink> : data = <what>
FOR EACH BOUNDARY:
  ASSUMPTION (falsifiable sentence): "<both ends agree that ...>"
  IF FALSE, IMPACT: <blast radius>
  LIKELIHOOD IT IS FALSE: <low|med|high>
  TELL (observable signal): <divergence | latency | flipped bit | metric>
  WHERE LOGGED: <sink / dashboard>
  ASSUMPTION-FREE CONTROL: <canonicalize | handle-bind | constant-time | attest | least-privilege>
  DISPOSITION: <fix-now | tripwire | contingency | accept-and-watch>  OWNER: <name>

REGISTER = load("assumption-register.yaml")  # name, tell, control, owner

function audit(observation):
  for entry in REGISTER:
    signal = measure_tell(entry.tell, observation)   # e.g. parser_agreement, latency_spread
    if signal.violates(entry.assumption):
      emit_alarm(entry.name, owner=entry.owner, evidence=signal)

function measure_tell(tell, obs):
  # MOCK: returns a canary-safe reading; replace with real, read-only probes
  match tell:
    case "parser_agreement": return compare(parse_A(obs), parse_B(obs))
    case "latency_spread":   return timing_variance(obs)   # constant-time expected
    case "metric_vs_goal":   return divergence(obs.metric, obs.true_outcome)
  return NO_SIGNAL

Glossary

Hidden-Assumption Threat Register

A living artifact that lists, for each component, the silent premise it relies on, the observable tell of that premise being probed, and a control that holds if the premise fails.

Silent assumption

A premise a design depends on but never states in its specification, and therefore never tests, monitors, or defends.

Tell

An observable signal — a divergence, timing spread, flipped bit, or metric-versus-outcome gap — that indicates a silent assumption is being probed.

Assumption-free control

A defense that remains sound even if the assumption it protects is false, such as canonicalizing input to a single interpretation or binding a check to the exact action it authorizes.

Trust boundary

Any point where data or control passes from something you do not control into something you do; boundaries are where assumptions concentrate.

Parser differential

A security-relevant disagreement between two components about what the same input means, exploited to smuggle intent past a checker into an executor.

TOCTOU

Time-of-check to time-of-use — a class of race where the state validated by a check no longer holds when the authorized action runs.

References

1. Sassaman, Patterson, Bratus et al., The Science of Insecure Design / LANGSEC (2011)
2. Thompson, Reflections on Trusting Trust (CACM, 1984)
3. Kim et al., Flipping Bits in Memory Without Accessing Them — Rowhammer (ISCA, 2014)
4. Kocher et al., Spectre Attacks: Exploiting Speculative Execution (IEEE S&P, 2019)
5. Carlini et al., Extracting Training Data from Large Language Models (USENIX Security, 2021)
6. Daian et al., Flash Boys 2.0: Frontrunning and MEV (IEEE S&P, 2020)
7. Amodei et al., Concrete Problems in AI Safety (2016)
8. NIST AI Risk Management Framework (AI RMF 1.0, 2023)
9. OWASP Top 10 for LLM Applications
10. MITRE ATLAS — Adversarial Threat Landscape for AI Systems