Trusting the Model the Least: Scaling Containment with Capability

A security team asked to deploy an AI agent usually hears one of two answers about how careful to be. One says the model is the same one everybody uses, so ordinary application security will do. The other says the model is powerful and unpredictable, so everything must be locked down. Neither answer helps anyone decide what to build first. The useful observation is that risk from an agent comes from two things multiplied together: what the model is able to do, and what the deployment lets it do. A highly capable model that can only draft text for a human to read poses a different problem from a modest model that can move money. A sensible containment plan therefore has to be indexed by both. This article turns that observation into a decision: a grid of capability against authority, with the controls each cell requires, a way to place a deployment on the grid, and the order in which to build. It is the practical answer to the principle in this piece's title, that the more capable the model, the less the design should rely on trusting it.

What frontier safety frameworks already say

Developers of the most capable models have converged on a common pattern for managing severe risks. A December 2025 survey found that twelve developers had published frontier AI safety policies, and that nearly all of them define capability thresholds, evaluate models against those thresholds before and after deployment, require stronger model-weight security and deployment mitigations as thresholds are approached, and commit to halting development or deployment if the required mitigations cannot be put in place (METR, 2025). The pattern follows earlier proposals that dangerous-capability evaluations should drive decisions about training, deployment and security (Shevlane et al., 2023), and that frontier models warrant oversight proportionate to their capabilities (Anderljung et al., 2023).

Some frameworks go further and address the possibility that the model itself is the problem. One published framework, as summarised in the same survey, describes an illustrative threshold at which a model's situational awareness and stealth are such that, even when its outputs and scratchpads are monitored, the developer cannot detect or rule out the model significantly undermining human control. Another lists undermining safeguards, including colluding with oversight models or sabotaging safeguards over time, as a research category that needs more work before it can be measured (METR, 2025).

These frameworks are written for developers. Their mitigations are mainly about protecting model weights and preventing misuse of a model by people. They say much less about the organisation that deploys a model as an agent and grants it authority over its own systems. That deployer is the one who decides what the agent can touch, and the deployer is the audience for the rest of this article.

Two axes, not one

Capability measures what a model could do if it tried: how well it plans over many steps, how effectively it uses tools, how much it understands about its own situation, and whether evaluations can rule out deliberate deception or sabotage. Authority measures what the deployment allows the model to do: whether it can only read and advise, whether it can take actions that can be undone, or whether it can take actions that are irreversible, privileged or external, such as moving money, changing production systems, issuing credentials or sending messages outside the organisation.

The two are independent, and the security principle that connects them is old. Every program and every user should operate with the least privilege necessary to complete the job (Saltzer and Schroeder, 1975). For a model, least privilege means that authority should grow only when the controls around it have grown too, and that authority should be granted more cautiously as capability rises, because a more capable model is better at finding uses for any authority it holds that its designers did not intend.

This series' earlier findings sit naturally on these axes. Insider controls such as separation of duties and least privilege act mainly on authority. A second-model check is worth something only if its independence is measured, because similar models tend to make similar mistakes (Goel et al., 2025). Control protocols, evaluated against a model deliberately trying to subvert them, become necessary as capability rises (Greenblatt et al., 2024). A brake that the agent cannot reach matters more as authority rises, especially since some models have resisted shutdown in controlled tests when tasks were incomplete (Schlatter et al., 2025). Disclosure applies everywhere, but what it must capture grows with both.

How the safety argument changes as capability rises

A useful way to see why containment must scale is to ask what argument justifies a deployment as safe. Work on safety cases for advanced AI identifies several categories of argument: that the system is unable to cause the harm in question; that control measures would prevent the harm even if the system tried; that the system can be trusted not to try; and, eventually, deference to credible AI advisors (Clymer et al., 2024).

For a model well below any concerning capability, an inability argument may be enough: it simply cannot do the dangerous thing, so ordinary controls suffice. As capability rises, inability can no longer be shown, and the argument has to shift to control: the model might be able to cause harm, but the surrounding system would catch or stop it. A sketch of such a control safety case for an agent deployed inside a developer's own organisation shows what that argument requires, including red-team evaluations of the control measures that give the red team the model's own capabilities (Korbak et al., 2025). Trustworthiness arguments, which rely on the model's intentions rather than its constraints, are not yet mature enough to carry a high-stakes deployment on their own.

This is the core reason for trusting the model the least. As capability rises, the justification for safety has to move away from the model's limitations and towards the constraints around it, because the limitations are exactly what capability erodes.

Placing a deployment on the grid

The matrix in the next section uses three tiers on each axis. On capability: C1, assistive, where available evaluations and the developer's published reports indicate the model is well below concerning thresholds and shows no evidence of strategic deception; C2, capable agent, where the model can carry out long multi-step tasks with tools and shows situational awareness in tests, but evaluations do not indicate deliberate subversion of oversight; and C3, control-relevant, where the developer's evaluations, independent research or the deployer's own testing cannot rule out the model deliberately undermining monitoring or other controls.

On authority: A1, read and advise, where the agent's outputs reach the world only through a human who decides; A2, reversible action, where the agent acts directly but every action can be undone within a bounded scope; and A3, irreversible or privileged action, where at least one action cannot be undone, or grants access, or leaves the organisation.

Two rules keep placement honest. First, place on the highest authority any single tool grants, not the typical action; an agent that can send one external email is in A3 even if it mostly reads. Second, re-place whenever the model version changes, because capability is a property of the model and the deployment inherits it.

The containment matrix

The matrix below states, for each cell, the controls that should be in place before the deployment goes live. Each cell includes everything in the cells to its left and above it; the entries list what is added. The controls draw on the five measures this series has examined: insider controls such as least privilege, separation of duties and two-person approval; measured independence of any second model used as a check; a control protocol evaluated against a deliberately adversarial red team; a brake outside the agent's reach with defined behaviour for half-finished work; and a disclosure record whose evidence is captured at the action gateway.

Reading the matrix by row shows that capability changes the kind of control required: from ordinary access control, to monitoring whose effectiveness is measured, to protocols designed on the assumption that the model is working against them. Reading it by column shows that authority changes how much must be guaranteed before an action happens rather than detected after it. The bottom-right corner is where both pressures meet, and it is where the matrix says the honest answer is that today's tools do not yet justify the deployment.

The order to build

The matrix says what a cell needs; it does not say what to build first when starting from little. The order matters because several controls depend on others, and building them out of order produces controls that cannot be tested.

Start with the authority boundary: an action gateway through which every tool call passes, with least privilege and short-lived credentials. Nothing else can be enforced or observed without it. Second, capture evidence at that gateway, recording model version, configuration fingerprint, inputs seen and actions attempted, because the brake, the monitor and the disclosure record all depend on that evidence. Third, build the brake, since it acts on the gateway and credentials already in place. Fourth, measure the independence of any second model before relying on it as a check. Fifth, evaluate the full control protocol against a red team, which is the most expensive step and only meaningful once the previous four exist to be evaluated.

This order also gives a natural ceiling at each stage. A deployment that has completed only the first three steps has earned at most the C1 row; one that has completed all five may claim C2, and may consider C3 with read-only authority.

Where today's tools run out

Three cells deserve particular attention because current practice does not yet supply what they need.

The C3 by A3 cell is the clearest. If evidence cannot rule out a model deliberately undermining oversight, and the deployment allows irreversible or privileged actions, then any safety argument has to rest on control, and control evaluations for that setting are still a research programme rather than an established practice. The developer frameworks surveyed acknowledge the same frontier: they treat undermining of safeguards as something still to be measured (METR, 2025). The honest recommendation is not to grant that authority.

The C3 by A2 cell is borderline. A control safety case is possible in principle (Korbak et al., 2025), but deployers rarely have the red-team capacity to produce one, and there is no shared standard for what such a case must show. The C2 by A3 cell depends on measured independence between checking models, and there is no agreed method or threshold for that measurement. These are precisely the places where shared industry standards would let a deployer meet a known bar rather than invent one: standard control evaluations for deployers, standard independence measurements, and standard expectations for brake behaviour and incident records.

Limitations and threats to validity

The matrix is a synthesis, not an adopted standard, and its tier boundaries are judgements. Capability tiers depend on evaluations that may underestimate what a model can do, and the developer frameworks themselves treat any single evaluation as a lower bound rather than a ceiling. A deployer that relies on a developer's published evaluations inherits their limits.

The controls in each cell are necessary conditions in this analysis, not sufficient ones. A deployment that satisfies its cell can still fail through implementation errors, novel attacks or interactions between agents that the matrix does not model.

The frameworks summarised here change frequently. The survey cited reflects policies as of late 2025, and both the frameworks and the research on control evaluations are moving quickly, so the gaps named here may narrow, or new ones may appear.

Practitioner Toolkit

Copy-paste, strictly defensive artifacts you can use today. Nothing here attacks a real system.

DEPLOYMENT: 
MODEL PROVIDER AND VERSION: 

CAPABILITY TIER
  Can evidence rule out deliberate subversion of oversight? yes / no  -> no = C3
  Long multi-step tool use or situational awareness in tests? yes / no -> yes = C2, no = C1
  Evidence used (developer reports, own tests): 

AUTHORITY TIER (highest single tool, not typical use)
  Any irreversible, privileged or external action? yes = A3
  Any direct action that can be undone? yes = A2
  Otherwise A1
  Tool granting the highest authority: 

CELL: C_ x A_
Required controls for this cell (from the matrix, including cells left and above): 
Controls in place: 
Controls missing: 
Gap cell? yes / no  -> yes = reduce authority or hold
Signed off by: 

Glossary

Capability tier

A classification of what a model could do if it tried, from assistive through capable agent to control-relevant.

Authority tier

A classification of what a deployment allows the model to do, from read and advise through reversible action to irreversible or privileged action.

Frontier AI safety policy

A developer's published protocol linking capability thresholds to required security and deployment safeguards, with conditions for halting.

Capability threshold

A level of model capability at which a framework requires stronger safeguards before training or deployment continues.

Safety case

A structured argument, supported by evidence, that a system is acceptably safe in a given deployment.

Control safety case

A safety case arguing that control measures would prevent unacceptable outcomes even if the model tried to subvert them.

Least privilege

The principle that every component should hold only the access it needs for its task.

Action gateway

A control point through which every tool call an agent makes must pass, where policy is enforced and evidence is recorded.

References

1. METR, Common Elements of Frontier AI Safety Policies (December 2025 update)
2. Shevlane et al., Model Evaluation for Extreme Risks (arXiv 2305.15324, 2023)
3. Anderljung et al., Frontier AI Regulation: Managing Emerging Risks to Public Safety (arXiv 2307.03718, 2023)
4. Clymer, Gabrieli, Krueger and Larsen, Safety Cases: How to Justify the Safety of Advanced AI Systems (arXiv 2403.10462, 2024)
5. Korbak, Clymer, Hilton et al., A Sketch of an AI Control Safety Case (arXiv 2501.17315, 2025)
6. Greenblatt, Shlegeris, Sachan and Roger, AI Control: Improving Safety Despite Intentional Subversion (arXiv 2312.06942, ICML 2024)
7. Goel et al., Great Models Think Alike and this Undermines AI Oversight (arXiv 2502.04313, 2025)
8. Schlatter, Weinstein-Raun and Ladish, Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs (arXiv 2509.14260, 2025)
9. Saltzer and Schroeder, The Protection of Information in Computer Systems, Proceedings of the IEEE 63(9), 1975