Confidential Inference & Training — Confidential AI & Trusted Execution

Most security controls protect data when it is sitting on a disk or moving across a network. Artificial intelligence creates a harder moment: the data must be read, decoded, compared, multiplied, and transformed while a model runs. Confidential inference and confidential training are ways to reduce who and what can see those live computations. The goal is not magic secrecy. It is a narrower, testable trust boundary around the parts of an AI system where sensitive data, model weights, prompts, outputs, and training examples are exposed in memory.

Start with the three states of data

A useful security model begins with three simple states. Data at rest is stored somewhere, such as a database, file share, or model registry. Data in transit is moving between systems, such as from an application to an inference endpoint. Data in use is being processed by a central processing unit (CPU), graphics processing unit (GPU), or other accelerator.

Encryption at rest and encryption in transit are now common. They protect storage media and network paths, but they usually do not protect the moment when software must decrypt data to work on it. During that moment, data can exist in ordinary memory, where privileged software, a compromised administrator account, a malicious hypervisor, or a debugging tool may be able to inspect it.

Confidential computing tries to protect data in use. It creates a trusted execution environment (TEE), which is an isolated area where code and data can run with stronger protection from the rest of the system. Confidential inference uses this idea when a trained model answers requests. Confidential training uses it when a model learns from data.

📌

Plain meaning. Confidential computing is about reducing exposure during computation, not replacing every other security control.

What makes an execution environment trusted

A trusted execution environment is trusted because it can be isolated, measured, and proved. Isolation means the environment is separated from other code on the machine, including code with high privileges outside the environment. Measurement means the platform records what code and configuration were loaded. Proof means another party can verify that measurement before sending secrets.

That proof process is called attestation. In attestation, hardware-backed evidence says, in effect, this is the code that is running, on this kind of protected platform, with these settings. A relying party, such as a key management service, checks that evidence against policy. If the evidence matches, it releases a secret such as a decryption key.

This matters because confidential AI is not only about running in a special machine type. The valuable property is the chain of trust. Sensitive inputs should be released only after the system proves that the expected code is running inside the expected protection boundary.

An environment is trusted when it is isolated, measured, proved by attestation, and only then handed a key.

Confidential inference: protecting prompts, weights, and outputs

Inference is the act of using a trained model to produce an answer. A user sends a prompt or input, the model processes it, and the service returns an output. In many systems, the most sensitive moments are the prompt arriving, the model weights being loaded, and the generated output being prepared for return.

Confidential inference places the model-serving process inside a protected environment. The model weights may remain encrypted until attestation succeeds. User inputs can be encrypted on the client side or protected in transit, then decrypted only inside the trusted execution environment. Outputs can be re-encrypted before they leave that boundary.

This pattern is useful when prompts contain regulated data, confidential business plans, source code, health information, or personally identifiable information (PII). It is also useful when the model itself is valuable intellectual property. The point is to make it harder for cloud operators, infrastructure administrators, compromised host software, or neighboring workloads to inspect live prompts and model state.

⚠️

Not anonymity. Confidential inference can protect computation, but it does not by itself hide user identity, billing records, network metadata, or application logs.

The client sends an encrypted prompt; weights unlock only after attestation and output is re-encrypted before it leaves.

Confidential training: the harder version

Training is harder than inference because it touches more data for longer periods. A training job may read millions or billions of records, shuffle them, create temporary batches, compute gradients, update model weights, checkpoint progress, and write metrics. Each of those steps can expose data or derived information.

Confidential training aims to keep the training data, intermediate values, and model updates inside protected execution boundaries. In some designs, each data owner encrypts its data and releases keys only to an attested training job. In other designs, multiple organizations collaborate without giving each other direct access to raw data.

The protection boundary must include more than the training script. Data loaders, feature transformations, checkpoint writers, distributed training workers, GPU memory, temporary storage, and telemetry paths all matter. A single unprotected side path can undo the value of protecting the main compute process.

Protect raw training records before they enter the pipeline.
Verify the training code and container image before releasing data keys.
Keep checkpoints, gradients, and temporary files under the same confidentiality policy.
Limit metrics and samples, because they can leak examples from the training set.
Treat distributed workers as part of one trust decision, not as isolated islands.

Confidential training must protect more than the script: loaders, gradients, checkpoints, and GPU memory all leak.

What confidential AI does not solve

A protected execution environment is a strong tool, but it is not a complete AI security program. It cannot make a bad model accurate. It cannot prove that training data was collected lawfully. It cannot stop a user who is authorized to see an answer from copying that answer. It cannot remove the need for access control, network security, monitoring, governance, or incident response.

It also does not automatically solve model-level risks. A model may memorize rare training examples. A prompt injection attack may cause an application to misuse tools. A poorly designed retrieval system may send the wrong documents into a prompt. Confidential computing can reduce infrastructure exposure while those events happen, but it does not make the application logic safe.

Side channels deserve careful attention. A side channel is an indirect signal, such as timing, memory access patterns, resource use, or error behavior, that may reveal something about protected data. Modern confidential computing platforms reduce many risks, but high-value systems still need threat modeling, patching, and testing against realistic attackers.

⚠️

Boundary mistake. The most common design error is protecting the model server while leaving logs, caches, checkpoints, or feature stores outside the trust boundary.

Confidential AI shrinks infrastructure exposure, but it does not repair application logic or model-level risks.

The real decisions leaders and practitioners face

The first decision is whether the risk justifies the cost and complexity. Confidential inference and training can affect performance, instance choice, accelerator availability, debugging, observability, and deployment speed. They are most compelling when sensitive data must be processed in environments where the infrastructure operator, platform administrator, tenant neighbor, or collaboration partner should not be able to inspect it.

The second decision is what must be inside the protected boundary. For inference, that might be prompts, model weights, retrieval results, and tool responses. For training, it may include data preparation, feature engineering, gradients, checkpoints, and evaluation samples. Drawing this boundary is a business and engineering decision, not merely a product setting.

The third decision is how to prove and govern the design. In Microsoft environments, teams may use Microsoft Foundry for model development workflows, Microsoft Entra ID for identity, and Foundry Tools where managed AI capabilities are appropriate, while still requiring attestation, key release policies, logging rules, and data governance that match the threat model. The same principle applies in any platform: do not confuse a service label with a verified trust chain.

Who is not supposed to see the data: cloud operator, internal administrator, partner, tenant neighbor, or attacker?
Which assets need protection: prompts, embeddings, model weights, training records, gradients, checkpoints, or outputs?
What proof is required before secrets are released?
What performance overhead is acceptable for the protected workload?
What evidence will auditors, customers, or regulators expect?

Operating confidential workloads day to day

A confidential design is only as strong as its operating discipline. Images and code must be built reproducibly enough that measurements can be trusted. Attestation policies must be reviewed when software changes. Keys must be rotated and released only to approved measurements. Logs must be useful for investigation without recording the very secrets the system is meant to protect.

Operations teams also need a plan for failure. If attestation fails, the system should fail closed, meaning it refuses to release secrets rather than continuing in an unprotected mode. If a vulnerability is found in a runtime, driver, container image, or model-serving framework, teams need a way to revoke old measurements and approve patched ones.

Machine learning operations, often called MLOps, adds another layer. Models change, data changes, and pipelines are retrained. Confidential AI must be part of that release process, not an exception path handled by a small group after deployment.

A practical way to start

Begin with one high-value inference path rather than the whole AI estate. Pick a workload where the sensitive input is clear, the model-serving path is well understood, and the success criteria are measurable. Map every place the prompt, retrieved context, model weights, output, and logs go. Then decide which parts require protection in use.

Next, design the key release flow. The question is simple: what exact proof must a runtime present before it can decrypt the model or input data? If the answer is vague, the design is not ready. If the answer is precise, it can become policy, test cases, and audit evidence.

Only then expand to training or multi-party collaboration. Confidential training has more moving parts and more places for derived data to leak. Teams that first learn to operate confidential inference usually make better choices when they later protect data pipelines, distributed training, and checkpoints.

✅

Good first target. A narrow, well-instrumented inference service is often the best pilot because it has a clear boundary and an obvious data path.

Key takeaways

Confidential AI protects data in use, the moment when prompts, model weights, outputs, and training records are being processed.
Attestation is the control that turns a protected runtime into a verifiable trust decision.
Confidential inference is usually simpler than confidential training because it has a shorter and clearer data path.
The protected boundary must include side paths such as logs, caches, checkpoints, metrics, and temporary files.
Confidential computing reduces infrastructure exposure, but it does not replace identity, governance, application security, or model risk management.
Start with a precise threat model and a narrow workload before expanding to complex training pipelines.

Glossary

Attestation: Attestation is a proof process that lets a system verify what code and configuration are running inside a protected environment.
Confidential computing: Confidential computing is a set of hardware and software techniques that protect data while it is being processed.
Confidential inference: Confidential inference is the use of protected execution to keep model inputs, weights, and outputs less exposed while a trained model answers requests.
Confidential training: Confidential training is the use of protected execution to reduce exposure of training data, intermediate values, gradients, and model checkpoints.
Data in use: Data in use is data that is actively being processed by software or hardware rather than stored or transmitted.
Key management service: A key management service is a system that creates, stores, controls, and releases cryptographic keys according to policy.
Side channel: A side channel is an indirect signal, such as timing or resource use, that can reveal information about protected data.
Trusted execution environment: A trusted execution environment is an isolated area for running code and handling data with stronger protection from the surrounding system.