Hermetic Builds, Provenance and SLSA — Frontier AI Security: Weights and Provenance

If an artifact matters, the process that produced it matters too. A model file, dataset, container image, benchmark package, or evaluation harness may look like a finished object, but it is also the last stop in a chain of downloads, scripts, build workers, credentials, approvals, and packaging steps. Hermetic builds reduce the chance that something unplanned enters that chain. Provenance records what happened. SLSA, short for Supply-chain Levels for Software Artifacts, gives teams a common way to describe how strong those protections are.

The assembly line is part of the attack surface

Security teams often inspect the final artifact: the container image, the model checkpoint, the dataset archive, or the release bundle. That inspection is useful, but incomplete. An attacker does not need to break the finished artifact if they can tamper with the process that creates it.

A build process can be influenced in quiet ways: a script can fetch the latest version of a dependency, a worker can inherit a secret, a package can be swapped in a registry, a dataset preparation job can pull from an unpinned location, or a release step can run on an untrusted machine. Each of these is an opportunity to alter the output while leaving the final file looking ordinary.

The core idea is simple: treat the build system as a production system, not as plumbing. It should have controlled inputs, limited network access, auditable steps, and evidence that can be checked later.

📌

Main idea. Build integrity is confidence that an artifact was produced as intended, not merely that it looks acceptable after the fact.

An attacker can alter the output by tampering with the build process, not just the final file.

Hermetic builds seal the line

Hermetic means sealed. In build engineering, a hermetic build is one whose inputs are declared and gathered before the build runs, and whose build environment cannot reach out to the internet or other undeclared sources while it is running. The build should not be able to improvise.

This matters because many supply-chain failures start with an implicit input. A script downloads a package by name instead of by exact digest. A tool asks a public service for a current version. A test step fetches sample data from a remote bucket. Those actions may be convenient, but they turn the build into a live negotiation with the outside world.

A sealed build removes that negotiation. Inputs are pinned, cached, and verified in advance. The build worker runs with no general network egress. If the build needs a compiler, base image, dataset shard, tokenizer file, or model conversion tool, that input is named explicitly. If it was not declared, the build cannot use it.

Hermeticity is closely related to reproducibility, but they are not identical. Hermeticity removes undeclared external inputs. Reproducibility also requires control over time stamps, random seeds, file ordering, hardware behavior, and nondeterministic tools. A hermetic build is the foundation; deterministic practice makes repeated outputs match.

⚠️

Convenience trap. Any build step that downloads something at runtime is a new input, even if the script treats it as a harmless helper.

A hermetic build pins its inputs in advance and runs with no network egress.

Provenance records what actually happened

Provenance is a trustworthy record of how an artifact was made. It answers practical questions: which source revision was used, which dependency versions were included, which build script ran, which builder produced the output, when it ran, and which parameters or environment settings mattered.

Good provenance is not a comment in a release note. It is structured evidence produced by the build system, linked to the artifact, and protected against tampering. Ideally it is signed by the build platform or otherwise anchored in a trust mechanism, so a verifier can tell whether the record came from the expected system.

Provenance helps in two directions. Before release, it lets policy checks reject artifacts that were built in the wrong place, with the wrong inputs, or without required controls. After an incident, it lets responders trace exposure quickly: which releases used the affected package, which dataset included the bad shard, which model conversion job used the vulnerable tool.

Hermetic builds and provenance reinforce each other. The sealed build reduces the chance of surprise inputs. The provenance record lets you prove which inputs and steps were used. One controls the process; the other makes the process inspectable.

SLSA gives a shared yardstick

SLSA, pronounced salsa, stands for Supply-chain Levels for Software Artifacts. It is an industry framework for describing and improving the integrity of software build pipelines. Its value is not that it invents every control from scratch, but that it gives teams a shared vocabulary and a progression of stronger guarantees.

Without a shared framework, one organization may say “secure build” and mean code review, another may mean signed artifacts, and a third may mean isolated builders with generated provenance. SLSA turns that ambiguity into claims that can be compared and tested. It asks whether the build is scripted, whether provenance is generated, whether the build service is trusted, whether the process resists tampering, and whether consumers can verify the result.

The exact implementation details vary by environment, and SLSA has evolved over time, but the direction is stable: move from informal build practices toward controlled, auditable, policy-enforced production of artifacts. A stated SLSA level is not a magic shield. It is a compact way to say which build-integrity promises are being made and how they can be checked.

Use SLSA to compare build assurances across teams and suppliers.
Treat a claimed level as evidence to verify, not a slogan to accept.
Map SLSA controls to the artifacts that matter most first, then expand coverage.

SLSA gives a progression from a basic scripted build to a tamper-resistant one.

Why this applies to models and datasets

Artificial intelligence artifacts are often discussed as if they appear fully formed: a model checkpoint, a training dataset, an evaluation set, a tokenizer, a fine-tuning adapter, or a benchmark harness. In reality, each is assembled by code. Data is collected, filtered, transformed, deduplicated, labeled, split, serialized, compressed, uploaded, and packaged. Models are trained, fine-tuned, converted, quantized, merged, wrapped, and released.

Those steps are supply-chain steps. A poisoned data shard, swapped tokenizer, malicious dataset loader, altered evaluation prompt set, compromised conversion tool, or unexpected package download can change the artifact that downstream users trust. The risk is not limited to application code.

Hermetic build discipline for artificial intelligence does not always mean the same thing as compiling a small binary. Training large models may involve distributed hardware, nondeterministic numerical behavior, long-running jobs, and massive datasets. Even so, the principles still apply: declare inputs, pin versions and digests, restrict network access, record the environment, capture seeds and configuration, identify the builder, and generate provenance that travels with the output.

For evaluation environments, the case is especially direct. Benchmarks, sandboxes, scoring scripts, dataset loaders, and model wrappers are software supply-chain artifacts. If they are built informally, an evaluation result can be influenced by the assembly process rather than by the model capability being measured.

The decisions leaders and practitioners face

The hard part is rarely agreeing that build integrity matters. The hard part is deciding where to apply scarce engineering effort first. A useful starting point is to rank artifacts by blast radius: public releases, production deployment images, model weights, evaluation harnesses, privileged internal tools, and datasets used to make high-stakes decisions usually deserve early attention.

Practitioners then face concrete design choices. Should builds run with default-deny network egress? Which dependencies must be pinned by cryptographic digest rather than version name? Which registries and storage buckets are allowed as sources? Who is allowed to trigger release builds? Which provenance fields are mandatory? Which policy failures block release and which only raise alerts?

Leaders face governance choices. They need to decide what evidence suppliers must provide, what SLSA target is appropriate for different artifact classes, how exceptions are approved, and who owns the risk when an artifact cannot meet the target. A modest, enforced requirement is usually better than an ambitious policy that everyone bypasses.

The practical goal is not perfection on day one. It is to make the normal path the safe path: trusted build service, sealed inputs where feasible, generated provenance, signed outputs, and automated verification before deployment or publication.

Start with artifacts that are released externally or used in production.
Require provenance for high-value artifacts before trying to cover every internal experiment.
Block undeclared network access in release builds, then expand the pattern to training and evaluation jobs.
Write exception processes down, because undocumented exceptions become the real policy.

✅

Good operating rule. If an artifact is important enough to deploy, publish, or cite, it is important enough to know how it was built.

What a practical control set looks like

A mature build-integrity program is a set of habits that work together. Inputs are pinned by digest or immutable reference. Build workers are ephemeral, meaning they are created for a job and discarded afterward. Secrets are not exposed to untrusted build steps. Network access is denied unless explicitly required. Build definitions are reviewed like code. Outputs are signed. Provenance is generated automatically and stored where consumers can retrieve and verify it.

For artificial intelligence systems, add data and model-specific records. Capture dataset manifests, filtering code versions, labeling tool versions, training configuration, model initialization source, random seeds where meaningful, hardware class, framework versions, conversion tools, quantization settings, and evaluation harness revisions. Not every field will be equally important for every artifact, but missing evidence should be a conscious decision.

Verification is the closing step. A provenance record that no one checks becomes an archive, not a control. Deployment systems, model registries, evaluation runners, and release pipelines should be able to reject artifacts that lack required provenance, were built by an untrusted builder, used unapproved sources, or fail the organization’s stated SLSA policy.

The result is not blind trust in the build system. It is bounded trust: the system is constrained, its actions are recorded, and consumers have a way to decide whether the evidence is good enough for the use at hand.

⚠️

Do not stop at records. Provenance only reduces risk when consumers verify it and enforce policy based on it.

Key takeaways

Hermetic builds reduce surprise by preventing build steps from fetching undeclared inputs while they run.
Provenance is structured evidence that links an artifact to its inputs, builder, steps, and important settings.
SLSA, the Supply-chain Levels for Software Artifacts framework, gives organizations a common way to state and compare build-integrity guarantees.
Models, datasets, benchmarks, and evaluation harnesses are assembled artifacts, so software supply-chain discipline applies to them too.
The practical objective is controlled production plus verifiable evidence, not vague claims that a build is secure.
Leaders should set artifact-specific requirements, starting with high-blast-radius releases and production systems.

A mature build-integrity program combines habits that reinforce one another.

Glossary

Hermetic build: A build that runs with declared inputs in a sealed environment, usually without general network access, so undeclared outside material cannot enter during the build.
Provenance: A trustworthy record of how an artifact was produced, including its inputs, builder, steps, and relevant settings.
SLSA: SLSA stands for Supply-chain Levels for Software Artifacts, an industry framework for describing and improving software build integrity.
Build integrity: Confidence that an artifact was produced by the intended process, from the intended inputs, without unauthorized tampering.
Artifact: A produced object that others consume, such as a binary, container image, model checkpoint, dataset archive, benchmark package, or release bundle.
Pinned input: An input identified by an immutable version, digest, or equivalent reference so the build receives exactly the expected material.
Network egress: Outbound network access from a system, such as a build worker connecting to the internet or to an internal service.
Digest: A cryptographic fingerprint of data that changes if the underlying content changes, commonly used to verify exact identity.