The AI Supply Chain for Eval Artifacts — Frontier AI Security: Weights and Provenance

Artificial intelligence (AI) evaluations, often called evals, are meant to measure a system, not become a way to compromise it. Yet an eval run commonly imports outside models, datasets, adapters, prompt sets, benchmark packages, and scoring tools. Those artifacts can change what the system does, what result gets reported, or what code runs inside the environment. The practical lesson is simple: eval artifacts are supply-chain inputs. Treating them as casual research data leaves a gap that ordinary software security already learned how to close.

The old lesson from software dependencies

Modern software is assembled from many outside pieces. A team may write its own application, but it also depends on libraries, packages, build tools, container images, and plugins that came from somewhere else. If one of those pieces is malicious, compromised, or silently replaced, the weakness can spread into every system that relies on it.

That is supply-chain risk: the risk that a trusted system is harmed through something it imports or depends on. Software teams did not solve this completely, but they built a useful discipline around it. They check where components came from, verify cryptographic signatures and hashes, pin exact versions, review dependencies, and keep records of what entered a release.

AI evaluations need the same discipline. The component types are different, but the trust problem is familiar.

What counts as an eval artifact

An eval artifact is any outside file, package, or bundle that is pulled into an evaluation run. That includes model checkpoints, datasets, adapters, prompt libraries, benchmark definitions, scoring scripts, container images, and sometimes hidden helper code used to load or transform the data.

These artifacts are not all executable in the same obvious way that a software library is. Some are weights, some are text, some are metadata, and some are scripts. But the security question is not whether they look like code. The question is whether they can affect the behavior, safety, confidentiality, or measured result of the eval. In practice, they can.

A model file may require a loader. A dataset may include a custom processing script. An adapter may steer a model in a narrow but important way. A benchmark package may include answer keys, task templates, or scoring logic. Each one becomes part of the evaluation system once it is trusted and used.

An eval artifact is any outside file pulled into a run that can affect its behavior or result.

Why 'it is just data' is the dangerous phrase

The phrase sounds harmless because people imagine data as passive. A table of examples or a file of weights does not look like a program with functions and system calls. But many AI artifacts are handled by software that interprets them, transforms them, or executes helper logic on their behalf.

Unsafe deserialization is one example. Deserialization means turning stored bytes back into a live object in memory. Some formats can trigger code execution during that process if they are loaded unsafely. Dataset loaders are another example: a dataset may arrive with code that downloads, preprocesses, or reshapes the examples. Poisoned content is a third case: the file may not execute code, but it can bias a benchmark, leak answer material, or train an evaluator to trust the wrong behavior.

The attacker does not need the artifact to be called code. They only need it to cross into the evaluation environment with more trust than it deserves.

⚠️

The naming trap. Calling an artifact data does not make it safe if the eval environment loads it, executes its helpers, or bases decisions on it.

Apply the classic controls to AI artifacts

The fix is not a new security philosophy. It is the ordinary software supply-chain playbook extended to models, datasets, adapters, and benchmarks.

Provenance means knowing where an artifact came from and how it was produced. Signing means using cryptography so tampering can be detected. Hash verification means checking that the bytes you received match the exact bytes you expected. Version pinning means using the reviewed artifact, not whatever the repository serves under a moving name such as latest. Staging and scanning mean holding artifacts in a controlled area before they are allowed to run. Inventory means recording exactly what went into each eval so results and incidents can be traced later.

These controls do not prove that an artifact is benign. They make trust explicit, make changes visible, and make mistakes easier to contain.

Require a known source for every model, dataset, adapter, and benchmark package.
Check signatures where available and verify hashes for every artifact actually used.
Pin immutable versions rather than relying on mutable tags, branch names, or repository defaults.
Quarantine new artifacts before execution, especially if they include loaders or preprocessing code.
Keep an inventory that links each eval result to the exact artifact versions used.

Extend the classic software controls to models, datasets, adapters, and benchmarks.

What these controls prevent

Supply-chain controls are easy to dismiss until you map them to concrete failures. Without pinning, a benchmark can change between two runs while keeping the same friendly name. Without hash verification, a cached artifact can be altered and still look like the expected file. Without provenance, a team may not be able to tell whether a dataset came from a trusted source, a mirror, a fork, or an unreviewed upload.

The same applies to adapters and model checkpoints. A small adapter may be enough to change behavior in a targeted domain. A checkpoint may require a risky loading path. A benchmark bundle may include scoring code that quietly changes the result. Controls do not remove the need for technical review, but they make the object of review stable and identifiable.

A useful rule is this: if changing an artifact could change the eval outcome or the safety of the environment, that artifact belongs under supply-chain control.

Controls make changes visible and the object of review stable and identifiable.

The decisions leaders and practitioners must make

The hardest part is often not the cryptography. It is deciding what the organization will allow, who can approve exceptions, and how much friction is acceptable for different kinds of eval work.

A research team may need fast experimentation. A frontier-model safety evaluation may need stronger isolation, stricter provenance, and a complete run record. A public benchmark used for a paper may need contamination controls and reproducibility. An internal red-team eval may need artifact handling closer to malware-lab practice because it intentionally exercises risky behavior.

Good governance turns those differences into explicit tiers instead of informal judgment. The policy should say which artifact sources are approved, which formats are allowed, when custom loaders are forbidden or sandboxed, how secrets are kept out of eval environments, and what evidence must be saved with each run.

Which repositories, registries, or storage locations are trusted enough for eval intake?
Are mutable tags allowed, or must every eval use immutable versions and recorded hashes?
Who may approve an artifact with custom loading code or unclear provenance?
Which evals require network egress to be blocked by default?
What evidence is needed to reproduce a result or investigate a later incident?

📌

A practical split. Low-risk experiments can be lightweight, but evals that affect deployment, publication, or safety claims need stronger artifact controls.

Build a run record, not just a result

An eval score without its artifact record is weaker than it looks. If a model passes a benchmark, the organization should be able to say which benchmark version, which dataset hash, which prompt set, which adapter, which scoring code, and which container or environment produced the result.

This is where AI evals can borrow from build provenance. A build provenance record describes what inputs and process produced a software artifact. Supply-chain Levels for Software Artifacts (SLSA) is one well-known framework for raising confidence in software build integrity. The exact framework matters less than the habit: record the inputs, make the process reproducible where practical, and prevent unreviewed changes from slipping into the path.

For evals, that record is also an incident-response tool. If a malicious loader, contaminated benchmark, or swapped artifact is discovered later, the team can find which runs used it, which conclusions depended on it, and which results need to be withdrawn or repeated.

A trustworthy score records every input that produced it, not just the result.

The mindset shift

The point is not to slow every evaluation to a crawl. The point is to stop giving AI artifacts a special exemption from rules that already protect software systems. If an artifact can change behavior, execute through a loader, alter a score, or affect a safety claim, it is part of the system.

Once that is accepted, the rest becomes ordinary engineering. Know the source. Verify the bytes. Pin the version. Review risky behavior. Run untrusted inputs in controlled environments. Record what happened. These habits close the gap created by treating models and datasets as informal research material.

A careful kitchen checks the seal and source of every ingredient, not only the ones that look dangerous. AI evals need the same posture. The harmless-looking crate at the door is still part of dinner.

✅

Simple rule. If an artifact enters the eval environment, it should enter through a controlled supply-chain path.

Key takeaways

AI eval artifacts are supply-chain inputs, not casual research files.
Models, datasets, adapters, benchmarks, and loaders can affect behavior, results, or environment safety.
Classic controls such as provenance, signing, hash verification, version pinning, review, and inventory apply directly to AI artifacts.
The phrase 'just data' is risky because many data artifacts are interpreted by code or trusted in ways that affect decisions.
Leaders should define artifact intake rules, exception paths, and evidence requirements based on eval risk.
A trustworthy eval result includes a record of the exact artifacts and environment that produced it.

Glossary

AI artifact: A model, dataset, adapter, benchmark, prompt set, loader, or other file used as an input to an artificial intelligence system or evaluation.
Supply-chain input: An outside component that becomes part of a system's operation, results, or trust boundary.
Provenance: Evidence about where an artifact came from, how it was produced, and whether it is the artifact the user intended to use.
Cryptographic hash: A short fixed value computed from data so that even a small change to the data produces a different value.
Signature: A cryptographic proof that an artifact was approved by a holder of a particular signing key and has not changed since signing.
Version pinning: Locking a system to an exact artifact version or hash instead of accepting a moving label such as latest.
Adapter: A small add-on that changes or specializes a model's behavior without replacing the whole model.
Deserialization: The process of turning stored bytes back into live objects or structures that software can use.
Supply-chain Levels for Software Artifacts: A framework, often abbreviated SLSA, for improving confidence that software artifacts were built and delivered through a trustworthy process.