The AI Supply Chain — Model & Data Attacks

You do not buy or build artificial intelligence in one piece. An artificial intelligence (AI) system is assembled from many upstream choices: what data was collected, who labeled it, which code shaped it, which model learned from it, which service runs it, and how people use its answers. The AI supply chain is the path through those choices. If you can see the path, you can ask better questions about quality, security, cost, legality, and accountability.

What a supply chain means in AI

A supply chain is the set of inputs, transformations, and handoffs that make a product possible. For a loaf of bread, it includes grain, milling, baking, packaging, transport, and the shop shelf. For AI, it includes data, software, model training, testing, deployment, monitoring, and the humans who decide how much to trust the output.

This matters because AI systems rarely fail for one simple reason. A chatbot may give a dangerous answer because its training data included bad examples, because a safety rule was removed during tuning, because a prompt was written poorly, because a dependency changed, or because a user applied the tool outside its intended setting. The visible answer is the last link in a long chain.

The phrase can sound abstract, but the core idea is practical: every AI output has a history. Managing the AI supply chain means keeping enough knowledge of that history to improve the system, fix it when it breaks, and explain why it should be trusted in a given use.

📌

First principle. An AI system is not just a model; it is a model plus the materials, tools, services, rules, and people around it.

From raw material to useful answer

The simplest AI supply chain starts with data. Data may be text, images, audio, sensor readings, business records, or examples created by people. The data is cleaned, filtered, labeled, or transformed so a machine can learn patterns from it.

Training is the process of adjusting a model so it performs a task. A model is a mathematical structure that turns inputs into outputs, such as a document summary, a fraud score, or a recommended next action. After training, the model is evaluated to see whether it behaves well enough for its intended use.

Deployment puts the model into a real setting. Inference is the moment the deployed model receives a new input and produces an output. At that point, the supply chain has become an operating system: data keeps arriving, users respond, logs are created, and the system may need updates.

Data becomes an answer through training, evaluation, deployment, and inference.

The inputs you inherit

No team starts from nothing. Even when an organization says it built its own AI system, it usually inherits many parts. It may use open source code, pretrained models, cloud infrastructure, third-party data, external labeling firms, specialized hardware, and application programming interfaces (APIs) provided by other companies.

Each inherited part carries assumptions. A dataset may reflect one country more than another. A pretrained model may have learned copyrighted, private, biased, or outdated material. A software library may contain a security flaw. A cloud service may change its price, performance, or terms. A contractor may follow labeling instructions differently from an internal team.

This is why provenance is central. Provenance means knowing where something came from and what happened to it along the way. In AI, provenance is not only a legal concern. It is how engineers debug behavior, how security teams trace exposure, and how leaders decide whether a system is fit for a sensitive job.

Most AI systems inherit open source, third-party data, and managed services.

How AI supply chain risk differs from ordinary software risk

Traditional software usually follows explicit instructions written by people. AI systems also include learned behavior. That means a defect can be statistical rather than obvious. The system may work well on average but fail for a subgroup, a rare case, a new location, or an input pattern that was missing from training.

AI also has a more fluid boundary between input and instruction. In many systems, especially those based on large language models, a user prompt can strongly shape behavior. If the system can call tools, search documents, write code, or trigger business processes, then a malicious or careless input can become part of the operating path.

Another difference is opacity. A conventional software bill of materials (SBOM) can list software components and versions. AI needs similar inventory discipline, but it must also describe data sources, model lineage, evaluations, safety controls, and known limits. A list of files is not enough.

Data risk: sensitive, biased, low-quality, or poorly licensed data can shape behavior long after collection.
Model risk: a model may be unsuitable for a domain even if it performs well on public benchmarks.
Dependency risk: a library, model service, or API can change, fail, or introduce a vulnerability.
Use risk: people may apply a system outside the task, population, or level of supervision it was designed for.
Feedback risk: outputs can be reused as inputs, causing errors or narrow patterns to compound over time.

⚠️

Not just bugs. In AI, a system can be working as built and still be unsafe for a particular use.

Assurance is a loop, not a gate

Assurance means gathering evidence that a system is suitable for its intended use. In AI, assurance cannot be a one-time approval at launch. The world changes, users change, data changes, attackers adapt, and suppliers update their components. A model that was acceptable last quarter may be weak today.

Good assurance starts with inventory. Teams need to know which models are in use, which data sources feed them, which dependencies they rely on, who owns them, and what decisions they influence. Without inventory, there is no reliable way to patch, retire, or explain systems.

Evaluation comes next. Evaluation should test the tasks that matter in context, not only a generic score. For a customer support assistant, that may include accuracy, refusal behavior, privacy leakage, tone, escalation to humans, and performance on difficult customer cases. Monitoring then checks whether real-world behavior stays within expected bounds.

Assurance is a loop of inventory, evaluation, monitoring, and updates.

The decisions leaders and practitioners actually face

The AI supply chain becomes real when someone must decide what to build, buy, approve, block, or monitor. A leader may ask whether to use an external model service or host a model internally. An engineer may ask whether a dataset is clean enough. A security team may ask what happens if a model provider changes its endpoint. A product owner may ask whether a human must review outputs before action.

There is no universal best answer. Building more in-house can provide more control but requires scarce expertise, stronger operations, and continuing maintenance. Buying or using a managed service can move faster but increases dependence on supplier terms, uptime, data handling, and change notices. The right decision depends on the harm that could result from failure, the sensitivity of the data, the need for explanation, and the organization’s ability to operate the system responsibly.

A useful habit is to connect every supply chain question to a decision. Do not ask only, "Where did this model come from?" Ask, "Would the answer change whether we can use it for hiring, medical triage, loan review, or code suggestions?" Evidence is valuable when it changes action.

What data will the system see, and is any of it private, regulated, or contractually restricted?
Who is responsible when the model is wrong: the supplier, the internal owner, the human reviewer, or all of them?
What evidence is required before release, and who has authority to stop deployment?
How quickly can the organization replace, roll back, or patch a model or dependency?
Which uses are explicitly out of bounds, even if the system can technically perform them?

A practical starting point

The first step is not a grand platform. It is a clear inventory. List the AI systems in use, including experiments that have quietly become business processes. For each one, record the owner, purpose, model source, data sources, suppliers, users, evaluation results, monitoring signals, and known limits.

Then classify systems by consequence. A tool that rewrites marketing copy does not need the same controls as a tool that influences credit, employment, medical care, legal work, security alerts, or access to public services. Supply chain discipline should be strongest where mistakes are hardest to detect, hardest to reverse, or most harmful.

Finally, make change visible. AI supply chains are living systems. Data distributions shift, model versions change, prompts evolve, policies are rewritten, and suppliers update services. Treat these changes like material events, not background noise. If a change could alter behavior, cost, security, or compliance, it deserves review.

Find the AI systems and name accountable owners.
Record the upstream data, models, code, suppliers, and human handoffs.
Rank systems by potential harm and business dependence.
Define the tests and monitoring each risk level requires.
Review material changes before they reach users.

✅

Good enough to start. An imperfect inventory that is maintained is more useful than a perfect framework that nobody updates.

Key takeaways

The AI supply chain is the history of the data, code, models, services, and people that produce an AI output.
AI risk is not only a model problem; inherited data, dependencies, prompts, suppliers, and user behavior all matter.
Provenance and inventory are the foundation for debugging, security, compliance, and trust.
Assurance must continue after launch because models, data, users, suppliers, and threats change.
Build-versus-buy decisions should be judged by consequence, control, dependency, and the ability to operate responsibly.

Match control strength to how much harm a failure could cause.

Glossary

Artificial intelligence (AI): Computer systems that perform tasks usually associated with human judgment, such as recognizing patterns, generating language, or making predictions.
Model: A mathematical system that turns inputs into outputs after being trained or configured for a task.
Training: The process of adjusting a model so it learns patterns from examples or feedback.
Inference: The moment a trained model receives a new input and produces an output.
Dataset: A collection of examples, records, or observations used to train, evaluate, or operate an AI system.
Provenance: Information about where something came from and what happened to it over time.
Dependency: An external component, service, library, dataset, or model that a system relies on to work.
Software bill of materials (SBOM): An inventory of software components and versions used in a system.
Evaluation: Testing and measurement used to judge whether an AI system is suitable for a specific purpose.