TL;DR
AI supply chain security means protecting every outside component an AI system is built from, such as models, datasets, packages and agent tools. Start by listing each component, its exact version and its owner in an AI bill of materials (AIBOM). Then admit new artifacts only after they pass a scan, a signature or digest check and your own evaluation. Pin exact versions and serve approved copies from a private registry instead of pulling straight from public hubs. Treat MCP servers, plugins and hosted model versions as dependencies too, because they can change under you. Finally, rehearse the response so your team can find every affected deployment and roll back to a verified version.
Key Takeaways AI supply chain security covers models, adapters, datasets, packages, containers, MCP servers, agent skills and hosted model APIs. Model files can run code when loaded, so a public download deserves the same caution as an unknown executable. An AIBOM that records digests, sources and owners turns “are we affected?” into a query instead of a week of searching. A signature proves who published an artifact and that it is unchanged, but it never proves the model is safe. Every artifact should pass one admission pipeline that quarantines, scans, verifies, evaluates and records an owner. Pinned versions, a private registry and a rehearsed rollback limit the damage when an upstream component turns bad. Watch on YouTube
How MCP Makes AI Smarter, Faster, and Business-Ready | Model Context Protocol
Kanerika’s experts explain how the Model Context Protocol connects AI agents to business tools and data, which is exactly why every MCP server you install becomes part of your AI supply chain.
The Email Tool That Added a Stranger to Every Message
Picture an AI agent that sends invoices and password resets through an npm package called postmark-mcp. The name looks official, but Postmark never published it. According to MITRE ATLAS case study AML.CS0053 , someone else registered the name on npm, shipped working versions and waited.
In September 2025, once the package passed 1,000 weekly downloads, version 1.0.16 put the attacker’s address on the BCC line of every email the tool sent. The tool kept working, so nothing looked wrong. The model and the prompt were fine, and the weak link was a component the team pulled in once and never checked again.
That is AI supply chain security, which secures the parts an AI system is built from rather than AI for logistics. The defense is the same for models, datasets, packages and agent tools. Know exactly what you run, admit only what you have checked and be able to pull any part back out.
What Is AI Supply Chain Security and How Does It Differ From Software Supply Chain Security?
AI supply chain security is the practice of verifying, controlling and monitoring every third-party component that shapes how an AI system behaves. It covers pre-trained weights, LoRA adapters, tokenizers and the data used to train or fine-tune them. It also covers RAG corpora, Python packages, containers, MCP servers, plugins, agent skills and the hosted model behind an API.
Classic software supply chain work still applies, but AI artifacts behave like code even when they look like data. For example, a model file can run code on load, and a dataset can change what a model learns. So a clean hash proves a file is unchanged, yet it says nothing about what the file will do.
Dimension Software supply chain AI supply chain Main artifacts Source code, packages, containers All of those, plus weights, adapters, datasets, embeddings, MCP servers and hosted model versions How code runs When you install or execute it Also when a model file is loaded or an agent calls a tool What a hash proves The file matches the release The file matches the release, but not that its learned behavior is safe Update path Versioned releases you choose to take Also provider-side model updates and tool updates you may not notice Usual owner Platform or AppSec team Split across data owners, ML engineers, platform, security and app owners
The Standards That Name This Risk
Standards bodies treat this as its own risk class. The OWASP Top 10 for LLM Applications lists supply chain as LLM03:2025 and names LoRA adapters and Hugging Face as new exposure. It also lists data and model poisoning separately, as LLM04. OWASP’s Top 10 for Agentic Applications adds ASI04 for agentic supply chain flaws in MCP and A2A ecosystems.
MITRE ATLAS files these attacks under AML.T0010, AI Supply Chain Compromise, which has included an AI Agent Tool sub-technique since March 2026. For build practices, NIST published SP 800-218A in July 2024 as a secure development profile for generative AI. If you already run an AI security framework , this is the part that decides what may enter it.
On the governance side, the NIST AI RMF covers this under GOVERN 6. It asks for policies and procedures that address AI risks from third-party software, data and other supply chain issues.
Map the Dependency Chain for One Reference Stack
Because abstract lists hide real gaps, take one ordinary system and trace it end to end. The reference stack here is an internal retrieval-augmented generation assistant with one tool-calling agent. It answers staff questions from company documents and also files tickets through an MCP server.
Layer How it gets attacked Main control Owner Base model Malicious weights file, tampered repo revision Pin revision and digest, scan, prefer Safetensors ML platform Fine-tune and LoRA adapters Poisoned or swapped adapter Sign adapters, record lineage to the base model ML engineering Training and fine-tune data Poisoned samples, licence surprises Hash-pinned snapshots, source allowlist Data owner RAG corpus and embeddings Planted documents, hidden instructions Ingest review, canary documents Application owner Python packages and containers Dependency confusion, typosquats Hash-locked installs, one internal index Platform team MCP servers, plugins, skills Namesquats, rug-pull updates Allowlist, pinned versions, permission diff review Security and app owner Hosted model API Provider-side version change Pinned version, regression evals App owner and vendor management
Artifact Trust Versus Supplier Trust
Two kinds of trust also sit inside that table. Supplier trust asks whether you trust an organization, such as a model lab or a package maintainer. Artifact trust, by contrast, asks whether this exact file, at this digest, from this channel, is the one you approved.
In practice, real incidents live in the gap between the two. Both torchtriton and postmark-mcp borrowed a trusted name while shipping an untrusted artifact. For that reason, a name check passed, while a digest check against an approved record would have failed.
In addition, every row needs a named owner, or the control decays the first time a deadline slips. Put those owners in the register your AI governance program already keeps, so supply chain decisions get the same review as other model risks. The map also exposes shadow AI , because any model or tool nobody can place on it is unmanaged.
Where Attacks Enter the AI Supply Chain
Attacks enter at five points in the reference stack, and each one needs a different check. Model files can carry code, data can carry triggers, and packages can arrive under a borrowed name. Agent tools and hosted models can then change after you approved them.
Malicious Model Files and Unsafe Serialization
Many PyTorch and older Keras model files are Python pickles. Pickle rebuilds objects by calling functions, so a crafted file can run commands the moment someone loads it. In February 2024, JFrog’s researchers found around 100 Hugging Face models with real harmful payloads.
One of them, baller423/goober2, used pickle’s __reduce__ method to open a reverse shell to a remote host. So loading that model was enough to hand an outsider a terminal on the machine.
Hugging Face does run pickle scanning and shows the imports it finds. Still, it flags dangerous files rather than blocking them, and its docs say the scanner is not 100% foolproof. PyTorch has also tightened its defaults, since torch.load uses weights_only=True from version 2.6. That narrows the code-execution surface without closing it.
Safetensors stores weights without executable code, and it became a PyTorch Foundation project on 8 April 2026. So prefer it, but don’t treat the format as a guarantee. HiddenLayer showed that the Hugging Face Safetensors conversion service could be abused to send malicious pull requests and hijack models submitted for conversion.
The gap between exposure and checking is wide. HiddenLayer’s 2026 AI Threat Landscape Report found that 93% of organizations use open-weight models from public repositories. Yet fewer than half consistently scan the models they bring in.
Poisoned Training, Fine-Tuning and RAG Data
Data attacks don’t need code execution at all. A handful of planted samples can teach a model to react to a trigger phrase, and one planted document can steer a RAG answer. OWASP files this risk as LLM04:2025, and ATLAS lists it as Poison Training Data.
That is why controls start with provenance. Pin every training and fine-tune set as a hash-addressed snapshot, ingest only from allowlisted sources and record who approved each one. Then add canary documents and spot reviews to the ingest step.
A checksum still can’t find poison that was already present when you approved the snapshot. So pair it with evaluations that look for odd behavior on targeted prompts. Good data lineage from source to model output is what tells you which models a bad snapshot touched. Retrieval-time controls, such as permission filters and hidden-instruction scanning, are covered in the RAG security guide.
Compromised Packages and Dependency Confusion
The torchtriton incident shows how ordinary package tooling becomes an AI risk. Between 25 and 30 December 2022, Linux users who installed PyTorch-nightly with pip received a malicious torchtriton package from PyPI. It shared its name with a dependency on PyTorch’s own nightly index, and pip gave PyPI precedence.
According to the PyTorch disclosure , the binary uploaded system data, including files from $HOME/.ssh, through encrypted DNS queries. However, stable PyTorch builds were not affected. That is a useful reminder that nightly and preview channels deserve their own, lower trust level.
ML environments are prone to this because notebooks and training images often pull from several indexes at once. Still, the fix is plain discipline. Use one internal index that proxies approved upstreams, lock dependencies with hashes, and refuse any install that resolves outside that index.
Malicious MCP Servers, Plugins and Agent Skills
Agent tools are executable dependencies that hold live credentials, which makes them the newest and riskiest layer. The postmark-mcp case combined two old tricks. First came a namesquat, and then a rug-pull update after the package had earned trust. As of November 2026, ATLAS tracks this pattern as AML.T0010.005, AI Agent Tool.
Tool descriptions also add a third trick. An MCP server describes its tools in text the model reads, so a poisoned description can steer the agent without changing any code. Review that text as part of the artifact, and diff it on every update.
Admit an MCP server the way you would admit a library with production access. Check the publisher, pin the exact version and review the permissions it requests. AI agent skills need the same treatment, since a skill bundle can carry scripts as well as instructions.
Credentials matter as much as code here, too. Short-lived, scoped AI agent authentication limits what a compromised tool can reach. For runtime defenses such as tool-call policies and sandboxes, see the AI agent security tools roundup. Then read the wider risk picture in the agentic AI security guide.
Checklist
Enterprise Agentic AI Checklist
A practical checklist for rolling out AI agents in the enterprise, so the tools, permissions and integrations your agents depend on get reviewed before they reach production.
Get the Checklist → Silent Vendor Model Swaps and Version Drift
The hosted model behind an API is a dependency you cannot scan, so version control has to do the work. Microsoft Foundry , for example, gives each Azure OpenAI deployment an update policy. A gpt-4o deployment on version 2024-08-06 set to auto-update switches to 2024-11-20 on its own. Azure gives at least two weeks’ notice before a new version becomes the default.
First, pin a specific version wherever the provider allows it, and avoid aliases that always point at the newest model. Then run your regression suite from an LLM evaluation framework on every provider change, planned or not. Then review each provider’s update policy as part of your AI vendor risk assessment before you sign.
Output drift alone doesn’t prove a swap, because prompts and retrieved context shift too. Still, an unexplained jump in eval scores is a reason to ask the vendor. Routing calls through an LLM gateway that logs the model version behind each response makes that question easy to answer.
Kanerika Service
MLOps Consulting Services
Kanerika’s MLOps services cover model monitoring and alerting, real-time drift detection and governance framework development, with model registry configuration where the scope needs it.
Explore MLOps Consulting Build an AI Bill of Materials (AIBOM) You Can Query
An AIBOM is the inventory that makes every other control possible, because it ties each artifact to where it runs. When an advisory lands, it answers one question fast. Which of our deployments use this file?
An SBOM, for example, lists software packages and their versions. An AIBOM adds models, datasets, adapters and the services an AI system calls, plus the lineage that connects them. So you can trace a single poisoned dataset to every fine-tune and index built from it.
What it records SBOM AIBOM Components Packages, libraries, containers Also models, adapters, tokenizers, datasets and embeddings Lineage Package dependency tree Base model to fine-tune to adapter, dataset snapshot to index Data Not covered Source, collection process, intended use, sensitive data flags External services Rarely recorded Hosted model provider, endpoint, pinned version, MCP servers Evidence Licences, known vulnerabilities Also signatures, scan results and evaluation references
Keep the minimum record short enough that teams actually fill it in. Each AIBOM entry should carry these fields.
Identifier and version or revision Cryptographic digest of the exact artifact Source location and supplier Licence and permitted use Dependencies, such as the base model behind an adapter Accountable owner and approval date Intended use and every deployment that runs it Open Formats That Can Hold an AIBOM
As of November 2026, two open formats can carry these records. CycloneDX ML-BOM covers models, datasets and their dependencies, including dataset provenance and training method. CycloneDX added it in version 1.5, and version 1.7, first released in October 2025, is still the current line, with patch 1.7.2 out in September 2026. Version 2.0 has been announced but not released.
SPDX is the other option, and it works similarly. SPDX 3.0 has an AI profile and a Dataset profile, with fields such as known bias and intended use. A trimmed record for the reference stack’s adapter could look like this.
component: support-assistant-lora
type: machine-learning-model (adapter)
revision: 3f9c2e1
sha256: 9b1d...e44a
base_model: approved base model, revision a71d0c4
trained_on: tickets-snapshot-2026-09 (sha256 51ac...0d2f)
supplier: internal ML engineering
licence: internal
owner: support-platform team
deployed_to: helpdesk-assistant (prod), helpdesk-assistant (staging)
evidence: scan passed, signature verified, eval suite v12 passedWhat an AIBOM Cannot Tell You
An AIBOM can’t tell you everything, though. It won’t reveal a behavioral backdoor, a tool an agent picks at runtime or the weights behind a hosted API. Reconcile it with what actually runs in production, using LLMOps observability data on which models and tools get called.
Verify Provenance and Sign What You Promote
Provenance answers where an artifact came from and who produced it, and signing makes that answer checkable later. The OpenSSF AI/ML Working Group released model-signing v1.0 in April 2025, with authors from Google, NVIDIA and HiddenLayer. It signs models of any format and size, and it works with Sigstore, certificates or plain key pairs.
As of November 2026, the latest release is v1.1.1, which is also published on PyPI as model-signing. For the pipeline that builds or trains an artifact, SLSA describes how to record build provenance that others can verify.
Still, be precise about what a signature proves. It shows who signed the artifact and that nobody changed it afterwards. It does not show that the model is benign, unbiased or fit for your use case, because a signed backdoored model is still backdoored.
Therefore, pin by digest or commit revision, never by a mutable tag such as latest. Verify the signature and digest when you first admit an artifact, again at promotion and once more at deploy. Three checks may sound redundant until a registry mirror or a build cache serves a different file under the same name.
Checklist
Generative AI Checklist
A practical checklist for secure generative AI adoption and governance, covering data access, vendor review and monitoring for the models and services you bring in.
Get the Checklist → Run Every Artifact Through an Admission Pipeline
Controls only work when they sit on the one path every artifact must take into production. An admission pipeline is that path, and it applies the same checks to a model, a dataset, a package or an MCP server.
Take intake only from an allowlisted source. Quarantine the artifact in an environment with no production credentials and no open egress. Scan model files for unsafe code, and run your usual package and container scanners. Next, verify the signature and digest against the publisher’s record. Evaluate behavior on your own safety and regression tests. Finally, approve with a named owner and write the decision into the AIBOM. Publish the approved copy to your private registry. Verify the digest again at deploy time. Open-source scanners cover step three for most teams. ModelScan from Protect AI checks H5, pickle and SavedModel files for unsafe code, and picklescan inspects pickles and can scan Hugging Face repos directly. For a side-by-side comparison of scanners, see the LLM security tools guide.
Pass, Block and Exception Rules
Most decisions are routine once the rules are written down. The table below shows the conditions that should stop an artifact and what clears each one.
Condition Decision What clears it Missing provenance Block Publisher record, or an internal rebuild from verified source Pickle with unsafe imports Block Convert to Safetensors inside quarantine, then rescan Digest mismatch Block Fresh download from the source and an investigation Unpinned version or a “latest” alias Block Pin a revision or a provider model version MCP server requests new permissions Exception review Security sign-off on the permission diff Unknown licence Exception review Legal sign-off with an expiry date All checks pass Pass Owner approval and an AIBOM entry
Private Registries, Least Privilege and Runtime Checks
A private registry is where approved artifacts live, so it stops dependency confusion and gives you one place to revoke from. Even so, mirroring a bad artifact doesn’t make it safe, so the registry is only as good as the gate in front of it. For Python packages, pip’s –require-hashes mode refuses to install any file whose hash is not listed in the requirements file.
Two controls cover what the pipeline cannot see. First, give agents and tools least privilege, as an agentic AI permission control framework lays out, so a bad component reaches as little as possible. Second, monitor runtime behavior and add AI guardrails , because an approved artifact can still misbehave in production. A periodic AI security assessment then tests whether the whole pipeline holds.
Incident Response When a Component Is Compromised
Supply chain incidents start outside your walls, usually with an advisory or a researcher’s post. How fast you respond depends on whether the AIBOM can answer the scoping question in minutes. Keep the runbook short and specific.
Log the trigger with the affected name and the bad versions or digests. Query the AIBOM for every deployment, fine-tune and index that used them. Contain the damage by quarantining the artifact, disabling the integration, restricting egress and revoking any credentials it could reach. Preserve evidence, including logs, the artifact and environment state, before you replace anything. Roll back to the last verified version from your private registry. Rebuild or re-evaluate the fine-tunes and indexes the bad component touched. Close the control gap that let it in, and record what changed. The torchtriton disclosure, for instance, is a good worked example. PyTorch told affected users to uninstall torch, torchvision, torchaudio and torchtriton, purge the pip cache and move to nightly builds newer than 30 December. Because the payload read SSH files, rotating any keys on affected machines belongs in step three as well.
Above all, practice this before you need it. A drill on a staging copy, for example, shows whether your AIBOM query finds every affected deployment in minutes or in days. Rank which components deserve a drill first with AI threat modeling , then test the response through AI red teaming exercises.
A 30/60/90-Day AI Supply Chain Security Rollout and the Metrics That Show Progress
A full program takes time, but the first month removes the worst exposure. Then each later phase builds on the inventory.
Days 1 to 30. Inventory production AI, pin model and package versions, and ban unsafe model loading.Days 31 to 60. Stand up the admission pipeline, the private registry and an MCP server allowlist.Days 61 to 90. Add signing at promotion, run an incident drill and write change-notice clauses into vendor contracts.Meanwhile, measure control coverage rather than scanner alert counts. Track the share of production deployments with a current AIBOM and the share of artifacts with verified provenance. Then add open exceptions by age, the number of mutable dependencies left and the time it takes to find every affected deployment.
Also record each gap with an owner and a due date. Then an AI risk assessment template keeps that progress visible to the people who fund it.
AI Assessment
See Where Your AI Program Stands
Score your organization’s AI maturity across data, governance and security to see which models, pipelines and integrations need supply chain controls first.
Start Your AI Assessment → How Kanerika Helps With AI Supply Chain Security
Kanerika’s AI governance services cover several controls this guide depends on. According to the service page, that work includes the items below.
Access controls and versioning for every dataset in use Data lineage from source to model output A model registry with a full audit trail across production Post-change audits after AI model updates Independent validation for inherited legacy AI systems LLMOps governance with model version control and secure access management Kanerika’s IMPACT framework drives every AI governance engagement, tying compliance controls to business outcomes you can measure. Its responsible AI work also sets risk tiering with clearly defined accountability and ownership, the same owner-per-component rule an AIBOM enforces.
The same discipline shows up in Kanerika’s infrastructure work for a leading global logistics provider. That client had poor visibility across regional servers and unclear ownership of hosted applications, and a malware attack forced several servers offline. Kanerika centralized the estate’s data in Power BI, set clear ownership, removed redundant applications and decommissioned outdated servers.
As a result, security vulnerabilities fell 90%, accountability rose 85% and the estate reached 100% ITSM readiness. That project secured an IT estate, not an AI system. Still, the method is the one this guide asks for. Know what you run, give every component an owner and remove what nobody can vouch for.
Case Study
90% Fewer Security Vulnerabilities with IT Optimization
Kanerika gave a global logistics provider’s IT estate clear inventory and ownership, removed redundant applications and retired outdated servers, cutting security vulnerabilities by 90%.
Read the Case Study → Kanerika also holds ISO 27001, ISO 27701 and SOC 2 Type II certifications. If your registry and promotion pipeline need building first, MLOps consulting is the place to start. To scope an AIBOM and admission pipeline for your own stack, book a conversation with Kanerika’s team .
Securing What Your AI Depends On
You secure models, datasets, packages and agent tools the same way. Keep an AIBOM that records the digest, source and owner of everything you run. Admit only what passed one pipeline of quarantine, scanning, signature checks and evaluation, and serve it from a private registry.
Then plan for the day a trusted name turns bad, the way torchtriton and postmark-mcp did. Pinned versions and a rehearsed rollback turn that day into a contained fix rather than a breach.
Frequently Asked Questions
What is AI supply chain security?
AI supply chain security is the practice of verifying, controlling and monitoring every third-party component an AI system depends on. That includes pre-trained models, LoRA adapters, training and RAG data, Python packages, containers, MCP servers, plugins, agent skills and hosted model APIs. The goal is to know what you run and admit only checked artifacts.
Is AI supply chain security the same as using AI in supply chain management?
No. AI supply chain security protects the components an AI system is built from, such as models, datasets, packages and agent tools. Using AI in supply chain management means applying AI to logistics, procurement or inventory planning. The two topics share words, but they involve different teams, risks and controls.
What is the difference between an AIBOM and an SBOM?
An SBOM lists software packages, libraries and containers with their versions. An AIBOM adds models, adapters, tokenizers, datasets, embeddings and hosted model services, plus the lineage connecting them. It also records digests, owners, intended use and evaluation evidence, so you can trace a poisoned dataset or model to every deployment.
Can downloading a model from Hugging Face run malicious code?
Yes. Many PyTorch and older Keras models are Python pickles, and loading a crafted pickle can run commands. JFrog found around 100 Hugging Face models with real harmful payloads in 2024. Hugging Face scans pickles and flags dangerous imports, but it does not block every file, so scan and quarantine before loading.
Is Safetensors safer than pickle, and what does it not prevent?
Safetensors is safer because it stores weights without executable code, so loading the file cannot run commands. It does not stop a backdoored model, poisoned training data, a malicious package or custom loading code. Research has also shown that conversion services can be abused, so still verify the source and digest.
How do you secure MCP servers and agent skills?
Treat them as executable dependencies with live credentials. Install only from an allowlist, check the publisher, pin exact versions and review the permissions and tool descriptions they declare. Diff those permissions and descriptions on every update, give each tool short-lived scoped credentials, and record every server and skill in your AIBOM.
How can you tell if an API provider changed the model behind an endpoint?
Pin a specific model version where the provider allows it, and log the version returned with each response. Run a fixed regression evaluation suite on a schedule and after every announced change. An unexplained shift in scores is a signal to ask the vendor, though drift alone does not prove a swap.
What should you do first after finding a compromised AI dependency?
Query your AIBOM to find every deployment, fine-tune and index that used the bad version or digest. Then contain it by quarantining the artifact, disabling the integration, restricting egress and revoking credentials it could reach. Preserve evidence before rolling back to the last verified version from your private registry.
Does signing a model prove that it is safe?
No. A signature proves who published the model and that the file has not changed since it was signed. It says nothing about whether the model is benign, unbiased or suitable for your use case. Pair signature checks with scanning and your own evaluations before you approve a model for production.
