TL;DR
AI threat modeling is a structured way to find how an AI system can be attacked before it ships, and it runs in six steps. You scope the system, draw its data flows, mark trust boundaries, list threats, rank them and record them with owners. STRIDE is the method you walk across that diagram, one element at a time. MITRE ATLAS and the two OWASP Top 10 lists are catalogues that tag and cross-check the threats you find. MAESTRO adds a layered view when the system includes agents that call tools. The result is a threat register that feeds the risk assessment, the red-team test plan and the next review.
Key Takeaways AI threat modeling covers attacks on AI systems, from poisoned documents to agents that take actions nobody approved. The data-flow diagram does most of the work, so label every flow as instructions, evidence or actions. STRIDE is a procedure, while MITRE ATLAS and the OWASP lists are catalogues, so use them together rather than picking one. Write each threat as one sentence naming the attacker, entry point, action, asset and consequence. Every high-priority control needs a test that proves it works, because a system prompt is not an access control. Re-run the model whenever the model, data sources, tools or permissions change, not on a fixed calendar. Watch on YouTube
AI Security for Enterprises: Kanerika’s CISO on Prompt Injection, ISO 27001 & SOC 2
Kanerika’s CISO explains how prompt injection actually reaches enterprise AI systems and which controls and certifications keep them in check.
A Support Assistant, a Shared Drive and One Poisoned PDF
Picture a support assistant that answers staff questions from the company document library. It passes every demo, so it goes live. Then, three weeks after launch, someone uploads a PDF with a hidden line telling the model to quote any HR file it can reach.
The next person who asks about leave policy gets a colleague’s salary review in the answer. Yet nothing in that chain was a classic bug. The model simply followed the text it was given. And the retrieval layer held read access that nobody had ever drawn on a diagram.
AI threat modeling is the exercise that catches this on a whiteboard, before launch. It deals with threats to AI systems themselves. Using AI to speed up threat modeling of ordinary software is a separate question, and it gets a short answer near the end.
What Is AI Threat Modeling?
AI threat modeling is the practice of mapping an AI system’s parts, data flows and trust boundaries. It then lists how each part could be attacked and which control stops it. The output is a ranked threat register with an owner for every serious entry.
The discipline is old, so only the AI parts of the diagram are new. A model, a vector store, a system prompt and tool permissions become elements to analyze. In other words, they get the same scrutiny as a database or an API.
The Four Questions, Applied to AI
The Threat Modeling Manifesto frames every threat model around four questions.
Scope: what are we working on? Threats: what can go wrong? Response: what are we going to do about it? Review: did we do a good enough job? For an AI system, the first question is answered by a data-flow diagram that shows the model, the retrieval path, the tools and the prompts. STRIDE, together with the AI threat catalogues, then answers the second. After that, the third becomes a list of controls, each with a test, and the fourth is a review gate before release.
What Changes When the System Contains a Model
Microsoft’s guidance on threat modeling AI systems names the shifts that matter. Models behave non-deterministically, so the same input does not always produce the same output. They also tend to follow instructions, which makes them open to prompt injection.
Instructions now arrive as data, too. For example, a sentence inside a retrieved web page or email can steer the model as easily as the user can. So any channel that feeds text into the context window is also a control channel.
Tools also widen the blast radius. Some models can call APIs or write records, as agentic AI systems do. Then a bad output stops being an embarrassing answer and becomes an action. Memory and multi-agent setups extend that reach further.
Finally, training data is in scope. Microsoft’s AI and machine learning threat modeling guidance puts training data stores and their hosting systems inside the threat model. It also calls data poisoning the greatest security threat in machine learning, in Microsoft’s words, partly because there are no standard detections for it.
Threat Modeling vs Risk Assessment vs Red Teaming
These three activities get mixed up because they share vocabulary. Each one answers a different question, though, and each produces a different artifact.
Activity Question it answers Output When it runs Threat modeling How could this design be attacked, and what stops it? A data-flow diagram and a threat register During design, then again after every material change Risk assessment How much does each risk matter, and who accepts what is left? Risk ratings, treatment decisions and sign-offs Before deployment and at each review cycle Red teaming Do the controls hold up when a real attacker tries? Findings and evidence from attack attempts Before release and after major changes
In practice, the threat model feeds both of the others. For instance, a full AI security assessment uses the register as its threat input. An AI red teaming exercise then turns the top entries into attack scenarios.
Which Threats Are Specific to AI Systems?
Eight threat types show up in almost every AI threat model, and each one enters at a predictable point on the diagram. The table also gives each a catalogue ID. That way, a finding traces back to the OWASP Top 10 for LLM Applications 2025 or to MITRE ATLAS.
Threat Where it enters the diagram Example Catalogue tag Prompt injection (direct and indirect) User input, as well as retrieved documents, emails and web pages A hidden line in a PDF tells the model to reveal its instructions LLM01 Data and model poisoning Training data, fine-tuning sets and the ingestion pipeline A tampered policy document is indexed as trusted content, so answers change LLM04 Sensitive information disclosure Retrieval layer, model output and logs A user retrieves records that their own role cannot open LLM02 Model extraction and inversion Inference API Repeated queries rebuild a model’s behavior or recover training records MITRE ATLAS techniques Adversarial or evasion inputs Inference input for classic ML models A crafted input slips past a fraud classifier because of tiny changes MITRE ATLAS, NIST AI 100-2 Supply chain compromise Model files, packages, plugins and MCP servers A third-party connector is swapped for a malicious version LLM03 Excessive agency Tool calls and service permissions An assistant with write access deletes records when a request is vague LLM06 Unbounded consumption Inference API and agent loops A looping agent burns through the token budget until someone notices LLM10
How to Use the Threat List
Treat the list as a starting point for the diagram walk rather than a replacement for it. Our LLM security guide covers the controls for each threat. Similarly, the supply chain row gets its own treatment in our guide on AI supply chain security .
Two rows deserve special attention in generative AI security work. In particular, prompt injection and excessive agency combine badly, because injected text can steer a model that holds write access. So that pairing is behind several of the threats in the agent example later in this guide.
On-Demand Webinar
The Real Cost of LLM Security Risks and How to Reduce Them
An on-demand Kanerika webinar on the LLM threats that cost enterprises the most and the practical controls that reduce them.
Watch the Webinar → STRIDE vs MAESTRO vs MITRE ATLAS vs OWASP: Which Framework Does Which Job?
STRIDE is the method you use to walk the diagram, and the others help you tag and check what you find. By contrast, MITRE ATLAS is an attack catalogue and the OWASP lists are coverage checklists. MAESTRO, meanwhile, is a layered model for agent systems. So the honest answer to “which one” is usually “STRIDE plus one or two of the others.”
STRIDE: The Method for Walking a Diagram
STRIDE stands for Spoofing, Tampering, Repudiation, Information disclosure, Denial of service and Elevation of privilege. Microsoft uses these six STRIDE categories to classify threats. Its Threat Modeling Tool then applies them element by element, an approach called STRIDE per element.
Nothing in STRIDE is specific to AI, and that is actually its strength. Each category still applies once you translate it into AI terms.
STRIDE category What it looks like in an AI system Spoofing An agent or service identity is impersonated, or a tool server pretends to be a trusted one Tampering Poisoned documents in the index, altered model weights or edited system prompts Repudiation Tool calls or model decisions that are never logged, so nobody can prove what happened Information disclosure Cross-user retrieval, system prompt leakage and secrets in model output Denial of service Unbounded token consumption, agent loops and oversized inputs Elevation of privilege Excessive agency, when a model acts with more rights than the user who asked
MITRE ATLAS: An Attack Catalogue, Not a Procedure
MITRE ATLAS describes itself as a living knowledge base of adversary tactics and techniques involving AI. It draws on real attacks as well as red-team demonstrations. As of the October 2026 release (v2026.10) , it lists 16 tactics, 129 techniques, 92 sub-techniques, 44 mitigations and 77 case studies.
Those counts change often, so cite the release you used in the register. ATLAS tells you what attackers have already done. However, it does not tell you how to run a workshop, so it pairs with a method rather than replacing one.
OWASP LLM Top 10 and Agentic Top 10: Coverage Checklists
The OWASP Top 10 for LLM Applications 2025 runs from LLM01 Prompt Injection to LLM10 Unbounded Consumption. Use it after the diagram walk, as a check that you did not skip a common class of failure.
Agent systems get a second list as well. The OWASP Top 10 for Agentic Applications for 2026 was published on December 9, 2025. It covers ten risks, from ASI01 Agent Goal Hijack and ASI02 Tool Misuse to ASI10 Rogue Agents.
MAESTRO: A Layered Model for Agents
MAESTRO stands for Multi-Agent Environment, Security, Threat, Risk, and Outcome. The Cloud Security Alliance published it in February 2025 as a threat modeling framework for agentic AI.
It splits an agent system into seven layers. The first four are foundation models, data operations, agent frameworks, and deployment and infrastructure. Then come evaluation and observability, security and compliance, and the agent ecosystem. It also looks for cross-layer threats, such as poisoned data in one layer compromising the model in another.
NIST AI 100-2 for Classic Machine Learning
Predictive models, for example fraud scorers and image classifiers, face a different set of attacks. NIST’s AI 100-2 E2025 taxonomy of adversarial machine learning , published in March 2025, gives the shared vocabulary for evasion, poisoning and privacy attacks.
Choosing the Right Mix
Framework Type Use it when Output Do not use it for STRIDE Method You walk any data-flow diagram Threats per element and boundary Naming specific AI attack techniques MITRE ATLAS Attack catalogue You tag threats with real techniques and mitigations Technique IDs and case references Running the workshop itself OWASP LLM Top 10 2025 Coverage checklist You check an LLM or RAG app for missed risk classes LLM01 to LLM10 tags Ranking threats for your system OWASP Agentic Top 10 2026 Coverage checklist You check a tool-calling or multi-agent system ASI01 to ASI10 tags Systems with no tools or autonomy MAESTRO Layered model Agents have memory, tools and other agents Threats per layer, as well as cross-layer chains A simple chat or RAG app All of them together Combined Most production AI systems, once agents appear One register with STRIDE plus catalogue tags Nothing, since this is the default
So STRIDE runs per element and per boundary, and each threat gets an OWASP or ATLAS tag. MAESTRO layers come in only when the system has agents. For the wider control picture beyond threat modeling, also see our overview of the AI security framework options.
How to Do AI Threat Modeling in 6 Steps
The six steps below follow the four questions in order, and each one ends with an artifact you can hand to someone else. A first pass on a single AI system usually fits into one or two working sessions. After that, the diagram and the register live with the system and change with it.
Step 1: Scope the System, Assets and Attackers
Start with a bounded description. First, name the system, its version, its users and the business process it touches. “The HR policy assistant, v1.3, used by 4,000 staff through Teams” is a scope, while “our AI platform” is not.
Next, list the assets, because the model is rarely the asset that matters most. For instance, embeddings can leak the documents they came from. Credentials, the system prompt, tool permissions and the integrity of decisions all deserve a line too. Sensitive source data counts as well, so bring in your existing data governance classifications.
Then describe the attackers by capability, not by name. An outside user who can only type reaches very little. By comparison, an insider who uploads documents or a compromised connector reaches much more.
Finally, get the right people in the room for a 60 to 90 minute workshop. Invite the system owner, a data engineer, the ML or application engineer, a security lead and someone from operations. At the end, the artifact from this step is a one-page scope note.
AI Assessment
Know Which AI Systems Need a Threat Model First
Score your organization’s AI maturity across data, governance and security to see which systems carry the most risk and where to start.
Start Your AI Assessment → Step 2: Draw the Data-Flow Diagram
Draw every path that data takes, from ingestion and indexing through inference, retrieval, tool execution, logging and feedback. Also use a simple legend for processes, data stores, external entities and flows. Microsoft’s AI guidance specifically asks you to map prompt construction, memory access, tool calls, external data ingestion and human approval points.
Label each flow with three facts, so that reviewers see the risk. Note the identity it runs under, the class of data it carries, and whether it carries instructions, evidence or actions. That third label is where most AI threats hide, because evidence turns into instructions the moment a model reads it.
Separate the control plane from the data plane as well, a habit borrowed from classic security design. System prompts, policies and tool definitions belong to the first, while user content and retrieved documents belong to the second.
Before moving on, trace three requests through the diagram. Follow one normal request, one unauthorized request and one that carries attacker-controlled input. If you cannot follow all three end to end, the diagram is not finished yet.
Step 3: Mark the Trust Boundaries
A trust boundary is any point where data or control passes between parties that trust each other differently. AI systems have more of them than teams expect. In fact, six come up again and again.
User to application, when untrusted input arrives. Source repository to the ingestion pipeline, before documents enter the index. Retrieved content to the instruction context, where evidence meets the prompt. User identity to service identity, where the model starts acting on someone’s behalf. Model output to a write in a business system, where text becomes an action. Internal systems to an externally hosted model, once data leaves your control. Draw each boundary as a dashed line on the diagram. Then every flow that crosses one gets analyzed in the next step.
Step 4: Enumerate Threats per Element and Boundary
Walk the diagram with STRIDE, element by element and boundary by boundary. Next, for each threat you find, add an OWASP or ATLAS tag so a reviewer can trace it to the public catalogues.
Write every threat as one full sentence, so that it names the attacker, the entry point, any prerequisite, the action, the asset and the consequence. A weak entry reads “prompt injection risk.” A strong one reads “An employee plants instructions in a shared-drive PDF, so the assistant leaks HR records.”
The strong version tells the engineer exactly what to fix and the tester exactly what to try. It also lets a risk owner judge the impact without a follow-up meeting. Give each threat an ID now, because the register and the test plan will refer back to it.
Step 5: Prioritize and Map Controls
Score each threat for likelihood and impact on the scale your organization already uses. A 1 to 3 or 1 to 5 scale works, and there is no universal formula for AI threats. Using the same scale across systems matters more than the exact numbers.
Then map at least one control to every high-priority threat, along with a testable acceptance condition. For example, “retrieval filters results by the user’s source permissions” is a control. “A test user without HR access receives zero HR chunks for 50 probe queries” is the acceptance condition that proves it.
Keep one rule in view throughout this step. A system prompt is not an authorization layer, because the model can be talked out of it. Output filters and AI guardrails help, but they sit on top of access control, not in place of it. Microsoft’s guidance makes the same point: high-impact or irreversible actions should be constrained by validation, policy checks or human approval.
Step 6: Write the Threat Register and Hand It Off
The register is the deliverable. It holds every threat with its ID, scenario, tags, scores, controls, tests, owner and status. Version it with the diagram, so anyone can see which design a threat applies to. Auditors working to a broader AI governance framework will ask for exactly this trail.
Then hand it to the people who act on it. The risk owner rates and accepts residual risk, and the register slots straight into an AI risk assessment template . The red team turns the top entries into test cases, and engineering picks up the open controls. Teams mapping to the NIST AI RMF can file it under the Map and Measure functions, as our NIST AI RMF playbook explains.
A threat model that nobody reads is just a diagram on a wall. The hand-off makes it part of how the system gets built and approved. That is also how it fits a wider AI governance program.
Worked Example 1: Threat-Modeling a RAG Assistant
Take the HR policy assistant from Step 1. It is a retrieval-augmented generation system, so it answers questions by pulling chunks from an index of company documents. This architecture is illustrative, though it matches a very common enterprise pattern for retrieval-augmented generation .
The Data-Flow Diagram
Its diagram has two paths. One moves documents into the index, while the other answers a user. Both paths share the vector store in the middle.
INGESTION PATH
[SharePoint / file shares] --(documents + ACLs)--> [Ingestion job] <-- boundary B2
[Ingestion job] --(chunks + embeddings + ACL metadata)--> [(Vector store)]
QUERY PATH
[Employee] --(question, user identity)--> [Assistant app] <-- boundary B1
[Assistant app] --(query embedding, user groups)--> [(Vector store)]
[(Vector store)] --(top-k chunks = EVIDENCE)--> [Prompt builder] <-- boundary B3
[Prompt builder] --(system prompt + chunks)--> [Hosted LLM] <-- boundary B6
[Hosted LLM] --(answer)--> [Assistant app] --> [Employee]
[Assistant app] --(prompt, answer, user id)--> [(Audit log)]Trust Boundaries on the Diagram
Four boundaries matter here, and numbering them makes the register easier to read later. B1 sits between the employee and the app, and B2 sits between the document sources and the ingestion job. B3 is where retrieved chunks join the prompt, and B6 is where data leaves for the hosted model.
Of these, B3 is the one teams tend to miss. Retrieved chunks are meant to be evidence, but the model reads them with the same attention it gives instructions.
Threats Found and the Controls for Each
ID Threat Boundary and tags Control Test R-01 Cross-user retrieval, because chunks lost their source ACL metadata at ingestion B2, B3. Information disclosure, LLM02 Carry ACLs into chunk metadata, then filter by user groups at query time A user without HR access gets zero HR chunks across 50 probe queries R-02 Indirect prompt injection planted in an uploaded document B3. Tampering and elevation of privilege, LLM01 Mark retrieved text as untrusted evidence, and also limit output scope Seeded injection documents fail to change the assistant’s behavior R-03 Vector store exposed, so embeddings can be inverted to recover source text Data store. Information disclosure, LLM08 Private networking and access control on the store, with no public endpoints A scan confirms no anonymous access to the index R-04 Poisoned source document indexed as trusted policy B2. Tampering, LLM04 Ingest only from approved libraries that have owners and change history The ingestion job rejects an unapproved library
In assistants like this one, R-01 and R-02 deserve the first fixes, because an ordinary user or uploader can trigger them. For a deeper look at the controls behind each row, see our guide to RAG security .
Notice that none of these threats lives inside the model itself. They sit in the plumbing around it, which is exactly why the diagram matters more than the model card.
Case Study
65% Self-Service Resolution with an AI Support Agent
Kanerika built an AI support agent connected to knowledge bases and Zendesk that resolves 65% of member queries and cut ticket volume by 42%.
Read the Case Study → Worked Example 2: Threat-Modeling a Tool-Calling Agent
Now give the same assistant tools, so it can open HR tickets, update an employee’s address in the HR system and send confirmation emails. Because the diagram from Example 1 still applies, the threat IDs carry over and new ones start at A-01.
Extending the Diagram With the Action Path
An action path now joins the query path. First, the planner proposes a tool call and a policy gateway decides whether it is allowed. After that, a scoped connector or Model Context Protocol server runs it against the business system. The result flows back into the context, and a memory store keeps notes between sessions.
In addition, a human approval step sits in front of any irreversible action. On the diagram, it is a process with its own identity, not a footnote.
New Trust Boundaries
Four new boundaries appear. One separates an action the model suggests from an action the system authorizes. Another separates the user’s identity from the service identity the connector uses.
A third separates read operations from writes. Finally, a fourth separates trusted policy from arguments the model generated, because a model can fill a tool parameter with anything it has read.
Threats Found
A-01 is the one that keeps security leads up at night. Injected content in a retrieved document or email persuades the agent to call the address-update tool for the wrong employee. It maps to ASI01 Agent Goal Hijack and ASI02 Tool Misuse. In MAESTRO terms, it starts in data operations and then lands in the agent framework layer.
A-02 is quieter but just as dangerous, an over-scoped service credential. The connector runs as an account that can edit every employee record, so one bad call has company-wide reach. That is ASI03 Identity and Privilege Abuse, and it sits in the deployment and infrastructure layer.
A-03 is a malicious or swapped MCP server that returns poisoned results, mapped to ASI04 Agentic Supply Chain Vulnerabilities. A-04 is memory poisoning, where a planted note shapes later sessions, mapped to ASI06. A-05 is an unlogged action, a plain STRIDE repudiation threat that MAESTRO places in evaluation and observability.
Controls and How to Test Them
The controls follow the boundaries, so each tool gets an allowlist of actions. The gateway validates every argument against policy rather than trusting the model. Credentials are scoped per user or per task, and irreversible writes wait for human approval.
Every action is also logged with the requesting user, the tool, the arguments and the result. The most useful test is a negative one, and it belongs in your wider AI agent evaluation suite. Feed the agent an injected instruction to update a different employee, then confirm the gateway rejects the write and the log records the attempt.
This example deliberately stays at the level of the threat model. Our guide to agentic AI security maps the full agent attack surface. Likewise, the guide to AI agent security tools compares runtime enforcement options. Permission design is covered separately in our agentic AI permission control framework .
AI Threat Register Template
The register is where the two examples end up, and it is the artifact auditors and risk owners actually read. One row per threat is enough, so long as every column is filled in for high-priority entries.
ID Element or flow Threat scenario STRIDE and tag Score (L x I) Control and test Owner and status R-01 (v1.3) Vector store to prompt builder A user retrieves chunks from documents that their role cannot open I, LLM02 3 x 3 ACL filter at query time, then 50 probe queries with zero leaks Data engineering lead, open R-02 (v1.3) Retrieved chunks into context An uploaded PDF carries instructions, so the answer changes T and E, LLM01 2 x 3 Untrusted-evidence marking, tested with a seeded injection set App engineering lead, in progress A-01 (v2.0) Planner to HR update tool An injected email triggers an address update for the wrong employee T and E, ASI01 and ASI02 2 x 3 Gateway argument checks and approval for writes, then a negative write test Platform security lead, open
Add two more columns in your own copy, for the last review date and the risk assessment reference. That reference then links each row to the decision record, so residual risk is accepted by a named person.
Roles should also be explicit. The security lead documents the threats, engineering implements the controls, and the system owner or risk owner accepts what is left. When those three are the same person, however, the register tends to drift.
Can AI Automate Threat Modeling?
Partly. A large language model can read a diagram description and draft a first list of STRIDE threats in minutes. That draft is a useful prompt for the workshop. Still, it cannot know your actual permissions, data classes or business impact, so a human still owns scope, boundaries and priorities.
Tooling helps with the structure around the analysis, while runtime products such as the LLM security tools we compared enforce the controls afterwards. The Microsoft Threat Modeling Tool is a core part of Microsoft’s Security Development Lifecycle and offers guided STRIDE per element analysis. OWASP Threat Dragon is a free, open-source application for drawing threat model diagrams and listing threats per element.
Treat AI-drafted threats like a junior analyst’s notes. They make a good starting list. But every entry still needs a human to confirm it applies and to rewrite it as a full threat sentence.
Checklist
AI Governance Checklist
A practical checklist for the policies, owners, reviews and evidence that turn a threat register into governed AI in production.
Get the Checklist → When to Re-Run the Threat Model
An AI threat model goes stale when the system changes, not when the calendar turns. So tie reviews to change events and version the diagram and register with each release.
Trigger What to re-check Model swap or provider change Data residency, output behavior and injection resistance New data source Ingestion boundary, ACL carry-over and poisoning exposure New tool or MCP server Action path, argument validation and supply chain trust Permission change Service identity scope, as well as read and write separation More autonomy Approval points, irreversible actions and logging Incident or near miss The threat that fired, then its neighbors on the diagram
A review is good enough when three things hold. First, someone who runs the system has checked the diagram. Second, every trust boundary has at least one analyzed threat. Third, every high-priority threat has an owner and a passing or scheduled test.
How Kanerika Approaches AI Threat Modeling
Kanerika builds AI assistants and agents on Microsoft, Databricks and Snowflake platforms. That means a threat model can start from the real connectors and permissions rather than a whiteboard sketch. The sequence runs from scoping with data owners to a diagram built from real service identities. Then come the register, platform controls and a hand-off to governance.
Kanerika Service
AI Governance Consulting Services
Kanerika sets up the policies, review triggers and evidence trail that keep AI threat models and controls current as systems change.
Explore AI Governance Services The two worked examples above also mirror system shapes Kanerika delivers. One is a support agent connected to knowledge bases and Zendesk that resolves 65% of member queries and cut ticket volume by 42%. The other is a compliance agent that reads internal databases and public sources to vet experts three times faster, with 70% fewer backlog cases.
Three pitfalls are common in systems like these. Connectors inherit a broad service account’s scope, retrieval ignores the source system’s ACLs, and teams skip approvals for writes they consider low risk. All three are easy to spot once the diagram is honestly labelled, and easy to miss when it is not.
Kanerika delivers under ISO 27001 and SOC 2 Type II certifications. For clients that run Microsoft Purview , its sensitivity labels supply the data class for each flow on the diagram. Kanerika’s agentic AI services team builds the agents themselves. Kanerika’s AI governance services help teams put the register into practice, from policies to evidence reporting. Before that, the AI governance checklist is a good place to start.
Case Study
3x Faster Expert Vetting with an AI Compliance Agent
Kanerika built a compliance agent that checks internal databases and public sources and produces cited reports. Expert vetting became 3x faster, with 70% fewer backlog cases.
Read the Case Study → From Diagram to Register: What AI Threat Modeling Gives You
AI threat modeling answers one practical question. How could this specific system be attacked, and what stops each attack? Six steps get you there, from a scoped system and a labelled diagram to ranked controls and a register.
STRIDE drives the walk, while ATLAS and OWASP tag the findings, and MAESTRO joins when agents do. The finished register then goes to the risk owner, the red team and the engineers. After that, it gets updated every time the system changes. That loop is what turns a one-off security review into responsible AI in daily practice.
Frequently Asked Questions
What is AI threat modeling?
AI threat modeling is a structured way to find how an AI system could be attacked before it ships. Teams map the model, data flows, tools and trust boundaries, list threats with STRIDE and AI catalogues, rank them, and record each one with a control, a test and an owner in a threat register.
Can STRIDE be used for AI systems?
Yes. STRIDE works well for AI systems once each category is translated into AI terms. Tampering covers poisoned documents and altered weights, information disclosure covers cross-user retrieval, and elevation of privilege covers excessive agency. Walk it per element and per trust boundary, then tag each threat with an OWASP or MITRE ATLAS ID.
Is MITRE ATLAS a threat modeling framework?
MITRE ATLAS is a knowledge base of adversary tactics and techniques against AI, built from real attacks and red-team demonstrations. It is a catalogue rather than a procedure. Use a method such as STRIDE to walk your diagram, then use ATLAS technique IDs to tag threats and find documented mitigations for them.
What is the difference between MAESTRO and STRIDE?
STRIDE is a general method with six threat categories that you apply to each element of a data-flow diagram. MAESTRO, published by the Cloud Security Alliance in 2025, is a seven-layer model built for agentic AI that also looks for threats crossing layers. Many teams use STRIDE first and add MAESTRO for agents.
Is the OWASP LLM Top 10 enough to threat-model an AI app?
No. The OWASP Top 10 for LLM Applications is a coverage checklist of common risk classes, so it shows what you might have missed. It does not describe your architecture, trust boundaries or business impact. Draw the data-flow diagram, walk it with STRIDE, and then use the OWASP list as a final cross-check.
How is AI threat modeling different from red teaming and risk assessment?
Threat modeling asks how a design could be attacked and produces a diagram and threat register. Risk assessment decides how much each risk matters and who accepts what remains. Red teaming tests whether the controls actually hold against attack. The threat register usually feeds both of the other two activities as their starting input.
How often should an AI threat model be updated?
Update an AI threat model when the system changes rather than on a fixed calendar. Typical triggers are a model or provider swap, a new data source, a new tool or MCP server, a permission change, more autonomy, or an incident. Version the diagram and threat register with each release so reviewers know what applies.
Can AI tools automate threat modeling?
Partly. A language model can draft a first list of STRIDE threats from a diagram description, which speeds up the workshop. It cannot know your real permissions, data classes or business impact, so people still own the scope, trust boundaries and priorities. Treat AI-drafted threats as notes that each need human review.
Who should take part in an AI threat modeling session?
Bring the system owner, a data engineer, the ML or application engineer, a security lead and someone from operations. Together they know the data sources, the permissions, the prompts and the runtime behavior. A first session on a single AI system usually takes 60 to 90 minutes, followed by written follow-up on the register.
What should an AI threat register contain?
An AI threat register should list each threat with an ID, the system version, the affected element or flow, a full threat sentence, its STRIDE category and catalogue tag, likelihood and impact scores, the control, the test or evidence, an owner, a status, and a link to the risk assessment record.
