TL;DR
LLM security best practices start with four core controls, then add data protection for chat, index security for RAG and tool limits for agents. The four core controls are an inventory, permissions at the data source, prompt injection defenses and output validation. This guide ranks 12 controls against the OWASP Top 10 for LLM Applications 2026, with a setup step and a test for each. Self-hosted models add supply chain and hosting work, while API-based apps lean harder on data and identity controls. Firewalls alone cannot stop these attacks, because the attack arrives as ordinary text. With IBM’s 2026 report putting the global average breach at $4.99 million, spend the first 30 days on the four core controls.
Key Takeaways
LLM risk lives where instructions and data share one channel, so network tools such as firewalls miss most of it.
The OWASP LLM Top 10 2026 replaces System Prompt Leakage with the broader Hidden Context Exposure at LLM08 and moves Excessive Agency up to LLM03.
Permissions belong at the data source, so retrieval runs with the asking user’s rights instead of a shared super-user account.
Agents should climb a permission ladder from read to draft to act with approval, and every MCP server they reach needs vetting.
Every control needs a test, so security evals belong in the same CI pipeline that ships prompt and model changes.
After the AI Omnibus, EU AI Act high-risk obligations start December 2, 2027 for Annex III use cases, so 2026 and 2027 are the window to map controls.
Watch on YouTube
AI Security for Enterprises: Kanerika’s CISO on Prompt Injection, ISO 27001 & SOC 2
Kanerika’s CISO explains how enterprises handle prompt injection and where AI security fits next to ISO 27001 and SOC 2 controls.
A Supplier PDF, a Pricing Table and an Email Tool
Picture a procurement assistant (an illustrative scenario) built on retrieval-augmented generation. It answers buyer questions from supplier contracts, such as payment terms. It can also read internal pricing records and send email on a buyer’s behalf.
A supplier uploads a PDF with one line of white text that no person will ever see. Once indexed, that line tells the assistant to email the current pricing table to an outside address. The assistant obeys, because to a model, retrieved text and commands arrive in the same stream.
OWASP lists this pattern under LLM01 as indirect prompt injection. The hostile instruction hides in content the model reads, not in anything a user types. The buyer who asked the question never sees it happen.
Every request in that chain is legitimate traffic, so no firewall rule fires. A quiet leak like this is what breach averages price in. IBM’s Cost of a Data Breach Report 2026 puts the global average breach at $4.99 million, up 12% on the prior year.
What LLM Security Covers and Why App Security Alone Misses It
LLM security is the set of controls that stop a language model application from leaking data, obeying hostile text or taking actions nobody approved. It covers the model, the prompts, the data the model reads, the tools it can call and the logs it leaves. Put simply, it protects the parts of an AI app that a firewall cannot read.
This guide covers the application layer, meaning the chat assistants, RAG apps and agents your teams build. Our AI security framework covers org-wide governance. The framework sets the policy, and the 12 controls below put that policy into code.
Three traits set LLM apps apart from ordinary software. Instructions and data share one channel, so a document can carry a command. Outputs are untrusted, because a model can produce SQL, code or links that look right and are not.
The third trait is that tools turn text into actions. A bad answer can then become a sent email, a changed record or a payment nobody approved.
That is why network controls see so little of the risk. Firewalls and web application firewalls read packets and request patterns rather than intent. A prompt injection looks like a normal question, so it passes. Data loss prevention rules struggle too, since a model can paraphrase a secret instead of copying it.
This article is a ranked checklist rather than an architecture model. For layer-by-layer design, read about the seven-layer generative AI security model . Here the focus is which of 12 controls to build first, how to set each one up and how to prove it works.
AI Assessment
See Where Your AI Program Stands
Take Kanerika’s AI maturity assessment to see where your AI program stands before you choose which of the 12 controls to build first.
Start Your AI Assessment → Which Deployments Need Which Controls
Risk grows with what the application can reach. A chat-only assistant mostly risks leaking what users paste in, while an agent with tools can act on a hostile instruction. The table maps four common deployment types to their main risk and the controls each type adds on top of the four every deployment needs.
Deployment Main risk Controls specific to this type (on top of the core four)
Chat-only assistant Sensitive data pasted into prompts; unsafe or wrong answers Data redaction (5), logging (11), usage caps (10) RAG application Users retrieving documents they should not see; instructions hidden in content Index and chunk permissions (4), data protection (5) Agent with tools Hostile text turned into emails, payments or record changes Tool allow-lists, MCP server vetting and approvals (6), kill switch (11) Self-hosted model Tampered model files, stolen weights, exposed endpoints, runaway compute cost Supply chain checks (8), network isolation (9), rate limits (10)
In practice, most real applications mix these types. A RAG assistant that can also file tickets is an agent, so it inherits both rows.
Where Controls Sit in an LLM Application
A typical request passes through eight stops, and seven of them can hold a control. The user’s message first hits a gateway that handles identity, rate limits and input screening. After that, the app assembles a prompt from system instructions and user text, and retrieval adds matching content from the index.
The model generates a response, an output check validates it, and only then can a tool act on it. Finally, logs record every step for detection and audits. The diagram shows the order, and each practice below points back to one of these stops. A category-by-category map of products for each of these stops is in LLM security tools .
The OWASP GenAI LLM Top 10 2026 at a Glance
OWASP maintains the Top 10 for LLM Applications, a ranked list of the main security risks in LLM applications. It gives security and engineering teams one shared vocabulary for threat reviews.
The OWASP GenAI Security Project published the 2026 edition on August 4, 2026, in its GenAI-LLM-Top10 repository on GitHub . Compared with 2025, it replaces System Prompt Leakage with the broader Hidden Context Exposure at LLM08 and moves Excessive Agency up to LLM03. The release also updates the ordering of the rest of the list, so Improper Output Handling now sits at LLM10. As of November 2026, the 2025 list sits in the legacy OWASP project repository , which describes itself as a historical archive.
ID Risk What it looks like in production Practice #
LLM01 Prompt Injection A support ticket or web page tells the model to ignore its rules 3 LLM02 Sensitive Information Disclosure The assistant quotes a salary file or a customer record in an answer 2, 5 LLM03 Excessive Agency An agent with send and delete rights acts on a hostile instruction 6 LLM04 Supply Chain A downloaded model file or plugin carries malicious code 8 LLM05 Data and Model Poisoning Tampered fine-tuning data plants a hidden behavior 8, 12 LLM06 Unbounded Consumption A script floods the endpoint and runs up token costs 10 LLM07 Misinformation A confident but wrong answer reaches a customer or a report 7, 12 LLM08 Hidden Context Exposure A system prompt or hidden context shows up in a user’s reply 3, 5 LLM09 Vector and Embedding Weaknesses A vector search returns chunks from another tenant or role 4 LLM10 Improper Output Handling Generated SQL or code runs with no checks 7
What the 2026 Changes Mean for Your Controls
Read the last column as a map. A risk with two practice numbers needs both controls, because each one covers a different stop in the request path. LLM07 is mostly about accuracy, and our guide to AI hallucinations covers it in depth.
Both headline changes, LLM03 and LLM08, are about what the model can do or see, not what users type. That moves effort toward agent permissions and context hygiene, which practices 6 and 5 cover below.
12 LLM Security Best Practices
The 12 LLM security best practices below are ranked by how much risk each removes for the effort it takes. Each one explains the control, gives setup steps and ends with a test. Four of them, practices 1, 2, 3 and 7, apply to every deployment and come first.
The rest depend on what your app can reach. Treat the tests as the real deliverable, since a control nobody has tested is a plan, not a defense.
1. Inventory Every Model, Prompt, Data Source and Tool
You cannot secure an LLM app you do not know exists. So the first job is a list of every model, prompt, data source and tool in use. Give each entry an owner and a data class.
Shadow AI is the hardest part to see. A sales rep pastes a pipeline report into a personal chatbot account, and that data now sits outside every control you own. Discovery means reading SaaS sign-ups, network traffic and expense claims, because people rarely report the AI tools they pay for themselves.
Agents add a newer entry type. Every Model Context Protocol (MCP) server an agent can reach is a tool with its own permissions, so it belongs on the list too. Keep production and experiments in separate entries, since a pilot that touches real customer data is already production.
How to implement:
Pull SaaS, network and expense data, then survey teams.
Record each app’s model and version, system prompt, data sources, tools, MCP servers and business owner.
Keep prompts and model versions in a registry with change history, so every edit is traceable like code.
Separate production from experiments, and tie the list to your AI governance process so new apps register before launch.
How to test it: ask five teams which AI tools they use, then confirm each tool appears in the inventory with a named owner.
2. Enforce Identity and Least Privilege at the Data Source
The model should never decide who sees what. Permissions belong at the data source, and retrieval should run with the asking user’s own rights.
Go back to the procurement scenario. A buyer who cannot open the pricing table in the source system should get nothing from it through the assistant. If the assistant reads through one service account with access to everything, every user inherits that access.
The fix is role-based access control (RBAC) for AI workflows. Permission-aware retrieval passes the user’s identity through, so the index returns only what that person could open anyway.
How to implement:
Authenticate users through your identity provider before any prompt runs.
Pass the user’s identity to retrieval and tool calls instead of using shared super-user service accounts.
Filter search results by the source system’s access lists when each query runs.
Give the app’s own credentials the narrowest read scope that still works.
How to test it: ask one question as two users with different roles, and confirm the restricted user sees nothing from blocked documents.
Kanerika applied this pattern for a global investment bank, and the Kanerika section below covers the results.
Case Study
43% Faster Data Retrieval for a Global Investment Bank
Kanerika built chat interfaces over RFI documents and enterprise databases on FLIP with secure role-based access control, reaching 43% faster information retrieval, 100% role-based compliance and 35% higher workforce efficiency.
Read the Case Study → 3. Treat Every Input as Untrusted: Direct and Indirect Prompt Injection
Prompt injection, LLM01, is text that makes the model ignore its instructions. Direct injection happens when a user types it. Indirect injection hides in content the model reads, such as a web page, an email or the supplier PDF from the opening scenario.
Filters miss some attempts, so the goal is to cut the hit rate and limit the damage when one lands. Good prompt engineering helps, but a well-written system prompt is not a security boundary. Keep secrets, keys and internal URLs out of it entirely, since system prompts can leak.
Input classifiers add a useful layer. Microsoft’s Azure AI Content Safety Prompt Shields detects both user prompt attacks and document attacks. For agents, it can also scan tool responses from supported tools, a preview feature. It analyzes text, so extract the text from scans, slides and PDFs before screening them, since those files can carry injected instructions too.
Check the defaults before you rely on it. As of November 2026, Microsoft Foundry turns user prompt attack detection on by default, but document attack detection stays off until you enable it. The standalone Content Safety API checks one user prompt and up to five documents per call.
How to implement:
Put system, user and retrieved content in separate message roles, then label retrieved text as untrusted data.
Run an input classifier on user prompts and on retrieved documents before they reach the model.
Move secrets out of system prompts and into a vault, so the model cannot read them.
Add AI guardrails at the gateway to block known attack patterns.
How to test it: plant a document with a hidden instruction in a test index and confirm the app neither follows it nor repeats it.
4. Secure RAG Pipelines and Vector Databases
A RAG pipeline is only as safe as its index. Vector and embedding weaknesses, LLM09, appear when the index forgets who may read each chunk or when poisoned content slips in at ingestion.
For example, one failure pattern is an index built from a shared drive with every permission stripped. Each chunk then becomes readable by every user of the assistant. The fix is to carry document-level access lists into the index and check them again at query time.
Whether you choose RAG or fine-tuning changes the risk too. Data baked into model weights cannot be filtered per user, so permissioned data belongs in retrieval.
How to implement:
Copy each document’s access list into chunk metadata when you ingest it.
Scan new content for hidden text, scripts and injection patterns before indexing.
Separate tenants into distinct indexes or namespaces rather than relying on query filters.
Re-check the user’s permission on every retrieved chunk before it enters the prompt.
How to test it: as a low-privilege user, search for a phrase that appears only in a restricted document and expect an empty result.
Kanerika Service
Generative AI Services
Kanerika designs and builds RAG assistants and LLM applications with permission-aware retrieval, injection screening and output checks planned in from the first release.
Explore Generative AI Services 5. Keep Sensitive Data and Secrets Out of Prompts and Context
Sensitive Information Disclosure (LLM02) and Hidden Context Exposure (LLM08) share one root cause. Data reached the model that the task did not need, so the cheapest fix is to stop it at the source.
Classify data before it feeds an AI app, then mask or drop what the task does not use. A support assistant needs an order status, not a full card number.
Labels have to exist before documents are indexed, so classify personal, payment card and health data (PII, PCI and PHI) first. An unlabeled chunk gives no retrieval filter a reason to hold it back. Microsoft Purview Information Protection applies sensitivity labels that retrieval can then respect.
Our guide to data anonymization techniques covers masking and tokenization in more depth. Our look at data security in AI covers the wider data controls.
How to implement:
Classify sources before indexing, then block restricted classes from AI use by default.
Redact personal data and secrets from prompts and retrieved chunks before the model call.
Set retention limits on prompt and response logs, and also redact those logs.
Confirm in your contract whether the model provider may train on your data.
How to test it: run an eval set seeded with fake card numbers and personal data, then measure how often any of it leaks.
6. Limit Agency: Tool Allow-Lists, Scoped Credentials and Approvals
Excessive Agency, LLM03 in the 2026 list, is what turns a bad answer into a bad action. An agent that can read, write, send and delete will sooner or later meet text that asks it to do the wrong one.
So grant tools in rungs. Start with read access, move to drafts a person reviews, then to actions with approval. Give full autonomy only to low-risk tasks you can reverse later. In the procurement scenario, an email tool limited to internal addresses, with approval for attachments, would have stopped the leak. A fuller grant model, with permission classes and a revocation path, is in the agentic AI permission control framework .
Model Context Protocol (MCP) servers need the same discipline. Treat each one as a third-party integration: pin its version, scope its credentials and review its tool descriptions before use. Those descriptions are text the model reads, so a tampered one can carry injected instructions straight into the agent’s plan.
Our agentic AI security threat model goes deeper on agent identity and multi-agent risk. Our look at OpenClaw security risks shows what broad agent access can expose.
How to implement:
List each tool an agent may call, then deny everything else.
Issue short-lived, task-scoped credentials per agent instead of shared keys.
Approve MCP servers by pinned version, and re-review their tool descriptions after every update.
Require human approval before payments, deletions, external messages and permission changes.
Cap the number of actions an agent may take per task, so a loop cannot run away.
How to test it: use an injected document to request a tool outside the allow-list, and confirm the call is refused and logged.
7. Validate and Encode Outputs Before Anything Executes Them
Improper Output Handling, LLM10, happens when other systems trust model output. So treat generated SQL, code, HTML and API calls like input from a stranger.
The safer pattern asks the model for a structured plan instead of raw code. Your application checks that plan against an allow-list, then builds the real query with parameters. The short example below shows the idea for a reporting assistant.
ALLOWED = {"orders": {"order_id", "status", "total"},
"suppliers": {"supplier_id", "name", "region"}}
def run_model_query(plan, db, user):
# The model returns a JSON plan, never raw SQL.
table, cols = plan["table"], set(plan["columns"])
if table not in ALLOWED or not cols <= ALLOWED[table]:
raise PermissionError("Plan is outside the allow-list")
col_sql = ", ".join(sorted(cols))
sql = f"SELECT {col_sql} FROM {table} WHERE owner_id = %s LIMIT 100"
return db.execute(sql, (user.id,))Table and column names come only from the allow-list, and the user’s ID travels as a bound parameter. Anything that renders in a browser needs HTML encoding too, or a reply can carry a script. Code an assistant writes should run in a sandbox with no network or production access.
How to implement:
Require structured output, such as JSON checked against a schema, for anything a system will execute.
Validate plans against allow-listed tables, fields, endpoints and parameters.
Use parameterized queries, and encode HTML before display.
Run generated code in an isolated sandbox that holds no secrets.
How to test it: send prompts that try to produce a DROP TABLE statement or a script tag, and confirm both are blocked.
Watch on YouTube
AI Agent Access Control: From Excessive Agency to Least Privilege
A Kanerika walkthrough of moving AI agents from excessive agency to least privilege, the goal behind the practice 6 permission ladder.
8. Secure the Model and Data Supply Chain
Supply Chain (LLM04) and Data and Model Poisoning (LLM05) attack the app before it ever runs. A model file, a fine-tuning dataset or a plugin can arrive already tampered with.
Teams that adopt open-source LLMs carry more of this work, because they pull weights, tokenizers and libraries from public hubs. Pin exact versions, then check where each artifact came from. Then keep an AI bill of materials that lists models, datasets and dependencies like any other software inventory. Our guide to AI supply chain security shows how to pin, scan and record each model and package.
The model itself is also worth stealing. Attackers can copy a model’s behavior by sending high volumes of queries to its API and training on the answers. Others go after the weights directly on a poorly secured host.
Model inversion is the quieter cousin of theft, rebuilding sensitive training data from what the model outputs. IBM’s 2026 breach report puts the average breach involving an AI model inversion attack at $6 million. Rate limits from practice 10 and alerts on unusual query patterns make extraction slow and visible.
How to implement:
Download models only from approved sources, then verify checksums or signatures.
Prefer safe model file formats, and scan model files and dependencies before use.
Pin model, library and dataset versions in an AI bill of materials.
Review fine-tuning data for planted triggers before training, and keep its lineage.
Store weights encrypted with tight access, and alert when one key sends systematic, high-volume queries.
How to test it: rebuild the app from its bill of materials in a clean environment and confirm every artifact matches its recorded hash.
9. Isolate Model Hosting and Network Paths
Where the model runs decides who can reach it. A public inference endpoint guarded by one static key invites misuse and cost abuse.
API-hosted models move hosting security to the provider, but your data leaves your network on every call. Private LLMs keep data inside your boundary, and in exchange your team owns patching, isolation and capacity. Neither choice is safer by default, so let data sensitivity and your team’s skills decide.
How to implement:
Reach hosted models through private endpoints or private networking instead of the public internet.
Keep self-hosted inference servers on internal networks behind the gateway.
Isolate tenants and environments so test traffic never touches production data.
Store API keys in a managed vault and rotate them on a schedule.
How to test it: scan from outside your network for any reachable inference endpoint, where the expected result is none.
10. Cap Consumption: Rate Limits, Token Budgets and Cost Alarms
Unbounded Consumption, LLM06, covers request floods and prompts built to burn tokens. The result is an outage, a surprise invoice or both.
Limits work best at the LLM gateway , set per user and per app. Give each request and each day a token budget, cap context and output length, and alert both finance and the platform team when spend jumps.
How to implement:
Apply per-user and per-key rate limits at the gateway, since every request passes through it.
Cap input size, output length and agent loop counts.
Set daily cost budgets with alerts at agreed thresholds.
How to test it: run a load script above the limit and confirm requests are throttled and an alert reaches its owner.
Talk to Kanerika
Plan an LLM Security Review
Walk through your LLM application with Kanerika to see which of the 12 practices you have and which to build first.
Schedule a Demo → 11. Log, Monitor and Rehearse an LLM Incident Runbook
Every investigation starts from a record of what the model saw and did. Log prompts, retrieved sources, responses and tool calls, with personal data redacted first.
Then alert on the signals that matter, such as injection classifier hits, refused tool calls, unusual data volumes and cost spikes. LLMOps observability tools collect these traces in one place, so investigators are not hunting across systems. The runbook should name who can switch an agent off, how to revoke its credentials and how to tell affected users.
How to implement:
Log prompts, context sources, outputs and tool calls with redaction, then set retention.
Send alerts to the monitoring stack or SIEM your security team already runs.
Build a kill switch that disables an agent or a tool in one step.
Rehearse the runbook each quarter with a tabletop exercise.
How to test it: during a drill, time how long it takes to revoke one agent’s credentials, and track that number.
12. Make Testing Continuous: Evals and Red Teaming in CI
Models, prompts and data all change, so a one-time test goes stale fast. Run security evaluations through your LLM evaluation framework in the same pipeline that ships prompt and model changes.
Open-source tools cover much of the ground. garak is NVIDIA’s open-source LLM vulnerability scanner. Microsoft’s PyRIT is a Python risk identification tool for generative AI. NeMo Guardrails adds programmable guardrails to LLM apps, which you can then test.
Tool Type What it tests
garak Open-source vulnerability scanner (NVIDIA, Apache-2.0) Scans an LLM for known weaknesses with automated probes PyRIT Open-source risk identification tool (Microsoft) Helps red teams find risks in generative AI systems NeMo Guardrails Open-source guardrail toolkit (NVIDIA) Whether programmable rails on your app hold under attack prompts
garak runs from the command line, so it fits into a CI job. The first command below lists the available probes. The second runs the encoding probes against an OpenAI model once OPENAI_API_KEY is exported.
garak --list_probes
python3 -m garak --target_type openai --target_name gpt-6-luna --spec probes.encodinggarak’s README example uses an older model id, and any current OpenAI model id works in its place. Changing the spec to probes.promptinject runs only the PromptInject framework’s methods, which suits a prompt injection gate.
Automated tools still do not replace people. Schedule human testing with LLM red teaming methods before major launches and after big model changes, and track each finding until it closes.
How to implement:
Build an eval set of attack prompts, leakage probes and expected refusals before the next release.
Run it in CI on every prompt, model or retrieval change, and block releases that regress.
Add manual red teaming before launches and after model upgrades.
How to test it: weaken a system prompt in a test branch and confirm the pipeline flags the drop in injection pass rate.
Which LLM Security Best Practices to Implement First
Start with the controls that stop data from leaving and actions from running unchecked. Those are the inventory, permissions at the data source, injection defenses and output validation, and every deployment needs all four.
An AI security assessment is the quickest way to learn which of the four you already have. It also shows which deployment types you really run, and that decides the order of the rest.
After the core four, the order depends on what you run. The table shows the next priorities for each deployment type.
Deployment Then add Before you scale
Chat-only assistant 5 data protection, 11 logging 10 consumption caps, 12 continuous testing RAG application 4 RAG and index security, 5 data protection 11 logging, 12 continuous testing Agent with tools 6 agency limits and MCP vetting, 11 kill switch and logging 10 consumption caps, 12 continuous testing Self-hosted model 8 supply chain, 9 network isolation 10 consumption caps, 11 logging, 12 testing
A 90-day plan keeps that order honest and gives each control an owner and a date.
First 30 days: build the inventory, move retrieval to user-level permissions, screen inputs and take secrets out of system prompts. Validate output on anything that executes code or queries.Days 31 to 60: extend output checks to every response, add redacted logging and rate limits, then tool allow-lists with approvals.Days 61 to 90: put security evals in CI, run a first red-team cycle and rehearse the incident runbook.
Automate as many of these checks as you can from the start. IBM’s 2026 report found that organizations using security AI and automation extensively saved $1.93 million compared with those using none.
Checklist
AI Governance Checklist
Settle ownership, policy, risk review and monitoring before your next LLM application goes live.
Get the Checklist → Metrics That Show Your LLM Controls Are Working
A control without a number is hard to defend when the budget is set. Pick a few metrics that tie to the practices, measure them on a schedule and watch the trend more than any single reading.
Track false positives next to detection rates, since a guardrail that blocks real questions soon gets switched off.
Metric How to measure it Healthy direction
Injection test pass rate Share of attack prompts in your eval set that the app resists Up PII leak rate Share of eval responses that contain seeded personal data Down Guardrail false-positive rate Share of benign test prompts that guardrails block or refuse Down Detection precision and recall Share of alerts that are real attacks, and share of seeded attacks that raise an alert Up High-risk tool calls with approval Share of payments, deletions and external messages that passed human approval Up Mean time to revoke an agent credential Minutes from decision to revoked access, measured in drills Down Prompts logged with redaction Share of production prompts captured in logs with personal data removed Up Red-team findings closed per cycle Findings closed compared with findings opened in each cycle Up
Set your own targets, because the industry has no shared benchmark for these numbers yet. Record a baseline in the first month, then agree each quarter’s goals with the risk owner and report the trend to leadership.
Frameworks and Regulations to Map Your Controls To
Frameworks tell auditors what you control, while engineers still decide what to build. Map each of the 12 controls to the frameworks your organization answers to, so one control produces evidence for several of them.
Datasheet
Elevate Data Governance , Compliance and Security of Your Organization
Kanerika’s datasheet on data governance, compliance and security, the evidence base behind every framework mapping.
View the Datasheet →
Framework What it is Key dates Use it for
OWASP Top 10 for LLM Applications 2026 Ranked list of LLM application risks Published August 4, 2026 Threat coverage for chat and RAG apps OWASP Top 10 for Agentic Applications 2026 Risk list for AI agents Published December 9, 2025 Agents that call tools NIST AI RMF and AI 600-1 US risk management framework and its generative AI profile AI 600-1 released July 26, 2024; AI RMF 1.0 under revision Governance structure and risk mapping ISO/IEC 42001 International standard for an AI management system Published by ISO and IEC A certifiable AI management system EU AI Act EU regulation for AI systems In force August 1, 2024; applicable August 2, 2026, with exceptions Legal duties for AI used in the EU MITRE ATLAS Knowledge base of adversary tactics and techniques against AI systems Maintained by MITRE Threat modeling and red-team planning Google SAIF Google’s Secure AI Framework, with an Agent Risk Self Assessment Maintained by Google Self-assessing agent and model risk
MITRE ATLAS and Google SAIF suit the engineering side, since they describe how attackers target AI systems and how to rate agent risk.
How the NIST and EU AI Act Timelines Moved
As of November 2026, NIST says AI RMF 1.0 is being revised under the White House AI Action Plan, so plan for a new version. The EU timeline moved as well, and the Commission adopted its AI Omnibus proposal on November 19, 2025.
Political agreement followed on May 7, 2026, and the Omnibus entered into force on July 27, 2026. It shifted high-risk obligations to December 2, 2027 for Annex III use cases and August 2, 2028 for AI built into Annex I products.
Prohibited practices and AI literacy duties have applied since February 2, 2025, and general-purpose AI model obligations since August 2, 2025. A new prohibition on AI that generates non-consensual intimate images or child sexual abuse material, such as nudification apps, applies from December 2026.
AI governance tools can hold the evidence for each mapping. Our overview of AI regulation tracks the wider rules beyond the EU and US.
Common LLM Security Mistakes
Each mistake below has a direct fix, and most trace back to treating the model itself as the security boundary.
Data and Access Mistakes
Trusting the system prompt to keep secrets. It feels private because users never see it, yet a model can be talked into repeating it. Fix: keep secrets out of prompts, then enforce access in code (practice 3).Letting shadow AI tools stay invisible. Staff adopt personal chatbot accounts faster than reviews can approve them, so data leaves without a trace. Fix: discover usage from SaaS, network and expense data, then offer an approved alternative (practice 1).Retrieving through one service account. The demo ran on an admin account, and nobody swapped it before launch. Fix: pass the user’s identity, then filter results when each query runs (practice 2).Indexing a shared drive without its permissions. Ingestion scripts copy text and drop metadata unless someone tells them to keep it. Fix: carry access lists into chunk metadata (practice 4).
Agent and Testing Mistakes
Giving an agent broad write access on day one. Broad rights make a pilot look capable but leave the model as the only check. Fix: start read-only, then climb the permission ladder (practice 6).Launching an agent with no kill switch. Teams plan for the agent working and forget to plan for stopping it mid-task. Fix: build a one-step switch that revokes its credentials, then rehearse it (practice 11). Our guide to rogue AI agents shows what happens when nobody can stop one.Executing generated SQL or code directly. The output looks right in testing, so teams let the model police its own work. Fix: require structured output, allow-lists and sandboxes (practice 7).Testing once at launch. Security review is treated as a launch gate, while prompts and models keep changing after it. Fix: run security evals in CI whenever a prompt, model or data source changes (practice 12).Logging nothing, or logging raw personal data. Logs are often set up for debugging rather than investigation, so they hold too little or too much. Fix: log with redaction, then set retention limits (practice 11).
None of these fixes needs a new platform. Each one is a setting, a code change or a process step that a team can test within a sprint. Most also leave the evidence an auditor will ask for.
Kanerika Service
AI Governance Services
Kanerika helps enterprises set AI policies, map LLM controls to frameworks such as OWASP, NIST AI RMF and the EU AI Act, and keep the evidence auditors ask for.
Explore AI Governance How Kanerika Secures Enterprise LLM Applications
Kanerika builds and secures LLM applications for enterprises in four stages. Assessment takes the inventory and maps every model, data source and tool against the 12 controls to find the gaps that matter most. Design then sets permissions at the data source and limits on every tool.
Implementation adds injection screening, output validation and logging, with a test for each control. Monitoring keeps evals running in CI and gives security teams the alerts and kill switch they need. Delivery uses FLIP, Kanerika’s platform, and its agents DokGPT for documents, Karl for data queries and Susan, a PII redactor.
Our AI governance services set the policies and evidence trail. Our generative AI services and LLM development teams build the applications with these controls designed in from the first release.
In delivery, the control teams skip most often is document-level permission sync in the vector index. Test it with a user who should see nothing, and expect an empty answer. The video from Kanerika’s CISO near the top of this guide covers how prompt injection defenses sit next to ISO 27001 and SOC 2.
Proof From Two Bank Engagements
The global investment bank case study shows control 2 inside an LLM app. Kanerika built chat interfaces over RFI documents (DokGPT) and enterprise databases (Karl) on FLIP, with secure role-based access control. The published results were 43% faster information retrieval, 100% role-based compliance and 35% higher workforce efficiency.
One engagement started a layer below the LLM, for a global bank with about 9,000 branches and 22,000 ATMs. Kanerika used Microsoft Purview Data Map and policies to classify PII, PCI and PHI across SAP, Dynamics 365, Oracle, Netezza and a Lakehouse. That was a data governance engagement, not an LLM build.
The Purview bank case study reports 0% data breaches and 100% adherence to compliance regulations. This classification layer is what lets controls 2 and 5 work, because the labels exist before any LLM reads the data. Our data governance services run that groundwork.
Case Study
Zero Breaches, 100% Compliance for a Bank with Purview
Kanerika classified PII, PCI and PHI across SAP, Dynamics 365, Oracle, Netezza and a Lakehouse with Microsoft Purview, with 0% data breaches and 100% adherence to compliance regulations.
Read the Case Study → Building the 12 LLM Security Controls in Order
The core four come first because together they block both failures in the opening scenario, which were data leaving and an action running unchecked. Name one owner who can approve a tool grant and switch an agent off, then run the inventory, because every later control needs that list.
Record a baseline for injection pass rate and PII leak rate, then make testing continuous so each control stays proven after every change. Start with the AI maturity assessment near the top of this guide, then build these 12 LLM security best practices in order.
Frequently Asked Questions
What are the most important LLM security best practices?
The most important LLM security best practices are an inventory, permissions at the data source, prompt injection defenses and output validation. After those, teams add RAG and data protection, agent limits and supply chain checks. Network isolation, usage caps, logging and continuous testing complete the list of 12. Each control needs a test that proves it works.
What is the biggest security risk with large language models?
Prompt injection is the biggest risk, and it holds the LLM01 spot in the OWASP Top 10 for LLM Applications 2026. Hostile text in a prompt, document or web page can make a model ignore its instructions. The damage grows with what the model can reach. Data access and tool permissions decide how far one successful attack can spread.
How can companies detect LLM attacks in real time?
Log prompts, retrieved sources, responses and tool calls with personal data redacted. Send those logs to the monitoring stack or SIEM the security team already uses. Alert on injection classifier hits, refused tool calls, unusual data volumes and sudden cost spikes. Rehearse a runbook so the team can revoke credentials and switch off an agent quickly.
Which tools are used for LLM security testing, and how often should you red team?
Common open-source options include garak, an LLM vulnerability scanner from NVIDIA, Microsoft’s PyRIT for risk identification and NeMo Guardrails for programmable guardrails. Run automated evals in CI on every prompt, model or retrieval change. Add human red teaming before major launches and after big model upgrades, and track every finding until it closes.
What is model poisoning and how does it happen?
Model poisoning is tampering with training or fine-tuning data, or with a model file, so the model learns a hidden behavior. It can enter through a public dataset, a compromised download or content added to a fine-tuning set. Teams defend with approved sources, checksums, pinned versions, an AI bill of materials and reviews of training data.
Are self-hosted LLMs more secure than cloud-hosted models?
Neither option is safer by default. API-hosted models move hosting security to the provider, while your data leaves your network on every call. Self-hosted or private models keep data inside your boundary, and your team then owns patching, supply chain checks, network isolation and capacity. Data sensitivity and team skills should decide the choice.
Which regulations apply to LLM security in 2026?
The EU AI Act became generally applicable on August 2, 2026, with exceptions. After the AI Omnibus, Annex III high-risk duties start December 2, 2027, and Annex I duties start August 2, 2028. A new ban on nudification and CSAM-generating AI applies from December 2026. Many teams also map controls to the NIST AI RMF and ISO/IEC 42001.
How is LLM security different from traditional cybersecurity?
Traditional security protects networks, hosts and code paths, and firewalls inspect packets and request patterns. LLM attacks arrive as ordinary text that looks like a normal question, so firewalls let them through. LLM security adds controls for prompts, retrieved content, model outputs and tool calls, and it treats every input and output as untrusted until checked.
What is the OWASP Top 10 for LLM Applications 2026?
The OWASP Top 10 for LLM Applications 2026 ranks the main risks in LLM applications. The OWASP GenAI Security Project published it on GitHub on August 4, 2026. It runs from Prompt Injection at LLM01 to Improper Output Handling at LLM10. Hidden Context Exposure replaces System Prompt Leakage at LLM08, and Excessive Agency moves up to LLM03.
Which LLM security controls should an enterprise implement first?
Begin with four controls in the first 30 days. Build an inventory, move retrieval to user-level permissions, screen inputs for prompt injection and validate outputs before anything runs them. Then add the controls your deployment needs. RAG apps need index security, agents need tool limits, and self-hosted models need supply chain checks.
How do you secure AI agents that can call tools and APIs?
Give each agent an allow-list of tools and deny everything else. Issue short-lived credentials scoped to one task, and require human approval for payments, deletions and external messages. Treat every MCP server as a third-party integration with pinned versions and scoped credentials. Log every tool call, cap actions per task and keep a one-step kill switch.
How do you secure a RAG application and its vector database?
Copy each document’s access list into the chunk metadata during ingestion, then check the user’s permission again at query time. Scan new content for hidden text and injected instructions before indexing it. Keep tenants in separate indexes or namespaces. Then test by searching as a low-privilege user for text that appears only in restricted documents.
How do you stop an LLM from exposing confidential data?
Stop sensitive data before it reaches the model. Classify sources, block restricted data from AI use by default and redact personal data from prompts and retrieved text. Enforce permissions at the data source so users only see what they could open anyway. Also bring shadow AI into the inventory, since personal chatbot accounts sit outside your controls.
Can prompt injection be completely prevented?
No single control prevents every prompt injection, because models read instructions and data in the same stream. Teams reduce the risk in layers. They separate system and retrieved content, screen inputs with classifiers, keep secrets out of prompts, limit tools and validate outputs. Jailbreak attempts on chatbots are a form of direct injection and need the same layers.
How do you govern shadow AI and unapproved LLM tools?
Start with discovery. Pull network, SaaS and expense data to find which LLM tools staff already use. Then publish an approved list with a sanctioned alternative for each common task. Route approved tools through single sign-on and a gateway so prompts are logged. Block unapproved uploads of confidential data and review the list every quarter.
