TL;DR
AI model risk management is the set of controls that finds, measures and limits the harm your models can cause, from inventory entry to retirement. Keep one inventory and tier every model by how much its output matters and how far it can act alone. Have someone outside the build team validate each model before launch, using golden test sets and repeated trials when an LLM has no single right answer. Gate deployment by tier, and monitor with thresholds that force a named owner to act. Treat every prompt edit, index refresh, base-model upgrade and new agent tool as a release. SR 26-2 replaced SR 11-7 for US banks on 17 April 2026, but it leaves generative and agentic AI to you.
Key Takeaways AI model risk management covers the risk of the models you build, fine-tune or deploy, from inventory to retirement. SR 26-2 replaced SR 11-7 and SR 21-8, is not enforceable, and is aimed mainly at banks with over $30 billion in assets. Footnote 3 of SR 26-2 puts generative and agentic AI out of scope, so NIST AI RMF and AI 600-1 fill that gap. Tiers should follow materiality, autonomy and reversibility, and each tier should set validation depth, approver and monitoring cadence. LLM validation works without ground truth when you use golden sets, calibrated LLM judges, repeated trials and red-team packs. Prompt edits, index refreshes, base-model upgrades and new agent tools are releases, and retirement needs its own sign-off. Watch on YouTube
What Does It Take to Deploy an Enterprise LLM?
Kanerika’s team walks through what it takes to move an enterprise LLM from pilot to production, the same point where model risk controls have to be in place.
Fifteen Years of US Model Risk Guidance, and the Rewrite Leaves LLMs Out
SR 11-7 set the template for model risk at US banks for fifteen years. On 17 April 2026 the Federal Reserve, OCC and FDIC replaced it with SR 26-2. Its footnote 3 then places generative and agentic AI models outside its scope.
Picture a fine-tuned support model that passed its launch tests in spring. By autumn the provider has upgraded the base model and someone has edited the system prompt. Nobody can say which version answered last week’s complaint.
That gap is what AI model risk management fills. This guide covers the risk of the AI models you build, fine-tune or deploy. Using AI to manage business risk runs in the opposite direction, and our guide to AI in risk management covers that topic.
What Is AI Model Risk Management (and What It Is Not)?
AI model risk management is the discipline of finding, measuring and limiting the harm a model can do through wrong or misused output. It also runs across the model’s whole life, from the day it enters the inventory to the day someone switches it off. The aim is simple to state but hard to keep up. Every model’s risk has to stay visible, owned and proportionate to what it decides.
The SR 26-2 guidance letter describes model risk as the potential for adverse financial consequences from decisions based on model output. It also warns that a sound model can still carry high risk when someone applies it to the wrong problem. That second idea matters for AI, because the same LLM can draft an internal memo on Monday and a customer letter on Tuesday.
However, two neighboring jobs sit outside this scope. Scoring a single proposed use case at intake is the job of an AI risk assessment template . Judging a supplier’s model belongs to an AI vendor risk assessment , and model risk management picks up once a model is yours to run.
What Counts as the Model
The harder question, though, is what counts as the model once you move past classic machine learning. The answer widens with every step toward autonomy, as the table shows.
Model type What “the model” includes Typical failure Predictive ML (credit score, churn, demand forecast) Trained weights, features and decision thresholds Drift as customers, prices or data sources change Fine-tuned or prompted LLM Base model version, fine-tune weights, system prompt, retrieval index and guardrails Confident but false answers, leaks of sensitive text AI agent The LLM plus its tools, permissions, memory and stop rules A plausible chain of steps that ends in the wrong action
Each row needs its own evidence, so one validation template rarely covers all three. For example, a version number for an agent has to name the prompt, the index and the tool set as well as the base model. Otherwise you cannot reproduce what the agent did on a given day.
The 2026 Baseline: SR 26-2 Replaced SR 11-7, and GenAI Is Left to You
US banking agencies rewrote their model risk guidance on 17 April 2026, and the change resets what following the regulator means. The Federal Reserve, OCC and FDIC issued SR 26-2 jointly. It also supersedes SR 11-7 and SR 21-8, the 2021 statement on BSA/AML models. On the same day the OCC’s Bulletin 2026-13 rescinded its matching 2011 bulletin and the old Comptroller’s Handbook booklet.
The guidance says it is most relevant to banking organizations with over $30 billion in total assets. Smaller banks with complex or widespread models may still find it relevant. Outside banking it binds nobody, although it remains the clearest public pattern for running model risk. For what bank supervisors expect from AI beyond models, see our guide to AI governance in banking .
What SR 26-2 Keeps From SR 11-7
First, the core vocabulary survives. Materiality still comes from a model’s purpose combined with its exposure. Because of that, immaterial models may need only identification, plus monitoring for the conditions that would make them material. Effective challenge still means critical review by objective experts who have the independence and standing to force a change.
Validation also rests on the same three components. Conceptual soundness asks first whether the design, assumptions and data choices hold up. Outcomes analysis compares outputs with what happened in the real world, while ongoing monitoring checks that performance holds as products, clients and data change.
The inventory and documentation duties also stay. Vendor models remain in scope even when the code and data are proprietary, so a bought model still needs validation and monitoring.
What It Changed
The tone shifted toward risk-based tailoring. The SR 26-2 attachment says non-compliance “will not result in supervisory criticism.” So it is not an enforceable standard, though unsafe practices can still draw supervisory action.
The attachment also says validation quality depends on rigor and effectiveness rather than on how the risk function is organized. That point especially helps smaller teams. A demanding review by a peer from another team can count for more than a separate validation unit that only signs forms.
What It Leaves Out
Footnote 3 is the line every AI team should read. It describes generative and agentic AI models as novel and still changing fast, so it places them outside the guidance. However, traditional statistical models and non-generative AI, such as a gradient-boosted fraud score, stay in scope.
So the GenAI layer of your program is your own design. Four public sources fill most of that gap. None of them is a drop-in replacement, but together they cover every stage.
Source Who it binds Lifecycle stages it covers GenAI coverage SR 26-2 (Fed, OCC, FDIC) US banks, mainly over $30B in assets; guidance, not enforceable Inventory, development, validation, monitoring, governance, vendor models Excluded (footnote 3) NIST AI RMF 1.0 and AI 600-1 Voluntary, any sector Govern, map, measure, manage, including inventory and decommissioning Yes, AI 600-1 lists 12 GenAI risks UK PRA SS1/23 UK banks and building societies with internal-model approval for capital Model identification, governance, development, validation, risk mitigants Covers AI used in modelling techniques EU AI Act, as amended by Regulation 2026/1744 Providers and deployers of high-risk AI in the EU Lifecycle risk management (Art. 9), post-market monitoring (Art. 72) High-risk duties from 2 Dec 2027 (Annex III) and 2 Aug 2028 (Annex I) ISO/IEC 42001 and 23894 Voluntary; 42001 is certifiable AI management system and risk process Applies to any AI system
What Fills the Gap
The NIST AI Risk Management Framework , released in January 2023, asks for an inventory of AI systems under GOVERN 1.6. It also asks for safe decommissioning under GOVERN 1.7. Its Generative AI Profile, NIST AI 600-1 , then adds 12 risks that GenAI creates or makes worse, including confabulation, information security and data privacy.
Banks in the UK work from the PRA’s SS1/23 model risk principles , whose current version took effect on 23 April 2026. In Europe, Regulation (EU) 2026/1744 pushed the high-risk obligations to 2 December 2027 for Annex III systems. Annex I systems then follow on 2 August 2028.
Systems that generate synthetic content and were placed on the market before 2 August 2026 get one more deadline. They must meet the Article 50(2) marking duty by 2 December 2026. Our EU AI Act compliance guide covers those duties in detail. The post on AI governance, risk and compliance then shows how they sit beside model risk.
Kanerika Service
AI Governance Consulting Services
Kanerika’s AI governance services cover bias testing, independent model validation and LLMOps controls, scaled to the risk each AI system carries.
Explore AI Governance Services What Changes When the Model Is an LLM or an Agent
The three validation questions in SR 26-2 still work for GenAI. Is the design sound, do the outputs match reality, and is the model still performing? What changes, however, is the evidence you can gather to answer them.
Dimension Classic ML Fine-tuned or RAG LLM AI agent Ground truth Labels arrive, sometimes months later Often no single right answer The right outcome depends on the whole task Determinism Same input, same output Same prompt can give different answers Different paths to the same goal, or a different goal What a version is Weights and features Base model, fine-tune, prompt, index and guardrails All of that, plus tools, permissions and memory Main failure modes Drift, bias Confabulation, prompt injection, data leakage Tool misuse, loops, a wrong action that cannot be undone Validation evidence Back-tests, benchmarks Golden sets, rubric scores, red-team results Trajectory evaluations in a sandbox Monitoring signal Input and output drift, accuracy Sampled rubric scores, groundedness, user corrections Tool-error rate, escalations, aborted runs
Confabulation is NIST’s term for confidently stated false content. It is also the failure that surprises business owners most, and our guide to LLM hallucination explains why it happens. Prompt injection matters for a different reason, because an attacker can steer the model through the content it reads.
Agents add a compounding problem. Suppose each of ten steps in a task has a 95% chance of going right. Then the whole task succeeds only about 60% of the time, even though every single step looked safe in testing.
So judge an agent on its trajectory, meaning the full sequence of reasoning, tool calls and stopping decisions, because a single answer tells you little. The AI agent guardrails that limit what the agent may do then become part of the model under review.
AI Assessment
See Where Your AI Program Stands
Score your organization’s readiness across AI and ML foundations, generative AI and AI agents to see which models need tighter risk controls first.
Start Your AI Assessment → Step 1: Build the Model Inventory and Tier Every Model
Every later control depends on a list nobody likes to keep. SR 26-2 expects an inventory detailed enough to understand model risk for each model and in aggregate. NIST GOVERN 1.6 also asks the same of AI systems. The hard part, however, is finding models that never went through a launch, such as a notebook that slipped into a nightly job.
The Minimum Inventory Record
A spreadsheet works for the first fifty models. What matters is that each entry answers the questions an auditor or an incident responder will ask first.
Field What to record Why it matters ID and owner Unique ID and the accountable first-line owner Someone answers for the model Intended use Approved uses and named out-of-scope uses Catches misapplication, the risk SR 26-2 warns about Model and versions Base model and version, fine-tune, prompt, index and tool versions Shows what answered on a given day Data lineage Training or fine-tune data sources and retrieval sources Explains output problems and licence exposure Dependencies Upstream data, downstream consumers and calling agents Shows what breaks on change or retirement Tier Current tier and the reason for it Drives every control that follows Validation Status, date and open findings Shows whether the model is cleared to run Monitoring and review Monitoring plan, next review date, retirement status Keeps the entry current
Lineage is the field teams skip most often, because it takes digging. A data lineage view from source to model output makes it cheap to fill, and it pays off the first time an output goes wrong.
Tier by Materiality, Autonomy and Reversibility
Start from the materiality idea in SR 26-2, which combines purpose with exposure. Then add questions that matter more for AI. Does the model suggest or does it act? How sensitive is the data it reads, how easily can a wrong output be undone, and how many people does it reach?
Tiers belong to the use, not the technology. For example, a summarizer that condenses internal meeting notes is Tier 3. Point the same summarizer at complaint files that feed customer letters, and it becomes Tier 1 even though nobody changed a line of code.
What Each Tier Triggers
The tier only matters if it changes what happens next. The table below is illustrative, since each organization sets its own thresholds and intervals.
Control Tier 1 (high) Tier 2 (medium) Tier 3 (low) Validation depth Full independent validation, red-team pack, trajectory tests for agents Targeted validation by a reviewer outside the build team Documented self-test plus peer sign-off Approver Model risk function and an accountable executive Model risk function Product owner Monitoring cadence Continuous signals, weekly review Weekly signals, monthly review Monthly signals, quarterly review Revalidation Yearly and on any material change Every 18 to 24 months and on material change On material change Retirement review Formal sign-off by the model risk function Model risk sign-off Owner sign-off, logged
Expect tiers to move. A model that starts in Tier 3 often climbs as more teams plug into it, so review the tier whenever a new consumer appears. For the intake scoring that sets the first tier, the risk assessment template has a ready form.
Step 2: Set Development Standards Before the First Training Run
Validation goes faster when the developer wrote down the rules before building. SR 26-2 notes that development is not purely technical, because developer judgment shapes the data, the method and the tests. Writing those choices down early turns them into evidence instead of memory.
A useful model card records purpose, intended users, known limits and failure modes found in testing. It also lists every data source with its licence. Freeze the evaluation set and the acceptance criteria before the first run, so nobody tunes the bar to fit the result.
Fine-tuned and open-weight models need extra records. Keep the lineage of the fine-tune dataset, and evaluate on your own data instead of trusting a public benchmark. Record the exact base-model version too, because a provider upgrade later counts as a change.
The choice between RAG and fine-tuning also belongs in the card, along with the reason for it. Documentation also carries the program through staff changes. SR 26-2 ties it to continuity and to tracking recommendations, responses and exceptions, which is exactly what a new owner needs on day one.
Datasheet
Elevate Data Governance , Compliance & Security
How Kanerika sets up data discovery, classification, access policies and compliance controls, the data foundation behind every model card and lineage record.
Download the Datasheet → Step 3: Validate Independently, Even Without Ground Truth
Validation is where LLM programs most often fall back on a demo and a thumbs-up. The questions from SR 26-2 still apply, though, and each one has a workable answer for generative systems.
Conceptual Soundness for an LLM System
Ask the developer to defend each design choice. Why this base model? Why fine-tune instead of retrieval or a prompt? How is the index chunked and filtered, and which guardrails sit around the model?
SR 26-2 notes that interpretability measures or benchmarking against other models may be more practical than evaluating theory for some models. That suits LLMs well. Interpretability also helps when a reviewer has to explain a decision to a customer. Our piece on explainable AI covers the methods.
Outcomes Analysis When There Is No Single Right Answer
First, build a golden set from real cases, with expert-written reference answers and a scoring rubric for each. Then score outputs against tolerance bands rather than one accuracy number. Slice the results by user segment, language and task type, because averages hide the groups that fail.
An LLM judge can score thousands of outputs cheaply, but only after you measure how often it agrees with human raters. If agreement is weak, then the judge is measuring something else. So keep a human-scored sample in every validation cycle, and the LLM evaluation framework guide goes deeper on rubric design.
Testing Non-Determinism
Run every test case several times, then report the spread, including the worst output. A model that gives the same wrong answer ten times is repeatable and still wrong. So treat consistency and accuracy as two separate results.
Then add a red-team pack mapped to the NIST 600-1 risks that apply, such as confabulation, information security, harmful bias and data privacy. Our guide to AI red teaming shows how to build those attacks and score them.
On-Demand Webinar
The Real Cost of LLM Security Risks and How to Reduce Them
An on-demand Kanerika webinar on the LLM security risks that cost enterprises the most, from prompt injection to data leakage, and the controls that reduce them.
Watch the Webinar → Validating Agents on Whole Trajectories
For agents, the unit of evaluation is the task. First, run each scenario end to end in a sandbox. Then check whether each tool call was appropriate, whether the agent stopped when it should have, and whether small errors added up along the way. The AI agent evaluation guide lists the metrics teams use for this.
The Validation Decision
End each validation with one of three outcomes. The validator approves, approves with conditions, or rejects, and every finding goes into the inventory entry.
Conditions are how urgent launches stay honest. SR 26-2 allows use before validation is complete when there is a real business need. In that case the business should understand the limits, tell stakeholders, and apply controls such as use limits or closer monitoring.
Independence is the other test. SR 26-2 asks for effective challenge from people with the expertise, objectivity and standing to change the outcome. So in practice the validator should not report to the build team.
Step 4: Gate Deployment by Tier
A deployment gate turns the validation result into a yes or a no for production. The tier decides who signs and what must already exist before anyone does.
For Tier 1, require four things before release. The validator has signed the report, the monitoring plan runs on live data, rollback has passed a test, and an owner is on call. Tier 2 can drop the on-call duty, while Tier 3 needs a logged owner sign-off.
When a business deadline wins, use the SR 26-2 exception properly. Record the conditions in the inventory, narrow the use, tighten monitoring, and set a date by which validation must finish.
Some models need a gate inside the workflow too. In one insurance project, Kanerika built AI/ML-powered fraud detection with RPA, so claims pass fraud checks before payment instructions reach the bank. The case study reports 36% cost savings and a 25% improvement in operational efficiency. The design idea carries over to model risk, because a check before an irreversible action limits what one wrong output can do.
Case Study
AI/ML-Powered Fraud Detection in Insurance
Kanerika built AI/ML-powered RPA fraud detection for an insurer, with claims checked before payment instructions reach the bank, delivering 36% cost savings and a 25% efficiency gain.
Read the Case Study → Step 5: Monitor With Triggers That Force an Action
A dashboard nobody acts on is decoration. Monitoring earns its place when every signal has a threshold, an owner and a named action. SR 26-2 calls this ongoing monitoring, while NIST asks for production monitoring under MEASURE 2.4 and post-deployment plans under MANAGE 4.1.
Europe also adds a legal duty for high-risk systems. Article 72 of the EU AI Act requires providers to run post-market monitoring proportionate to the risks.
Signals for Predictive Models
First, watch input drift, output drift, performance against outcomes once labels arrive, and breaks in upstream data quality. Article 15(4) of the AI Act adds a warning for systems that keep learning after deployment, since their own outputs can feed back into training. Our machine learning governance guide covers drift detection for classic models in more depth.
Signals for LLMs and Agents
For LLMs, score a sample of live traffic with the same rubric you used in validation. Then track refusal rate, groundedness or citation failures, user corrections, safety-filter hits, cost and latency. Agents also add tool-error rate, escalations to humans, and runs that loop or abort.
Tooling helps, but only after the thresholds and owners exist. Our reviews of AI observability tools and LLMOps observability compare what each platform captures. The table pairs each signal with an owner and an action; the triggers are examples, so set your own.
Signal Example trigger (illustrative) Owner Action Input drift on a Tier 1 feature Drift score above the agreed limit ML engineering Investigate, then revalidate if outcomes move Rubric score on sampled traffic Two weekly samples below the validation baseline Product owner Restrict use and open a finding Groundedness failures Above the tolerance band for a week AI engineering Roll back the last prompt or index change Agent tool-error rate Double the 30-day baseline Agent owner Pause the affected tool and review trajectories User corrections Spike after a provider model update Model risk function Targeted revalidation
When Labels Arrive Late or Never
Credit and fraud outcomes can take months to show up, and many LLM tasks never get a label at all. In those cases, use sampled expert review and proxy signals such as complaint and override rates. Then say plainly in the monitoring report what those proxies can and cannot prove.
The pressure is highest for models whose inputs change every day. Kanerika built a delivery-prediction system for a logistics company that moves high-value items such as artwork and luxury cars. The system draws on real-time traffic and weather data. The case study reports 87.89% prediction accuracy and fewer failed deliveries. A model fed by live conditions like these needs its accuracy checked on a schedule, because the world it predicts keeps moving.
Case Study
Delivery Prediction for Niche Logistics Using AI
Kanerika built an AI delivery-prediction system on real-time traffic and weather data for a logistics company that moves high-value items such as artwork and luxury cars, reaching 87.89% prediction accuracy and fewer failed deliveries.
Read the Case Study → Step 6: Treat Every Prompt, Index and Model Swap as a Release
Classic models change when someone retrains them. LLM systems, however, change through a prompt edit, an index refresh, a provider’s base-model update or a new tool, often without any formal release. Change control decides in advance which of those changes needs what level of revalidation.
The matrix below is illustrative. Define materiality before the first change arrives, so the debate happens once instead of in the middle of an incident.
Change Tier 1 Tier 2 Tier 3 System prompt edit Targeted revalidation Regression suite Regression suite Index refresh, same sources Regression suite Regression suite Log only New data source in the index Targeted revalidation Targeted revalidation Regression suite Base-model version bump or deprecation Full revalidation and re-approval Targeted revalidation Regression suite Fine-tune on new data Full revalidation and re-approval Full revalidation Targeted revalidation New tool or permission for an agent Full revalidation and re-approval Targeted revalidation Targeted revalidation Threshold or guardrail change Targeted revalidation Regression suite Log only
The EU AI Act works from the same idea. Article 3(23) defines a substantial modification as an unplanned change after market placement that affects compliance or the intended purpose. Your own materiality rules then play the same role inside the company.
Each release should also leave evidence behind. Keep the artifact versions, dataset references, evaluation results, approver, deployment record and rollback criterion together. Our review of AI governance tools compares the platforms that capture this evidence automatically.
Provider updates deserve their own watch. Subscribe to deprecation notices, pin model versions where the API allows it, and treat a forced upgrade as a change from the matrix above. Teams that also track where weights and packages come from will find the overlap with AI supply chain security useful.
Step 7: Retire Models on Purpose
Models rarely die cleanly. Instead they linger as endpoints nobody calls, or worse, as endpoints an agent still calls. SR 26-2 says nothing about retirement, but NIST GOVERN 1.7 asks for processes to decommission and phase out AI systems safely.
Retire a model when a successor replaces it or when performance stays below threshold. The same also goes for a use case that has ended, a base model the provider no longer supports, and findings left open past their deadline.
Then work through a short checklist. Find every downstream consumer and agent that calls the model, then switch off its endpoints and scheduled jobs. After that, revoke credentials and API keys, archive the evidence, and close the inventory entry with an approval.
Keep the records for as long as your own legal and regulatory obligations require, because no universal retention period applies. Agents need the same care, since an agent may still call a model everyone thinks is gone. Our guide to AI agent lifecycle management covers their end of life.
Who Owns What: Three Lines of Defense for AI Models
Clear ownership keeps the seven steps from collapsing into one team marking its own homework. Many organizations use the IIA’s Three Lines Model , which updated the older three lines of defense idea. The table maps it onto the model lifecycle, although the labels matter less than the separation of duties.
Activity First line (product owner, ML and AI engineering) Second line (model or AI risk, validators) Third line (internal audit) Executive or risk committee Inventory Registers models and keeps entries current Sets the standard, checks completeness Tests coverage Sees aggregate model risk Development Builds and documents Sets development standards Reviews the standards None Validation Supplies evidence, fixes findings Validates independently Tests whether validation is rigorous Sees Tier 1 results Deployment approval Assembles the release evidence Approves Tier 2 None Approves Tier 1 Monitoring and change Runs monitoring, raises changes Owns triggers and revalidation decisions Samples the change log Receives breach reports Retirement Decommissions Signs off Checks the closed entries Informed for Tier 1
SR 26-2 is specific about audit. Internal audit generally does not redo development or validation. Instead it checks whether the model risk practices are rigorous and effective, and the AI auditing framework guide sets out what that review covers.
The guidance also flags conflicts of interest, such as a validator whose reporting line runs through the development team. Small teams can still get independence. Borrow a peer validator from another product team, rotate reviewers, and buy external validation for Tier 1 models.
Checklist
AI Governance Checklist
A practical checklist for setting up AI governance, with the ownership, review and oversight steps a model risk program reports into.
Get the Checklist → A First AI Model Risk Management Program in 90 Days
A full program takes longer than a quarter, but a working one does not. The plan below assumes a small model risk function and an existing ML platform.
First month (days 1 to 30). Sweep for models, including shadow notebooks and embedded vendor features. Give each one an owner and a provisional tier.Second month (days 31 to 60). Write the validation standard and run it on two or three Tier 1 models. Wire monitoring with triggers for those models first.Final month (days 61 to 90). Put the change-control matrix into the release pipeline and retire one model end to end. Then send the first model risk report to the risk committee.Call the quarter done when every Tier 1 model has a named owner, a signed validation report, monitoring with triggers and a tested rollback. Anything short of that is a backlog, so report it as one.
If the wider governance structure is still missing, start with an AI governance framework so model risk has a committee to report to. The machine learning model management guide also covers the registry side of the same work.
Common Failure Patterns in AI Model Risk Programs
The same mistakes show up across sectors and team sizes. Each one below is cheap to prevent and expensive to find later.
Validation happens after launch, so findings arrive when users already depend on the model. The inventory lists models but not their prompts, indexes or tools. An LLM judge scores everything, yet nobody ever checked it against human raters. Thresholds exist on a dashboard, but no one owns the response. A provider updates the base model silently, and no revalidation follows. Agents pass per-answer tests and fail on whole tasks. Nothing is ever retired, so old endpoints keep serving stale answers. Most of these trace back to one missing link between the inventory and the release process. Fix that link first, because the other controls depend on it.
How Kanerika Approaches AI Model Risk Management
Kanerika’s AI governance services follow the IMPACT framework, which stands for Identify, Map, Prove, Analyze, Create, Transform. As Kanerika describes it, Identify surfaces governance gaps before they become audit findings. Prove tests governance controls on highest-risk systems first, and Transform embeds governance into ongoing AI operations.
The governance service covers data lineage tracking from source to model output and model impact assessments before deployment. It also includes a model registry with full audit trail across production, plus bias testing before deployment and monitoring after launch. The same page also defines model validation as an independent assessment by a party separate from the development team. Step 3 relies on exactly that distinction.
On the operations side, Kanerika’s MLOps consulting has three engagement models. An MLOps Readiness Sprint reviews current ML workflows and finds production readiness gaps. Production Setup sets up deployment pipelines and prepares models for launch, while Managed MLOps Support monitors models after launch.
Kanerika Service
MLOps Consulting Services
Kanerika’s MLOps consulting covers drift and performance monitoring, versioned datasets and models, audit-ready model records and retiring outdated models safely.
Talk to Our Experts The capabilities listed on that page include versioning datasets and models, maintaining audit-ready model records and retiring outdated models safely. Those also map directly onto Steps 1, 6 and 7 of this guide.
The two case studies earlier show the kinds of models these controls protect, a fraud check that runs before money moves and a delivery forecast fed by live traffic and weather. Neither page describes a model risk program, so read them as examples of where gates and monitoring matter. Kanerika holds ISO 27001, ISO 27701:2019 and SOC 2 Type II, and our overview of AI governance is a good next read.
Running Model Risk for Models That Keep Changing
AI model risk management comes down to a few habits kept without exception. Keep one inventory, let tiers decide the controls, and validate independently with methods that work without a single right answer. Give every monitoring signal an owner, treat prompt, index and model changes as releases, and retire models on purpose. SR 26-2 supplies the pattern for banks, but for generative and agentic AI the program you design is the standard you answer for.
Frequently Asked Questions
What is AI model risk management?
AI model risk management is the set of controls that finds, measures and limits the harm a model can cause through wrong or misused output. It covers the whole lifecycle, from inventory and tiering through independent validation, deployment gates, monitoring and change control to retirement. It applies to predictive models, LLMs and AI agents.
Does SR 26-2 or SR 11-7 apply to generative AI?
SR 26-2 replaced SR 11-7 on 17 April 2026, and its footnote 3 places generative and agentic AI models outside its scope. Non-generative AI and traditional statistical models stay in. Banks still need controls for GenAI, so most build them from their own risk practices, NIST AI RMF and the NIST AI 600-1 profile.
How is AI model risk management different from AI in risk management?
AI model risk management deals with the risk created by your own models, such as wrong answers, drift or misuse. AI in risk management runs the other way, using AI to detect fraud, score credit or watch for compliance breaches. A fraud model used for risk management still needs model risk management of its own.
How do you validate an LLM when there is no single correct answer?
Build a golden set of real cases with expert reference answers and a scoring rubric. Score outputs against tolerance bands, run each case several times to see the spread, and use an LLM judge only after checking its agreement with human raters. Add a red-team pack for confabulation, data leakage and prompt injection.
Can the team that built a model also validate it?
The developers should test their own model, but validation needs someone separate. SR 26-2 asks for effective challenge from people with the expertise, objectivity and standing to change the outcome. Small teams can borrow a peer validator from another product team, rotate reviewers, or buy external validation for their highest-tier models.
How often should an AI model be revalidated?
Set the interval by tier and add event triggers. A common pattern is yearly revalidation for high-tier models and longer intervals for lower tiers, with immediate targeted or full revalidation after a material change. Base-model upgrades, new data sources, fine-tunes and new agent tools usually count as material changes for high-tier models.
Does changing a system prompt require revalidation?
It depends on the model’s tier and on rules you set in advance. For a high-tier model, a system prompt edit usually calls for targeted revalidation against the golden set. For lower tiers a regression suite is often enough. Writing the change-control matrix before the first edit stops each change becoming a debate.
What happens to a model's records when it is retired?
Archive the validation reports, monitoring history, change log and approvals, then close the inventory entry with a sign-off. Keep the archive for as long as your legal and regulatory obligations require, since no universal retention period applies. Before switching off, find every downstream system and agent that still calls the model.
What should a model inventory record for an LLM application?
Record the owner, approved and excluded uses, the base model and its version, and the versions of the prompt, retrieval index and tools. Add data lineage, downstream consumers, the tier and its reason, validation status and open findings. Finish with the monitoring plan, next review date and retirement status for each entry.
