Azure AI Foundry (Microsoft Foundry): Models, Agents, Cost

Key Takeaways

• Azure AI Foundry is now Microsoft Foundry, a single Azure resource with projects and one set of RBAC, networking and policy controls.
• The catalog lists more than 10,000 models, such as GPT-6.1 Sol, DeepSeek-V4-Pro, Grok 4.7, Mistral Large 3 (preview) and Microsoft’s own Phi and MAI models.
• Global Standard is the default deployment type, while Data Zone, Provisioned and Batch exist for residency, reserved throughput and async jobs at a 50% discount.
• Prompt agents need no runtime code and hosted agents run your own framework; a prompt agent can also run on the Copilot Harness (preview).
• The Foundry platform carries no fee of its own, so cost comes from model tokens, search, container compute, evaluation runs and monitoring.
• Self-host with vLLM or Ollama only when your team must own the GPUs or the data boundary, and use serverless or managed compute otherwise.

Three Governed Agents by Next Quarter

Picture a team that already runs a working chatbot on Azure OpenAI. It answers HR policy questions well, so leadership likes it. Then the brief changes to three governed agents by next quarter, one each for HR, finance and IT support.

Each new agent needs its own tools, its own slice of company data, as well as an audit trail. But none of that exists in the chatbot today. So the team has to decide what Microsoft Foundry adds on top of a model endpoint, and what it will cost to run.

The rest of this guide follows that team’s decisions in order. First comes the product’s current name and the model choice. Then come where it runs, how agents are built and what the bill will look like.

What Is Azure AI Foundry (Now Microsoft Foundry)?

Azure AI Foundry is Microsoft’s platform for building, deploying and governing AI apps and agents on Azure, though Microsoft now calls it Microsoft Foundry. Microsoft Learn says it “unifies agents, models, and tools under a single management grouping.”

RBAC, networking and policies are shared across that grouping under one Azure resource provider. Azure OpenAI is just one model provider inside Foundry, next to Anthropic, Meta, DeepSeek, Mistral and Microsoft itself. So a team calling GPT-6.1 Sol and a team calling DeepSeek-V4-Pro use the same platform and evaluation tools.

In a generative AI tech stack, Foundry covers the model and orchestration layers. Teams that want help planning that layer can also look at Microsoft Foundry consulting from Kanerika.

Old Names and APIs Mapped to New

Microsoft has renamed the product and its APIs more than once, so older tutorials use terms that no longer match the portal. Microsoft gives no rename date, so distrust any article that quotes one.

What Foundry Adds Beyond a Model Endpoint

A bare model endpoint takes a prompt and then returns text. Foundry, by contrast, wraps that call with the parts an enterprise agent needs:

1. Agents in prompt, hosted and voice forms, as well as versioning and publishing.
2. Tools such as web search, code interpreter, OpenAPI and MCP connections.
3. Knowledge through Foundry IQ and Azure AI Search, so answers draw on company data.
4. Evaluation with quality and safety evaluators, together with red teaming and tracing.
5. Governance through an Entra identity per agent, RBAC, private networking and a control plane.

Microsoft’s own advice still keeps this in proportion. When an app only sends prompts and needs no tools, start with a single model call. Then add the agent layer once a real need appears.

Microsoft Foundry Architecture: Foundry Resource, Projects and Connected Services

Foundry has a two-level structure, and getting it right early saves rework later. The top level is the Foundry resource, while the level below it is the project.

The resource is the governance boundary. Networking, identity and policy are set there, and every project under it then inherits them. Projects, in contrast, are the isolation unit, usually one per team or use case. So an HR agent and a finance agent can share a network setup without sharing files or permissions.

Connected services then attach the rest of Azure. For example, a project links to storage, Azure AI Search and Key Vault for secrets. Teams can also bring their own storage account when data must stay in a specific place, as the Foundry architecture page describes.

Two migration details also matter for existing teams. Azure OpenAI resources can be upgraded in place into Foundry while keeping the same endpoint and API keys. Hub-based projects still work, but they stay in the Foundry (classic) portal.

A practical rule is to decide the resource layout, network model and access model before anyone builds an agent. Changing a network boundary after five projects depend on it is slow, because it usually means redeploying. Teams on a wider estate often plan this alongside their Azure cloud solutions work, starting with four questions:

1. How many Foundry resources do you need, for example one per region or business unit?
2. Which teams or use cases, such as HR or finance, get their own project?
3. Will traffic use public endpoints rather than private networking?
4. Which Entra groups get owner, contributor and reader roles on each project?

Written answers become the baseline every later project inherits.

Foundry Models: What Is in the Catalog Today and How to Choose

The Foundry catalog holds more than 10,000 models from Microsoft, OpenAI, Anthropic, Meta and others. Most teams start with Foundry Models sold by Azure, since Microsoft bills and supports that subset directly.

A list that long is easier to read when grouped by model family. The table shows example model IDs from the Foundry Models sold by Azure page. The Anthropic row comes from the models from partners and community page.

Checked against Microsoft Learn and the Microsoft Foundry Blog on November 3, 2026.

GPT-6.1 Sol gives a sense of scale, since in Foundry it has a context window of 1,050,000 tokens, of which 922,000 can be input. It also allows up to 128,000 output tokens and makes tool calls through the Responses API only.

Claude models in Foundry come in two hosting versions, Hosted on Azure and Hosted on Anthropic infrastructure. So check which one your residency rules need before deploying.

How to Choose a Model

Choosing well is mostly a testing habit, so test before you commit:

1. Shortlist two or three models that fit your context length, modality and license needs.
2. Build a task set from real prompts and the answers you expect.
3. Then score each candidate with Foundry evaluators on quality, cost and latency.
4. Use model router when task difficulty varies, so simple requests go to a cheaper model. Microsoft also added preview controls for it in September 2026.
5. Use instant access (preview) to call supported models by name with no deployment while you test.

Teams that also run open-source LLMs outside Azure often put an LLM gateway in front of every provider, so that routing and logging stay consistent.

Deployment Types: Global Standard, Data Zone, Provisioned and Batch

A deployment type decides where inference runs and how you pay for it. It affects residency as well as cost.

Microsoft’s guidance on deployment types is direct. Start on Global Standard, because it gets new models first, has the lowest price and covers the most regions. Move only when you have a residency rule, a need for reserved throughput or a batch workload.

One point still confuses many security reviews. Data at rest stays in the geography you designate, but a Global deployment may process inference in any region. So if a regulator cares where prompts are processed, Data Zone or a regional type is the safer start.

A finance team scoring invoices overnight is a good Batch candidate, since nobody waits for the answer. A support agent with steady daytime traffic may justify Provisioned capacity, but only once its load has been measured.

Self-Hosting Open Models (vLLM, Ollama) vs Managed Foundry Deployments

Open-weight models raise a second question after the model choice, which is who runs the GPUs. Foundry gives two managed answers, while vLLM and Ollama give two self-managed ones.

1. Serverless API: Foundry Models sold by Azure run on capacity Microsoft manages, and you pay per token. The deployments overview also names this as Microsoft’s preferred option.
2. Foundry managed compute (preview): Open-source, partner or custom weights run on dedicated GPUs that Foundry manages. Microsoft-curated vLLM, SGLang and NVIDIA NIM containers run on A100 80 GB, H100 80 GB or MI300X 192 GB.
3. Self-hosted vLLM: Your team runs vLLM on its own GPUs. It is Apache-2.0 licensed and uses PagedAttention as well as continuous batching for concurrent serving. The stable release is v0.32.0 from October 18, 2026.
4. Ollama or Foundry Local: Ollama is MIT licensed and suits laptops and small team servers, though it handles one request per model at a time by default. Foundry Local instead runs on Windows, macOS on Apple silicon or Linux with no Azure subscription. Microsoft also says it “isn’t designed as a server inference stack.”

Four Questions That Settle the Hosting Choice

Four questions settle most cases, so answer them before picking a path:

1. Who should run the GPUs?
2. Must prompts and weights stay inside your own boundary?
3. Does the license of the weights allow your use?
4. Do you need Foundry’s evaluation and governance around the model?

Take a bank that must keep a fine-tuned model in its own data center. It answers yes to the boundary question, so it leans toward self-hosted vLLM.

A product team with no appetite for GPU operations answers the first question the other way. It leans toward managed compute or serverless instead. Self-hosting also means running your own tracing stack, which the guide to LLMOps observability tools compares.

The vLLM vs Ollama comparison covers the self-hosted side. The guide to private LLMs covers the data boundary side.

Foundry Agent Service: Prompt Agents, Hosted Agents and Microsoft-Run Agent Loops

Foundry Agent Service is where models become agents that call tools and take actions. There are two main agent types, prompt agents and hosted agents, plus voice-based prompt agents. A prompt agent can also run on the Microsoft Copilot Harness (preview), where Microsoft runs the agent loop.

A prompt agent is configuration only. You define instructions, a model and tools through the portal, azd, the SDK or REST, and Foundry runs it. Voice-based prompt agents work the same way and run through Voice Live. A hosted agent, by contrast, is your own code in a container that Foundry manages.

Hosted agents can use frameworks such as Microsoft Agent Framework, LangGraph, OpenAI Agents SDK, Anthropic Agent SDK, Copilot SDK or custom code. So Microsoft’s agents overview sums up the advice as “start declarative and move to code.”

In code, a prompt agent takes only a few lines of Python. The snippet below is copied from Microsoft’s Foundry SDK quickstart, which installs azure-ai-projects 2.3.0 or later (checked November 3, 2026).

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition

# Format: "https://resource_name.services.ai.azure.com/api/projects/project_name"
FOUNDRY_PROJECT_ENDPOINT = "your_project_endpoint"
FOUNDRY_AGENT_NAME = "your-agent-name"

# Create project client to call Foundry API
project = AIProjectClient(
    endpoint=FOUNDRY_PROJECT_ENDPOINT,
    credential=DefaultAzureCredential(),
)

# Create an agent with a model and instructions
agent = project.agents.create_version(
    agent_name=FOUNDRY_AGENT_NAME,
    definition=PromptAgentDefinition(
        model="gpt-5-mini",  # supports all Foundry direct models
        instructions="You are a helpful assistant that answers general questions",
    ),
)
print(f"Agent created (id: {agent.id}, name: {agent.name}, version: {agent.version})")

The sample signs in through DefaultAzureCredential, so run az login first. Its model value is the quickstart’s example, and you replace it with a model deployed in your own project.

Microsoft-Run Agent Loops, Versions and Publishing

The Microsoft Copilot Harness in Foundry entered public preview on October 28, 2026. With this option the resource stays a prompt agent, while Microsoft runs the agent loop as well as the sandbox. Runtime is free before December 1, 2026, and after that it bills $0.0994 per vCPU-hour and $0.0118 per GiB-hour.

Model inference and tools, however, are billed separately on top of that. Every agent also gets automatic version snapshots with rollback. Once published, an agent gets a stable endpoint and can be shared through Microsoft Copilot, Teams and the Entra Agent Registry.

Hosted agents are generally where LLM-powered autonomous agents and their planning loops live. Teams that want graph-style control flow in that code often weigh LangGraph first, especially for branching workflows.

Agent Tools in Foundry: Toolboxes, MCP Servers and Agent-to-Agent (A2A)

Tools let an agent act on systems as well as write text, so tool access is also where most of the risk sits. Foundry groups tools into five families:

1. Built-in tools for web search, file search, code interpreter and memory.
2. Custom functions that call your own code.
3. OpenAPI tools that wrap an existing REST API.
4. MCP servers, which use the Model Context Protocol.
5. Agent-to-agent calls through A2A, with v1.0 now GA and v0.3 still in preview.

A toolbox puts a curated set of tools behind one managed MCP-compatible endpoint, with central authentication, governance and versioning. So a platform team can approve tools once and let many agents reuse them. MCP connections then authenticate with an API key, an Entra managed identity, OAuth on-behalf-of, or no authentication for public servers.

Routines and Tool Search went GA in September 2026, and then browser automation went GA on October 19, 2026. The guide to MCP vs A2A also explains how the two protocols split the work.

A practical rule is to give each agent the smallest tool set its job needs. For example, an HR agent that reads policy documents has no use for code interpreter or browser automation. Leaving them out also shrinks what a bad prompt can trigger.

Grounding Agents on Enterprise Data with Foundry IQ, Azure AI Search and Microsoft Fabric

Grounding is how an agent answers from your documents instead of from what the model memorized. Retrieval-augmented generation (RAG) fetches relevant passages at question time and then passes them to the model with the prompt.

Foundry IQ is a managed knowledge layer built on Azure AI Search agentic retrieval. It plans subqueries, searches several sources in parallel and enforces the caller’s permissions at query time, which makes it a managed form of agentic RAG.

Azure AI Search still suits teams that want index-level control over chunking, fields and ranking. Fabric IQ, the semantic layer in Microsoft Fabric, connects agents to Fabric data. The overview of Fabric IQ explains that layer in more detail.

The rule that matters most is to keep access control inside the retrieval path. An agent should only retrieve what the person asking may see, so permissions must travel with every query. Good RAG development builds that check into the retrieval layer, as the bank case study later in this post shows.

Cost needs a plan too, because indexing a whole SharePoint estate at launch can make search a large line item. Instead, index by use case and add sources as each agent proves its worth.

Evaluation, Tracing and Monitoring in Microsoft Foundry

Evaluation shows that an agent is good enough to ship, while monitoring shows when that stops being true. Foundry builds both into the platform, as the observability documentation describes. Its evaluators fall into four groups:

1. Quality evaluators for coherence, fluency, groundedness and relevance.
2. Safety evaluators for hate and unfairness, violence and protected materials.
3. Agent evaluators for tool call accuracy and task completion.
4. Custom evaluators for rules specific to your domain.

The AI red teaming agent, built on PyRIT, probes an agent with adversarial prompts. It works best with human review, because a person still has to judge which failures matter. The guide to LLM red teaming also covers that process.

Once an agent is live, continuous evaluation scores a sample of production traffic. OpenTelemetry tracing also sends spans into Application Insights, where dashboards show tokens, latency, errors and quality. Alerts then fire when outputs fail their thresholds, which is the core of AI agent observability.

A practical rule is that the team sets pass thresholds per use case and writes them into the release gate. Foundry supplies the measurement, while the threshold stays a business decision. An LLM evaluation framework helps set those numbers before the first release.

Security, Governance and the Foundry Control Plane

Security in Foundry starts with identity. Each new agent gets its own Entra agent identity, so access can be granted, reviewed and revoked per agent, just as it is for people. That identity starts with no resource permissions, and the other controls build on it:

1. RBAC at the resource and project level, covered in the guide to Azure identity management.
2. Private networking for prompt agents and bring-your-own VNet for hosted agents.
3. Customer-managed key encryption.
4. Prompt Shields content filters for user prompt attacks and for document attacks hidden in files, emails or web pages.
5. Azure Policy, so every new project inherits a security baseline.

The Foundry control plane adds a fleet view of inventory, observability, compliance and security, and it integrates with Microsoft Defender, Purview and Entra. It also tracks agents from Foundry and from non-Microsoft sources, along with their cost and token use.

Picture a security lead asked how many agents exist across the company. With a control plane, that answer comes from one view instead of a survey of every team.

Platform controls still leave work inside each agent. Teams add AI guardrails to agent logic and apply sensitivity labels through data governance with Microsoft Purview. For the policy side, see AI governance services.

Azure AI Foundry Pricing: What You Actually Pay For

Azure AI Foundry pricing has no platform charge of its own. Microsoft’s Foundry pricing page says “The Foundry platform is free to use and explore.” Individual features, it also adds, “are billed at their normal rates.”

Each service in a project therefore bills on its own meter, and model tokens are usually the largest one. Microsoft announced these GPT-6 family rates in its September 2026 Foundry update, and the table shows them as of November 3, 2026.

Checked against Microsoft Learn and the Microsoft Foundry Blog on November 3, 2026.

These rates apply to short-context requests, while long-context requests cost more. All three models are GA, with a 1.05M-token context and up to 128K output tokens. The Azure pricing page shows live rates by region, though the numbers above are Microsoft’s September announcement.

Global Batch also cuts token cost by 50% for jobs that can wait up to 24 hours. From December 1, 2026, Copilot Harness sandboxes add $0.0994 per vCPU-hour and $0.0118 per GiB-hour. Overall, a full bill usually draws on eight meters:

1. Model inference tokens.
2. Azure AI Search or Foundry IQ for retrieval.
3. Embeddings for indexing documents.
4. Container compute for hosted agents.
5. Evaluation runs, which consume tokens as well.
6. Storage for files, indexes and traces.
7. Networking, such as private endpoints.
8. Monitoring in Application Insights.

Budget per Accepted Result

Price per million tokens is a weak budgeting unit, because one user answer may take several model calls. A better unit is therefore the cost of a result a reviewer accepts. It adds up retries, tool calls and evaluation tokens, then divides by the answers that pass review.

Picture an agent that drafts contract summaries. If a cheaper model needs three attempts per accepted summary, it can cost more than a pricier model that gets it right the first time. So track that number per project, together with Azure cost optimization habits such as budgets and cost alerts.

Microsoft Foundry vs Azure OpenAI vs Azure Machine Learning vs Copilot Studio

Four Microsoft products overlap in name, so buyers often mix them up. They differ in who builds with them, and also in how much the platform does.

Azure OpenAI is no longer a separate platform decision for most teams, since its resources upgrade into Foundry. Azure Machine Learning comes in when a team trains its own models and runs MLOps around them. The comparison of MLflow, Hugging Face Hub and Azure ML model registries covers that side.

For most new projects, then, the real choice is between Foundry and Copilot Studio. For instance, a business analyst who wants an agent over a SharePoint site can start in Copilot Studio. A team that needs custom tools, hosted code or per-agent evaluation will want Foundry instead.

How to Choose Between Foundry, Bedrock and Vertex AI

Each major cloud now has its own AI platform, and the right one usually follows where your business already runs. The table gives a fit verdict for each one, drawn from each vendor’s own documentation.

Checked against the AWS and Google Cloud documentation on November 3, 2026. Google’s Vertex AI pages now redirect to Gemini Enterprise Agent Platform, so expect that name in its docs.

Feature lists change monthly, so test the verdict against these criteria before committing:

1. Where your data and identity already live.
2. Which models you need, checked against each vendor’s current catalog.
3. Your residency rules and the regions each platform can keep processing inside.
4. Existing cloud commitments, which often settle the choice before any feature does.
5. Who will operate the agents, and which cloud’s tools they already know.

Teams standardized on AWS or Google Cloud generally start with that cloud’s platform. Teams already on Microsoft 365, Entra and Azure usually start with Foundry, since identity, Purview and Teams publishing are in place. The AWS vs Azure vs Google Cloud comparison also gives a wider view.

Moving a Foundry Pilot to Production: A Release Checklist

A working pilot proves an idea, while production proves it can run safely every day. The gap is mostly operational, so a written list helps.

Before the Pilot: Microsoft’s Five-Step Path

Microsoft Learn lays out a five-step path for new builders:

1. Make a first model call.
2. Set up the CLI and the azure-ai-projects 2.x SDK.
3. Choose a model.
4. Build a prompt agent or a hosted agent.
5. Add tools, retrieval and memory.

Before Release

Start with four questions, and answer each one in writing before release:

1. Who can access which agents, and through which Entra groups?
2. What data may each agent retrieve, and is that enforced at retrieval time?
3. How are outputs evaluated before users see them, and which thresholds block a release?
4. How will problems be caught after go-live, through traces, alerts and continuous evaluation?

Those answers cover the design. Pilots also tend to skip five operational checks:

1. Is quota sized for peak load, with retry and backoff handling in the client?
2. Is there a model retirement plan? Microsoft keeps a model retirement schedule, so pin versions and test replacements early.
3. Can you roll back to a previous agent version quickly?
4. Does each project have its own cost alert?
5. Is there a named owner for prompts, models and data connections?

How Kanerika Delivers Microsoft Foundry Projects

Kanerika treats a Foundry project as a governance and data project first, so model choice comes later. Delivery then follows five stages:

1. AI tooling assessment and an adoption roadmap, built around use cases with a measurable business result.
2. Resource and project planning for structure, networking, identity and policy before any agent is built, with rollout by department.
3. Agent design and build with prompt and hosted agents, multi-agent orchestration, memory and retrieval, tested on real business scenarios.
4. Model evaluation across the catalog on accuracy, cost and latency, ending in a recommended model mix.
5. Identity, access and observability, with Entra ID, RBAC, content filters, network isolation, Azure Policy baselines, tracing and access-log reviews.

Engagements move from consultation to a proof of concept, then to a production rollout with governance in place. When agents touch labeled or regulated data, the same plan also covers Microsoft Purview implementation.

Kanerika’s own agents also plug into this pattern. Karl answers questions over structured data, KlarityIQ works over unstructured documents, and FLIP is Kanerika’s automation platform. Kanerika is also a Microsoft Solutions Partner for Data and AI.

Case Study: Permission-Aware Retrieval for an Investment Bank

Before the project, an investment bank searched its documents and databases through separate interfaces, and its access controls sat outside the retrieval process. Kanerika then deployed KlarityIQ over the bank’s RFI document repositories and Karl over its enterprise databases, behind one chat interface. Role-based access was then enforced at the retrieval layer.

The AI document retrieval project for an investment bank delivered 43% faster information retrieval and 100% role-based compliance at the point of retrieval. It also brought a 35% improvement in workforce efficiency. That is the same permission-aware retrieval pattern Foundry IQ and Entra-scoped agents implement.

Bottom Line: Starting With Microsoft Foundry

Azure AI Foundry, now Microsoft Foundry, gives a team one governed place to pick models, build agents and control what they cost. Each of the four big choices has a sensible default, and each default changes only when a measured need appears.

1. Model: shortlist two or three from the catalog, score them with Foundry evaluators, and add model router when task difficulty varies.
2. Deployment type: start on Global Standard, then move to Data Zone, Provisioned or Batch for residency, steady volume or async jobs.
3. Agent type: start with a prompt agent, and move to a hosted agent once you need your own framework or code.
4. Cost unit: budget per accepted result, not per million tokens, and give every project its own cost alert.

Three habits then shorten the path from that first agent to production:

1. Fix data access and residency decisions before the first agent is built.
2. Start with one prompt agent on Global Standard, then earn the move to hosted agents or provisioned capacity with measured load.
3. Write evaluation thresholds into the release gate, so every release meets the same quality bar.

Get those defaults right, and the models, agents, pricing and deployment choices in Azure AI Foundry stay simple to govern as the agent count grows.

Frequently Asked Questions