LLMOps Observability Platforms in 2026: LangSmith vs Arize vs Phoenix vs Langfuse vs W&B Weave

Key Takeaways

• Verdict: LangSmith for LangChain and LangGraph teams, Langfuse for MIT self-hosting, Arize AX for teams already running ML monitoring, Phoenix for free self-hosted tracing and evals, and W&B Weave for W&B experiment teams.
• Arize vs Langfuse: Arize AX wins when LLM and classic ML monitoring must share one view, while Langfuse wins on open licensing and published volume pricing.
• Langfuse vs Phoenix: both self-host their core features for free. Langfuse is MIT with a managed cloud on published tiers, while Phoenix uses the Elastic License 2.0 and has Arize AX as its upgrade path.
• LangSmith self-hosting needs the Enterprise plan, and Arize AX self-hosting does too, while Langfuse and Phoenix self-host for free.
• All of them accept OpenTelemetry traces, so instrument on OTel first. Prompts, datasets, evaluators and dashboards still have to be rebuilt on a switch.
• Vendors meter traces, spans, units and gigabytes differently, so price each one at your own volume. On Langfuse Core, 1 million units a month lists at $101.

A Clean HTTP 200 and a Wrong Answer

Picture a retrieval assistant that passes every test before it ships on a Monday. For three weeks after that, the logs show no errors, no timeouts and no infrastructure alerts. Every request still returns HTTP 200.

Then the complaints arrive. The answers are wrong, but they arrive with total confidence. A prompt update two weeks earlier changed how queries reached the retriever, and the assistant has answered from irrelevant context ever since.

Standard monitoring was never built to flag that failure, because nothing technical broke. Vectara’s hallucination leaderboard, updated October 22, 2026, puts summarization hallucination rates across 111 models between 1.8% and 24.2%. The median is 9.6%, so a typical model gets about one summary in ten wrong.

Each of those wrong answers still returns a clean status code. LLMOps observability is the practice that catches them, so the platform you choose decides how fast you see the next one. This guide compares LangSmith, Arize AX, Phoenix, Langfuse and W&B Weave on fit, cost at volume, self-hosting and lock-in, and it gives the verdict first.

Quick Verdict: Which LLMOps Observability Platform to Choose

Most teams can settle the choice with two facts: the framework they build on and where trace data may live. Find your situation in the table and stop at the first row that fits. The head-to-head sections below break a tie between two finalists.

The verdict rests on vendor pages checked on November 3, 2026.

Head-to-Head: Arize vs Langfuse, Langfuse vs Phoenix and Other Matchups

These are the pairings buyers compare most often. The table gives the short answer for each, and the sections after it explain who wins when.

Arize vs Langfuse: Monitoring Breadth or Open Code

Choose Arize AX when one team watches classic ML models and LLM apps together, because AX puts both in the same platform. Choose Langfuse when licensing and cost control matter more. Its core is MIT, self-hosting is free without usage limits, and its cloud volume pricing is published.

For tracing LLM calls, cost and latency tracking, and LLM-as-judge evaluation, the gap is small, since both cover all three. Volume is where they split. At 1 million units a month, Langfuse Core lists at $101, while AX Pro includes 50,000 spans and larger volume moves to custom Enterprise pricing.

Langfuse vs Phoenix: Two Free Self-Hosted Options

This is the closest matchup, because both are free to self-host. Phoenix promises no license fees, usage limits or feature gates, covering tracing, evals, datasets, experiments and prompt management. Langfuse self-hosts all core features free under MIT, with unlimited usage.

Pick Langfuse if you want MIT terms, or a managed cloud with published tiers later. Note that project-level RBAC, audit logs, retention policies and server-side masking need its paid Enterprise license. Pick Phoenix if you already use OpenInference instrumentation or expect to move up to Arize AX.

LangSmith vs Langfuse: Ecosystem Fit or Framework Freedom

LangSmith wins for LangChain and LangGraph teams, since its datasets, evaluators and annotation queues fit the framework natively. Langfuse wins for mixed frameworks, strict data residency or a fixed budget. LangSmith charges $39 per seat on Plus, then bills extra traces pay-as-you-go through a usage calculator rather than a listed rate. Langfuse publishes its per-unit rates, and it self-hosts free, while LangSmith self-hosting needs Enterprise.

LangSmith vs Arize: Framework Depth or Model Coverage

Choose LangSmith when the app is LangChain-native and the team wants prompt versioning and evals in one product. Choose Arize when LLM apps share a monitoring program with classic ML models. LangSmith prices by seat and base trace, while AX prices by span, so model an agent run on the AX meter before comparing. For a self-hosted Arize option without an Enterprise contract, Phoenix is the route.

Arize vs W&B Weave: Production Monitoring or Experiment Tracking

Pick Arize when the job is watching production models, LLM or classic ML, for drift and quality. Pick Weave when the job is iterating on prompts and fine-tuned models next to W&B experiment runs. Weave meters gigabytes, so trace size drives its bill. Arize meters spans, so trace depth drives its bill.

LangSmith Competitors for Agent Observability: Arize Phoenix, Langfuse and W&B Weave

Teams weighing LangSmith against Arize Phoenix, Langfuse and W&B Weave for agents should test three things. First, check how each renders a nested agent run, which LangSmith shows as a tree. Second, price a real agent trace, because span meters and data meters both grow with every tool call. Third, confirm tool-call spans keep their arguments and results.

Phoenix and Langfuse are the free self-hosted options, so they suit agent teams with residency rules. The agent section below lists the failure modes worth testing in a trial.

LLMOps Observability in Brief: The Five Loops to Test a Platform Against

LLMOps observability is the operating loop that keeps a live LLM application measurable after release. It joins tracing, evaluation, cost tracking and drift detection into one weekly cycle. Every working program runs five loops: instrument, evaluate before release, monitor in production, attribute cost and feed failures back into eval sets.

Teams running enterprise LLM deployments usually get tracing working early. Evaluation and the feedback loop are where programs stall, so test those hardest in a trial. For what to trace and how to set it up, read our LLM observability explainer and instrumentation how-to.

The Metrics a Platform Must Track, in Brief

In a trial, check that the platform charts latency p50 and p95, time to first token, cost per task, faithfulness, retrieval relevance, refusal rate and input-mix drift. Then check that it marks each prompt or model change on those charts, so a drop points straight at its cause.

Cost attribution is easier when an LLM gateway tags each call with a team and a feature first.

The Evaluation Layer in Brief: RAGAS, G-Eval and LLM-as-Judge

RAGAS scores RAG pipelines, G-Eval grades open-ended answers against a rubric, and LLM-as-judge is the general pattern behind both. All three plug into the four platforms through SDKs or custom evaluators.

Most teams run heuristic checks on every trace, an LLM judge on a 10-20% sample and a weekly human slice to calibrate the judge. Budget for that judge too. At 100,000 traces a month and a 20% sample, a small judge model costs about $5.50 a month and a larger one about $110, at prices checked on November 2, 2026. A wider LLM evaluation framework decides which checks gate a release.

LangSmith, Arize, Langfuse and W&B Weave: Platform by Platform

The four platforms overlap on tracing, so the useful differences sit in hosting, licensing, OpenTelemetry support and how each one prices volume. Each profile covers lifecycle fit, hosting, license, OTel, tiers and the team it suits. Tier facts come from vendor pricing pages checked on November 3, 2026.

LangSmith: Deepest Fit for LangChain and LangGraph Teams

LangSmith is the observability and evaluation platform from the team behind LangChain. It covers the full loop with tracing, datasets, evaluators, prompt versioning and annotation queues for human review. Teams weighing LangChain vs LangGraph get the tightest integration with either, including agent runs shown as nested trees.

Hosting is more flexible than many older comparisons suggest. The Enterprise plan runs in the cloud, as a hybrid with a SaaS control plane and a self-hosted data plane, or fully self-hosted. Cloud data can sit in the US or the EU.

LangSmith also accepts OpenTelemetry traces from any OTel-compatible agent, including non-LangChain frameworks, as its OpenTelemetry guide explains. That lowers the lock-in risk earlier reviews warned about.

On LangSmith’s pricing page, the Developer plan is $0 for one seat with up to 5,000 base traces a month, then pay-as-you-go. Plus costs $39 per seat a month with up to 10,000 base traces. Base traces keep 14 days of history, while extended traces keep 180 days as a paid upgrade.

Instrumentation in Python is one decorator on each step you want traced:

from langsmith import traceable

@traceable
def retrieve_documents(query: str) -> list:
    # your retrieval logic
    return docs

@traceable
def generate_response(query: str, docs: list) -> str:
    # your LLM call
    return response

Best fit: teams already building on LangChain or LangGraph who want tracing and evaluation in one product. Teams on other frameworks can still send their traces through OpenTelemetry.

Arize AX and Phoenix: Strongest for Teams Already Running ML Monitoring

Arize ships two products that are easy to confuse. Phoenix is the self-hostable tracing and evaluation tool. Arize AX, by comparison, is the hosted enterprise platform that puts LLM monitoring next to classic ML model monitoring.

Phoenix is released under the Elastic License 2.0, a source-available license. It lets you run and modify Phoenix inside your company but bars offering it to others as a managed service. For most internal platform teams, that limit never comes up. Phoenix is also free to self-host with no feature limits, covering tracing, evals, datasets, experiments and prompt management.

Phoenix is built on OpenTelemetry, so a single register call sets up the tracer provider:

import phoenix as px
from phoenix.otel import register

# Register an OpenTelemetry tracer provider that exports spans to Phoenix
tracer_provider = register(project_name="rag-assistant", auto_instrument=True)

On Arize’s pricing page, AX Free includes 25,000 spans a month, 1 GB of storage and 15-day retention. AX Pro costs $50 a month for 50,000 spans, 10 GB and 30-day retention. Enterprise is custom and runs as SaaS or self-hosted.

The span meter matters especially for agents. One agent run can emit many spans, so a free tier of 25,000 spans covers fewer user requests than the number suggests.

Best fit: teams that already monitor classic ML models and want LLM traces, evals and drift in the same place. It also suits groups running MLOps on Databricks or a similar platform who want one monitoring view across model types.

Langfuse: The Open-Source Default for Data Sovereignty

Langfuse is the open-source option most teams try first when trace data must stay inside their own cloud. It covers tracing, prompt management, datasets, evaluations and annotation in one product. It is framework-neutral, so teams choosing between LlamaIndex, LangChain or Haystack can keep the same backend whichever they pick.

The license is the main draw. Per Langfuse’s open-source page, all product capabilities are MIT-licensed without usage limits. Enterprise modules such as SCIM, audit logging and data-retention policies need a commercial license when self-hosted.

The repository license names ClickHouse, Inc. as the copyright holder. Read it before you fork, as you would for any dependency.

The Python SDK can wrap the OpenAI client, so calls are traced without decorators:

from langfuse.openai import openai  # drop-in replacement for the openai client

# Every call through this client is traced automatically
response = openai.chat.completions.create(
    model="gpt-6.1-sol",
    messages=[{"role": "user", "content": query}]
)

On Langfuse’s pricing page, Hobby is free with 50,000 units a month, 30 days of data access and two users. Core costs $29 a month with 100,000 units, then $8 per 100,000 extra units, falling to $7, $6.50 and $6 at higher volume. Core also keeps 90 days of data.

Pro costs $199 a month and also adds SOC2 and ISO27001 reports plus a HIPAA-ready region. A Teams add-on costs $300 a month, while Enterprise is $2,499 a month.

Best fit: teams with data sovereignty rules, mixed frameworks or a firm cost ceiling who want the full product under an open license.

W&B Weave: Best for Teams Already Living in Weights & Biases

Weave is the LLM tracing and evaluation toolkit inside Weights & Biases. Its strength, above all, is continuity. Traces, evaluations and datasets sit next to the experiment runs and training jobs a research team already tracks in W&B.

A function decorated with weave.op is traced along with its inputs and outputs:

import weave
from openai import OpenAI

weave.init("rag-assistant-project")
client = OpenAI()

@weave.op()
def generate(query: str) -> str:
    response = client.chat.completions.create(
        model="gpt-6.1-sol",
        messages=[{"role": "user", "content": query}]
    )
    return response.choices[0].message.content

Weave also accepts OpenTelemetry trace data at its own OTLP endpoint, according to its OTel documentation. The same endpoint exists on multi-tenant cloud, Dedicated Cloud and Self-Managed deployments, so OTel-instrumented apps can send spans to any of them.

On Weave’s pricing page on CoreWeave, the Free plan is $0 with 1 GB a month of Weave ingestion. Pro starts at $60 a month with 1.5 GB. Extra ingestion costs $0.10 per MB on Pro and $0.20 per MB on Enterprise.

Metering by data volume changes the math. A team logging long agent traces with large retrieved documents fills 1 GB much faster than a team logging short chat turns. So measure your average trace size before you compare.

Best fit: research-heavy teams that iterate on prompts and fine-tuned models side by side and want one line of history from experiment to trace.

Feature-by-Feature LLMOps Platform Comparison

A side-by-side view helps once you know which loop matters most to your team. The table compares the four on hosting, license, OpenTelemetry, entry tiers and workflow, using vendor pages checked on November 3, 2026.

Hosting, License, OTel and Tiers Side by Side

The lock-in row is our editorial assessment. It reflects how much of a typical setup lives in native SDK objects, so it shows what would need rebuilding on a switch.

What the Comparison Shows

Langfuse and Phoenix give the clearest exit paths, because you can run both yourself. LangSmith and Weave reward teams already inside their ecosystems, and leaving them means exporting data and rebuilding native features.

For retrieval-heavy apps, read this table next to your vector database choice. Retrieval quality shows up in every trace, whichever platform records it.

Pricing at Scale: What the Published Tiers Mean at Production Volume

Pricing pages look comparable until you notice the metering unit. LangSmith counts traces, Arize AX counts spans, Langfuse counts billable units, meaning any tracing data point such as a trace, observation or score, and Weave counts data ingested.

One user request can be one trace but ten or more spans, depending on how many retrieval and tool steps it runs. As a result, the same workload can sit inside one free tier and far outside another.

What the Tiers Cost at Volume

Langfuse is the one vendor here whose graduated rates are fully published, so it is the only one we price at volume. On the Core plan, 1 million units a month costs $29 plus 900,000 units at $8 per 100,000, or $101 a month.

At 10 million units, the next 9 million bill at $7 per 100,000. That adds $630, for a total of $731 a month. Both figures are list prices on the Core plan, before discounts.

Weave, by comparison, scales on a different axis. On Pro, every extra 100 MB of ingestion costs $10, so trace size matters as much as trace count.

Per-volume metering pushes teams to sample harder. Sampling is fine for routine traffic, but keep full capture for errors, high-cost requests and flagged users. The traces you need most should never be sampled away.

Retention is the other hidden cost. If base traces disappear after 14 days, any failure you want for the eval set has to be copied into a dataset before then. Build that export into the weekly review, or pay for longer retention on the traces that feed evaluation.

Run the numbers at your own trace volume and average trace size before you shortlist. A two-week trial on real traffic gives a better estimate than any pricing calculator.

OpenTelemetry, Lock-In and What Switching Costs

OpenTelemetry is the open standard for traces, metrics and logs, and it is the cheapest insurance against lock-in. If your app emits standard spans, the backend then becomes a configuration choice.

The GenAI semantic conventions define how to name LLM spans and their attributes, such as model, token counts and operation type. They now live in a dedicated repository, open-telemetry/semantic-conventions-genai. Pin the version you instrument against, because attribute names in this area still change.

All four platforms accept OTel traces in some form. A portable setup points one exporter at whichever backend you choose:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Change the endpoint URL to switch backends; the instrumentation stays the same
provider = TracerProvider()
exporter = OTLPSpanExporter(endpoint="http://your-backend/v1/traces")
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)

Spans, though, are only part of what a team builds. Prompt objects, datasets, annotation queues, evaluator definitions and dashboards live in each vendor’s native SDK or UI. None of them move when you change an exporter.

A platform switch usually means five pieces of work:

1. Re-point the exporter: change the OTLP endpoint and credentials, the smallest job when instrumentation is pure OTel.
2. Rebuild prompt management: move versioned prompts and their deployment labels into the new tool’s format.
3. Migrate datasets: export eval sets and golden answers, then re-import them and re-link them to evaluators.
4. Recreate evaluators: port judge prompts and thresholds, then re-check judge agreement on the new platform.
5. Rebuild dashboards and alerts: recreate cost, latency and quality views and wire alerts back into on-call tools.

Agents also add one more check. Tool calls made through the Model Context Protocol show up as spans too, so confirm the new backend renders them with arguments and results intact.

The practical rule follows from the list. Instrument on OpenTelemetry first, then treat every vendor SDK feature you adopt as a switching cost you have chosen to accept.

Privacy, Compliance and Governance of Prompts in Traces

Every trace is a copy of what a user typed and what the model said back. That makes your observability backend a data store with the same privacy duties as your application database.

A claims assistant, for example, may log policy numbers, names and medical details inside prompts. If those traces flow to a SaaS backend unredacted, you have created a second copy of regulated data outside your main controls.

Redaction is easiest at the edge. A masking step in the app, or in an OpenTelemetry collector you run, replaces names and numbers with typed placeholders before export. Debugging still works, because engineers can see that a policy number was present without seeing the number itself.

Five Controls for Prompt Data in Traces

Five controls cover most of the exposure:

1. Redact before export: strip or mask PII in the app or the collector, before spans leave your network.
2. Set retention on purpose: match trace retention to your data policy instead of the plan default.
3. Choose the data region: LangSmith cloud stores data in the US or the EU. Self-hosted Langfuse, Phoenix or LangSmith Enterprise keep traces in your own cloud instead.
4. Limit who sees raw traces: restrict trace access to the people who debug, and log that access.
5. Test for leaks: add prompt-injection and data-exfiltration cases from LLM red teaming to the eval set.

Teams running private LLMs or building toward sovereign AI often choose self-hosting for this reason alone. Keeping traces in-region is simpler than proving that a vendor’s controls match your own.

For a standards anchor, use NIST AI 600-1, the Generative AI Profile of the AI Risk Management Framework, published on July 26, 2024. Observability records give evidence for its measure and manage functions. They also slot into a wider AI governance framework and responsible AI program.

Agent Observability Inside LLMOps

Agents turn one request into a tree of model calls, tool calls and sub-tasks. Observing them means reading that tree, because the final answer rarely shows where a run went wrong.

Consider a research agent that plans, searches twice, calls a pricing API and writes a summary. If the API returns an empty result and the agent carries on, the summary reads well and is still wrong. The trace is the only place that failure is visible.

Agent Failure Modes Worth an Alert

The table lists the agent failure modes worth an alert. Our guide to AI agent observability covers each one in more depth.

In multi-agent systems, each hand-off between agents is another place for context to drop. Good agent orchestration records those hand-offs as spans, so the tree stays readable when something fails.

How to Choose the Right LLMOps Observability Platform

The verdict table at the top settles most choices from framework, hosting rules and existing tools. When two finalists remain, a trial on real traffic decides it.

Running a Fair Trial

Run the trial on real traffic and judge each candidate on the same five things. First, check how fast an engineer finds the cause of a known failure and how cleanly agent runs render. Then estimate what the volume would cost at full traffic. After that, check how retention and redaction fit your policy, and how much of the setup stays portable on OTel.

Maturity also changes the weighting. A team with one prototype needs fast tracing and a generous free tier more than enterprise access controls. A team with ten production apps needs cost attribution, retention policies and single sign-on before anything else.

Some teams end up with two tools. One workable split pairs a tracing and prompt platform with a second tool for a specialist job. For example, RAG evaluation can run in Phoenix while tracing stays in Langfuse.

Add that second tool only for a measured gap. Every extra platform means another SDK, another retention policy and another place for prompts to leak.

When None of These Fits

Some teams still need a different shape of tool. Helicone, Braintrust, Datadog LLM Observability and MLflow are other options, from gateway-style logging to LLM views inside an existing monitoring or ML platform. Our wider list of AI observability tools compares them in more detail.

A 30-60-90 Day LLMOps Observability Rollout

A rollout works best when each phase ends with one loop running for real. The plan below assumes one production LLM app and a small platform team, so widen the scope for a larger estate.

Days 1-30: Instrument and Baseline

1. Pick the platform with the verdict table above, then set up OpenTelemetry instrumentation.
2. Trace every production request, with token counts, latency and cost tags per feature.
3. Record two to four weeks of baseline latency, cost and refusal rates before you set thresholds.
4. Agree on retention, redaction and access rules with security before traces leave the app.

Days 31-60: Evaluate and Gate

1. Build the first eval set from real production traces, including every failure users reported.
2. Add heuristic checks on all traces, then an LLM judge on a sample.
3. Gate prompt and model changes in CI/CD so that every change runs the eval set first.
4. Calibrate the judge against a weekly slice of human-reviewed traces, so its scores stay trustworthy.

Days 61-90: Monitor and Close the Loop

1. Set alerts on p95 latency, cost per task, faithfulness and input-mix drift once baselines exist.
2. Route flagged traces into an annotation queue, then into the eval set.
3. Review cost per task by feature with product owners at least once a month.
4. Write down the switching plan, separating what is pure OTel from what is vendor-native.

Give each loop a named owner by day 90. Platform engineers usually own instrumentation and alerts, while product teams own eval sets and quality thresholds. Without that split, the feedback loop is the first thing to stop.

Teams on Microsoft platforms can fold the same gates into MLOps in Microsoft Fabric. Budget for people as well as licenses, and our guide to AI development cost shows where LLM project money tends to go.

How Kanerika Runs LLMOps Observability

Kanerika builds and runs production LLM applications for enterprises, and observability is part of the delivery plan from the first release. Our LLM development practice commits to monitoring output quality continuously after launch.

Our MLOps consulting team also brings CI/CD for models, drift detection and model monitoring to the same programs. The approach carries into our RAG development work and into deployments that use FLIP, Kanerika’s own platform.

We stay platform-neutral and pick the tool that fits each client’s framework, hosting rules and compliance needs. Engagements usually move through five stages:

1. Assess the stack fit: framework, hosting model and compliance rules decide the shortlist before any tooling work.
2. Instrument on OpenTelemetry: standard spans from the first production release, so the backend stays swappable.
3. Build the first eval set from production traces: then gate every prompt change in CI/CD against it.
4. Redact PII before traces leave the app: Susan, Kanerika’s AI PII redactor, masks personal data at the source before export.
5. Wire cost and drift alerts into existing monitoring: so LLM signals reach the on-call tools teams already watch.

Case Study: 43% Faster Document Retrieval for an Investment Bank

An investment bank handled unstructured RFI documents and structured data through manual processes. Analysts searched repositories by hand, while routine data requests waited on specialists.

Access controls also sat outside the retrieval process. As a result, every new retrieval path reopened a compliance question.

Kanerika deployed KlarityIQ to process the unstructured documents and Karl to query structured enterprise databases, with chat interfaces across both. Secure role-based access control was applied at the retrieval layer itself. The engagement delivered these results:

• 43% faster information retrieval across documents and databases
• 100% role-based compliance achieved at the point of retrieval
• 35% improvement in workforce efficiency

The engagement put controls at the point where data is retrieved. That is also the point where a trace can show which document reached the prompt and whether the user was allowed to see it.

The lesson for observability is where to look first. Retrieval is where access rules apply and where wrong context enters a prompt, so it is the step most worth instrumenting and evaluating. The full story is in the investment bank case study.

The Bottom Line on LangSmith, Arize, Phoenix, Langfuse and W&B Weave

LLMOps observability decides whether your team or your users find the next wrong answer. All four platforms trace and evaluate, but they part ways on hosting, license, metering and how tightly they bind you to an ecosystem.

Instrument on OpenTelemetry, start the eval set from real failures and recheck tier prices, last verified on November 3, 2026, before you commit. Then settle the choice by your stack. Choose LangSmith for LangChain teams, Langfuse for open self-hosting, Arize AX for ML monitoring shops, Phoenix for free self-hosted evals and Weave for W&B users.

Frequently Asked Questions