TL;DR
Databricks system tables are a set of built-in tables that give you SQL access to your own account’s operational data, like billing, compute usage, job runs, and audit history. They live under Unity Catalog, so the same governance and access controls that protect your other data also protect them. Instead of opening support tickets or exporting data by hand, a team can just query these tables directly to answer cost and reliability questions. As of 2026, Databricks ships 15 separate system schemas, and a few of the newer ones are still in preview or beta. Cost attribution starts by joining the billing usage table to the list-prices table, then breaking spend down by workspace, job, and warehouse. These tables aren’t real-time, so teams need to plan for reporting delay, schema changes, and different retention windows when they build dashboards or alerts on top of them.
Key Takeaways Databricks system tables are read-only, Unity Catalog-governed tables under the system catalog that expose billing, compute, job, query, audit, lineage, and AI usage data for an entire account. As of 2026, Databricks ships 15 distinct system schemas, from system.billing and system.access to newer AI Gateway and model-serving tables, with several still in Preview or Beta. Cost attribution starts with joining system.billing.usage to system.billing.list_prices, then attributing spend by workspace, job, SQL warehouse, and tag. System tables are not real-time telemetry, so alerting and dashboard design need to account for latency, schema evolution, and per-table retention windows. The teams that get lasting value build a governed metric layer on top of raw system tables instead of running ad hoc queries, so FinOps, security, and engineering share one set of numbers. Kanerika, a Databricks Consulting Partner, has applied this same governance-first discipline on Databricks migration engagements, including a healthcare analytics migration from Informatica to Databricks that delivered 71% higher reporting accuracy. A Platform Team Finds a 40 Percent Bill Increase With No Owner A platform engineering lead pulls up the monthly Databricks invoice and the total is 40 percent higher than last month. Nobody on the team changed a job configuration, nobody added a new pipeline, and the usual suspects (a runaway cluster, a stuck notebook) turn up nothing on first pass.
The team ends up combing through system.billing.usage line by line, joining it against job run history, until a single retrying pipeline surfaces as the source. That pipeline had been silently failing and retrying for nine days.
Watch on YouTube
Databricks Unity Catalog Explained
System tables live inside Unity Catalog’s system catalog. This walkthrough covers how Unity Catalog governs access, lineage, and audit across a Databricks account.
System tables are what made that join possible in the first place. Without them, the same investigation means opening support tickets, exporting REST API responses, and stitching together spreadsheets by hand. This guide walks through what Databricks system tables actually contain, how to enable and secure them, and how data engineering teams turn raw telemetry into a production-grade cost, reliability, and governance layer.
What Are Databricks System Tables? Databricks system tables are a set of read-only, Databricks-hosted analytical tables that live inside a special system catalog in Unity Catalog . They record operational data about an entire Databricks account, covering billing, compute, jobs, pipelines, SQL queries, audit events, data lineage, and increasingly AI workloads.
Think of them as the account’s own observability layer, shipped by Databricks rather than built by hand. Instead of polling REST APIs or exporting audit logs to a SIEM, engineers query system tables with standard SQL, the same way they would query any managed Delta table.
Three things set them apart from a normal table in the workspace.
They are read-only. Nobody writes to them directly, Databricks populates them from platform telemetry. Most of them cover the whole account, not just one workspace, so a single query can span every workspace in a region. Access is governed entirely through Unity Catalog grants, so the same GRANT SELECT pattern used for any other table applies here. According to Databricks’ own engineering blog on system tables , the goal is to give teams “a consistent layer to spot issues early” by exposing job metadata, execution behavior, lineage, and cost signals in one queryable place, rather than scattered across separate tools.
System tables versus system.information_schema. These are not the same thing. information_schema describes the structure of objects (which columns a table has, which schemas exist). System tables describe activity (who ran what, what it cost, how long it took). Confusing the two is one of the most common early mistakes teams make.
The Complete Databricks System Table Catalog in 2026 As of 2026, Databricks ships fifteen distinct system schemas, each covering a different operational domain. Coverage has expanded quickly since the feature’s general availability, and several schemas remain in Preview or Beta, so it is worth checking current status before building a dependency on any single one.
Schema What it covers Primary use case system.billing Billable usage records, list prices Cost attribution, FinOps system.access Audit events, table and column lineage, clean room activity Security, governance, impact analysis system.compute Cluster history, node timelines, SQL warehouse events Infrastructure and capacity analysis system.lakeflow Jobs, tasks, run timelines, pipeline updates Job and pipeline reliability system.query SQL query execution history SQL performance triage system.data_quality_monitoring Table freshness, completeness, monitoring incidents Data quality tracking system.data_classification Column-level sensitive-data classification results Privacy and compliance controls system.tags Governed-tag history for objects Ownership, chargeback, policy system.ai_gateway AI Gateway usage, external model spend AI workload cost and governance system.serving Model-serving endpoint data and usage Inference cost attribution system.mlflow Experiment and run metadata, run metrics ML experiment governance system.sharing Delta Sharing materialization events External data-sharing audit system.replication Cross-region replication state Disaster recovery visibility system.marketplace Marketplace listing and funnel events Data-product distribution tracking system.storage Predictive optimization and auto-upgrade operations Storage layer operations
Not every schema ships enabled by default. The billing schema is on from the start, but most others need an explicit enablement step, covered in the next section.
Kanerika Service
Databricks Consulting and Governance
Kanerika is a Databricks Consulting Partner that designs Unity Catalog governance, cost attribution models, and production monitoring layers for enterprise Databricks estates.
Explore Databricks Services How to Enable and Secure Databricks System Tables System tables are governed by Unity Catalog, so the first requirement is having at least one Unity Catalog-enabled workspace in the account. Once that condition is met, enabling and securing access follows a repeatable sequence.
Confirm account-admin access. Only account administrators can enable system schemas, and by default no one outside that group can query them.Enable the schemas you need. The billing schema is enabled by default. Everything else, including access, compute, lakeflow, and query, must be turned on individually through the System Schemas API.Grant scoped access, not blanket admin rights. Run GRANT USE CATALOG ON CATALOG system TO group, then USE SCHEMA and SELECT on the specific schemas each team needs.Build persona-based views. FinOps needs billing data, security needs audit and lineage data, and platform engineering needs compute and job data. Few people need all of it.Treat query text and object names as sensitive. The system.query.history and system.access.audit schemas capture raw SQL text and object names, which can expose business logic or personal data if surfaced too broadly.A persona-based grant model usually ends up looking something like the table below. It is a starting point, not a fixed standard, and most enterprises adjust the exact schema list per persona as their governance program matures.
Persona Schemas granted Access level FinOps analyst system.billing, system.tags Governed view, no raw audit access Platform engineer system.compute, system.lakeflow, system.query Direct SELECT on raw tables Security or governance lead system.access, system.data_classification Direct SELECT, audited access Executive or business stakeholder None directly Dashboard only, no table access
Databricks documents an explicit warning against exporting raw system-table data outside the platform without a security review first, since the underlying telemetry can include usernames, IP-adjacent metadata, and full query text. Building governed views instead of granting broad raw-table access is the safer default, and it mirrors the same least-privilege pattern most teams already apply to production data.
Building a Cost Attribution Model From system.billing.usage system.billing.usage is the core FinOps fact table. Every row represents a unit of consumption, tagged with account, workspace, SKU, usage time, DBU quantity, and custom tags.
On its own, that table gives DBU counts, not dollars. The next table, system.billing.list_prices, provides the SKU-level pricing needed to convert usage into an estimated cost. Joining the two by SKU and effective pricing period is the first real step toward attribution.
From there, most FinOps programs build attribution along four dimensions.
By workspace. Which business unit or platform team is driving account-wide consumption.By job or pipeline. Joining billing metadata with Lakeflow run information to calculate spend per workflow, task, and execution.By SQL warehouse. Connecting billing, warehouse metadata, and query history to explain SQL compute spend specifically.By tag. Standardizing cost-center, environment, and owner tags so untagged spend becomes visible instead of disappearing into a shared bucket.A Databricks Community technical blog on cost attribution shows this pattern in practice, joining warehouse-level query history against billing usage by hourly windows to allocate SQL warehouse cost down to individual users. That level of detail is difficult to get any other way once an account has more than a handful of workspaces.
One metric worth tracking that most dashboards skip is cost per successful run, not just total spend. A pipeline that fails and retries five times before succeeding shows up as five times the DBUs for the same business outcome, and that failed-run waste is often invisible until someone builds the query to surface it.
Budgets and anomaly rules are the last piece, and they only work once the attribution model above is in place. A resource monitor set on total account spend catches a runaway bill after the fact. A rule built on system.billing.usage segmented by workspace, SKU, and tag catches an unusual spike in one specific product line while it is still small enough to investigate calmly.
The practical sequence most FinOps teams follow is to let three to six months of attributed spend establish a normal range per workspace and SKU, then alert on deviations from that baseline rather than on an arbitrary fixed dollar ceiling. A fixed ceiling either fires constantly during legitimate growth or stays silent through a genuine anomaly, neither of which builds trust in the alert.
Visualizing Databricks System Tables in Power BI Raw SQL against system tables answers a question once. A dashboard answers it every day, for people who never open a notebook. Because system tables sit behind a standard SQL warehouse endpoint, connecting them to Power BI uses the same connector pattern as any other Databricks SQL data source.
The practical pattern most enterprises land on has three layers.
A governed reporting view in the warehouse that joins the relevant raw system tables and pre-aggregates the noisy columns.A DirectQuery or scheduled-refresh Power BI dataset built on that view, not on the raw system table directly, so a schema change upstream does not break the report.Role-based report access mirroring the Unity Catalog grants, so a FinOps dashboard and an engineering reliability dashboard can draw from the same underlying data without exposing each other’s detail.This is also where the dashboard stops being useful if it only shows totals. An executive-facing FinOps view needs trend lines and cost-per-workload economics, while the platform team’s view needs failure rates and duration regressions at the job level. Building both from the same governed metric layer keeps the numbers consistent across audiences, which is usually where dashboard credibility breaks down.
Row-level or object-level security in the semantic model should mirror the same Unity Catalog grants the underlying data already respects, so a viewer never sees a number in Power BI they could not have queried directly. Scheduling refresh against the SQL warehouse’s auto-suspend window also avoids paying for compute the dashboard does not actually need. Most teams get more mileage from two or three well-governed dashboards, a FinOps view, a reliability view, than from a single all-purpose report that tries to serve every audience and satisfies none of them particularly well in practice.
Watch on YouTube
Enhancing Decision-Making and Reducing Costs with Power BI
A closer look at how governed Power BI dashboards turn raw operational data into faster, cheaper decisions, the same pattern that applies to a Databricks system-tables dataset.
Monitoring Lakeflow Jobs and Pipelines With System Tables Job orchestration in Databricks generates a lot of operational exhaust, and the system.lakeflow schema is where most of it lands. job_run_timeline and job_task_run_timeline record run state, duration, and failure detail at both the job and task level.
That task-level granularity matters more than it sounds like it should. A job can report success overall while one of its tasks silently retried three times, and job-level success metrics alone will never surface that. Querying task run timelines directly is the only way to catch it.
A few patterns turn that raw run history into something a platform team actually uses.
Calculate a rolling success rate by job, owner, and environment, not just a single global number. Flag duration regression by comparing current run times against a trailing baseline, so pipelines that are quietly getting slower surface before they miss an SLA. Join reliability data back to system.billing.usage to see when retries and long runtimes are also driving up spend, since the two problems often share a root cause. One caveat worth building into any alerting design. System-table updates are not guaranteed real-time. Latency has improved substantially for Lakeflow tables specifically, but treating system tables as a live telemetry stream for time-critical alerts is the wrong mental model. They are closer to a same-day operational data warehouse than a monitoring agent. For teams running Delta Live Tables pipelines specifically, the same system.lakeflow schema is where those pipeline runs land alongside every other job.
Using system.query.history to Find SQL Performance Problems system.query.history records statement text, user, execution context, timestamps, status, duration, and compute metadata for every query run against a SQL warehouse. It turns “the dashboard feels slow” into a specific, rankable list of problem queries.
Two different rankings matter here, and conflating them leads to fixing the wrong thing.
Slowest recurring statements. Queries that run long and run often, ranked by total duration across all executions.Highest total compute demand. Short queries that run thousands of times can consume more aggregate compute than one long-running report, even though no single execution looks alarming.Comparing performance by SQL warehouse also separates a query-design problem from a warehouse-sizing problem, since the same query can behave very differently on an undersized warehouse under concurrent load. Joining query history back to billing usage moves the conversation from “this query is slow” to “this query is slow and it costs this much,” which is usually the version that gets budget approved for a fix.
Using Audit and Lineage System Tables for Security and Governance system.access.audit captures account activity broken out by service and action, covering authentication events, configuration changes, and administrative actions. It is the first place a security investigation starts when something needs explaining after the fact.
Lineage lives in a related but distinct pair of tables. system.access.table_lineage tracks reads and writes across tables, pipelines, and notebooks, while system.access.column_lineage goes one level deeper to field-level dependencies. Combined with data classification results, lineage answers a question audit logs alone cannot, which downstream tables and reports actually depend on a given sensitive column.
That combination is what makes change-impact analysis possible before a schema change ships, rather than after something breaks in production. Databricks documents system.operational_data and system.lineage as deprecated in favor of the current access schema, so any implementation guide or older tutorial referencing those names is worth double-checking against current documentation before reuse.
Monitoring Compute and SQL Warehouse Efficiency Cluster and warehouse records in system.compute behave like slowly changing dimensions. A cluster’s configuration, node type, and autoscaling policy can all change over its lifetime, so joins against compute history need to account for effective time ranges, not just a current snapshot.
Node timelines expose the gap between allocated compute and actual workload activity, which is usually where idle or overprovisioned clusters hide. Warehouse event history adds visibility into start, stop, resize, and scaling events behind SQL workload behavior, which is often the missing piece when a performance regression traces back to a configuration change nobody flagged at the time.
Separating a genuine utilization problem from a SQL design problem matters here too. Teams that treat every high-cost workload as a sizing issue tend to over-provision compute rather than fix the query or job that is actually driving the cost, a pattern that shows up repeatedly once workspace-level cost attribution is actually in place.
Extending System Tables to AI and Model-Serving Workloads The newest additions to the system-table family cover AI workloads specifically. Billing SKUs and serving metadata in system.serving now attribute inference cost by endpoint, while system.ai_gateway exposes request volume, token usage, and latency for traffic routed through the AI Gateway.
Joining AI activity with billing records lets platform teams report AI spend by workload and application, rather than by SKU alone, which is closer to how AI budgets actually get discussed at the executive level.
It is worth being precise about what this layer does and does not cover. System tables provide platform telemetry, cost, request volume, latency, and usage patterns. They do not evaluate model quality or output accuracy. That still requires dedicated inference tracing and model-evaluation tooling, and conflating the two is a common early mistake for teams standing up AI observability for the first time.
Designing a Production System-Tables Monitoring Layer Running ad hoc queries against system tables answers today’s question. A production monitoring layer needs to keep answering tomorrow’s, even as Databricks adds columns and new tables over time. A few design decisions separate the two.
Keep raw system tables untouched. Put metric definitions and business logic in governed downstream views instead, so different teams cannot calculate incompatible versions of the same number.Define shared dimensions once. Workspace, owner, workload, compute, SKU, and business unit should mean the same thing to FinOps, security, and engineering.Use incremental reads where streaming is supported. Processing new records only, rather than re-scanning a year of history on every refresh, keeps both cost and latency reasonable.Design for schema evolution. Databricks documents that system tables can gain new columns and nested fields without notice, so downstream models need to tolerate that gracefully.Separate dashboards from alerting logic. A stable KPI view can serve a Databricks SQL dashboard, a Power BI report, and an engineering alert without three separate metric definitions drifting apart.The reference shape looks like this. System tables feed governed operational views, which feed a KPI model, which feeds dashboards and alerts consumed by engineering, FinOps, security, governance, and leadership, each through their own scoped access.
Retention, Latency, and Limitations That Break Production Designs Not every system table follows the same retention window, and assuming otherwise is a common source of broken reports months after launch. Lineage tables, for example, retain a rolling one-year window, with older events removed on a rolling basis, while other schemas follow different rules documented per table in the official system tables reference .
A few limitations are worth designing around from day one.
System tables are not real-time telemetry. Update latency varies by domain and is not a documented SLA. New columns can appear without notice, so explicit field selection is safer than SELECT * in production pipelines. Region boundaries affect completeness. A dashboard querying only one region’s records in a multi-region estate will undercount. Preview and Beta schemas can change shape. Anchor production dependencies on GA schemas where possible. Querying system tables still consumes SQL warehouse compute, so dashboard refresh cost belongs in the overall cost model, not treated as free. Common Databricks System Tables Implementation Mistakes Most system-table implementations that stall or get abandoned share a small set of root causes, not a long list of unrelated problems.
Reporting DBUs instead of dollars. Usage quantity alone rarely satisfies a finance stakeholder asking what something cost.Ignoring slowly changing compute records. Joining only by cluster ID without an effective time range can attribute a workload to the wrong configuration entirely.Grouping all serverless usage together. Serverless jobs, pipelines, SQL, and AI services get lumped into one number that hides which one is actually driving cost.Building dashboards before defining ownership. An alert with no accountable owner rarely gets acted on, no matter how accurate the underlying data is.Granting broad system access to analysts. Unrestricted access to raw audit and query text is a governance gap waiting to surface in a review.Treating audit logs as lineage. The two schemas answer different questions and should not be used interchangeably.Hard-coding schemas that will change. Additive schema evolution is normal for system tables, and downstream pipelines need to tolerate it.None of these are exotic failures. Each one is a modeling or access decision made once, early, and never revisited as usage grew, which is exactly the gap a documented data-engineering checklist is built to close.
Checklist
Data Engineering Checklist for Enterprise Teams
A practical checklist covering pipeline reliability, data quality, observability, and cost, the same disciplines a system-tables monitoring layer depends on.
Get the Checklist → Real-World Implementation Example: From Fragmented Monitoring to an Account-Wide Operations Layer An enterprise running a multi-workspace Databricks estate (details anonymized here, drawn from patterns common across enterprise Databricks engagements) reaches a familiar point. Platform teams cannot reliably assign cost to a business unit, security cannot answer “who touched this table” without opening a ticket, and nobody can say with confidence which pipelines are the reliable ones.
The path out of that state typically follows the same sequence.
Combine the relevant data sources. Billing usage and pricing, Lakeflow run history, compute and warehouse history, query history, audit events, lineage, and tags.Design one metric layer. Workload cost, cost per successful run, retry waste, SQL performance, failure rate, missing ownership, and sensitive-data access, defined once and shared across teams.Add governance controls. FinOps, platform engineering, security, and domain teams get scoped views instead of raw table access.Build separate operational dashboards for reliability, SQL performance, FinOps, and governance, all backed by the same underlying metrics.Convert alerts into actions. Route failed-run spikes, cost anomalies, and untagged workloads to named owners rather than a shared inbox nobody checks.Kanerika has run this same sequence on Databricks migration and platform engagements , including a healthcare analytics migration from Informatica to Databricks that delivered 71% higher reporting accuracy once governed, centralized reporting replaced fragmented manual processes. That same discipline, building the governance and metric layer as part of the migration rather than bolting it on afterward, is what makes a system-tables monitoring layer stick instead of decaying back into ad hoc queries within a quarter.
Case Study
71% Higher Reporting Accuracy With Informatica to Databricks
A healthcare organization replaced fragmented manual reporting with a governed, centralized Databricks platform, delivering 71% higher reporting accuracy across its analytics function.
Read the Case Study → The shift is easiest to see by comparing the state of the platform before and after the metric layer lands.
Signal Before After Cost attribution coverage Total spend visible, ownership unclear Spend attributed to workspace, job, and tag Time to identify a failed workload Manual ticket and log search Direct query against job run timelines Retry and failure waste Invisible inside total DBU spend Isolated as its own tracked line item Governance evidence for audits Assembled ad hoc per request Standing lineage and access reporting
The pattern generalizes well beyond this one migration. Any team standing up a similar control layer is really building the same modern data and AI foundation, governed access, attributed cost, and durable pipelines, just starting from its own current state instead of Informatica.
Datasheet
Build a Modern Data and AI Foundation With Databricks
See how Kanerika helps enterprises modernize data, strengthen governance, and speed up Databricks migrations, the same foundation a system-tables control layer sits on.
View the Datasheet → When System Tables Alone Are Not Enough System tables are an account-level analytical core, not a replacement for every other observability tool. Knowing where the boundary sits prevents both under-building and over-building the monitoring stack.
Need System tables Better source Historical cost, reliability, lineage trends Strong fit N/A Real-time, sub-minute alerting Weak fit Databricks REST APIs, native monitoring Low-level Spark execution diagnostics Not covered Spark UI, cluster metrics Cloud infrastructure and network logsNot covered AWS/Azure/GCP native logging Query-plan level SQL diagnosis Partial (find the query) Query profile, EXPLAIN plans Application traces and business SLOs Not covered Application observability platform
The practical answer is to use system tables as the account-wide analytical core and bring in additional telemetry only for the specific operational decisions that genuinely need it, rather than trying to make one tool cover every layer of the stack.
How to Turn Databricks System Tables Into an Enterprise Control Layer Getting from raw system tables to a control layer the business actually relies on follows a consistent order, regardless of which schema comes first.
Start with the decisions the platform team needs to make, not the tables that happen to be available. Inventory which schemas are GA, Preview, or Beta in the account’s cloud and region, since availability varies. Define canonical joins and metric definitions once, before different teams build conflicting dashboards independently. Make ownership and tagging mandatory wherever technically possible, for every workload, job, and workspace. Build governed reporting views scoped to what each team actually needs. Add alerting only after historical baselines exist, so thresholds reflect real behavior instead of guesses. Review findings with the engineering owners regularly, and turn them into actual workload, compute, and governance changes. This is also where a Databricks Consulting Partner earns its place, not for writing the SQL against system.billing.usage, most platform engineers can do that themselves, but for the surrounding architecture. Kanerika works with enterprise teams on Unity Catalog governance design, FinOps operating models, workload optimization, and the broader data governance program that makes a system-tables control layer durable rather than another dashboard that stops getting maintained after the person who built it moves teams.
Databricks-specific delivery work like metastore and catalog design , MLOps governance , and serverless workload strategy all feed into the same underlying system-table data, which is part of why treating this as one coherent monitoring layer, instead of a collection of disconnected dashboards, tends to hold up over time.
Talk to Kanerika
Ready to Build Your Databricks Control Layer?
Kanerika works with enterprise platform teams on Unity Catalog governance, FinOps operating models, and workload optimization built on Databricks system tables. Talk to our Databricks practice.
Talk to Kanerika → Frequently Asked Questions
What are system tables in Databricks? Databricks system tables are a set of read-only, Databricks-hosted tables under a special system catalog in Unity Catalog. They record operational data for an entire account, covering billing, compute, jobs, queries, audit events, lineage, and AI workloads, so teams can query account activity with standard SQL instead of exporting logs or polling REST APIs by hand.
How do I enable system tables in Databricks? An account administrator enables system tables through the System Schemas API. The billing schema is enabled by default, but most other schemas, including access, compute, lakeflow, and query, need to be turned on individually. At least one Unity Catalog-enabled workspace in the account is required before any system schema can be enabled.
How do I access system.billing.usage in Databricks? Once the billing schema is enabled, an account admin grants USE CATALOG on the system catalog and USE SCHEMA plus SELECT on system.billing to the relevant group. From there, system.billing.usage can be queried like any other table, typically joined against system.billing.list_prices to convert DBU quantities into estimated cost.
How long does Databricks retain system table data? Retention varies by table rather than following one fixed rule. Lineage tables, for example, retain a rolling one-year window with older events removed on a rolling basis, while other schemas follow different documented retention periods. The current retention period for each table is listed in Databricks’ official system tables reference, and it is worth checking before building a report that assumes a specific window.
How do I use Databricks system tables to monitor costs? Start with system.billing.usage joined to system.billing.list_prices to turn DBU consumption into estimated dollars. From there, attribute spend by workspace, by job or pipeline using Lakeflow run data, by SQL warehouse using query history, and by tag for showback and chargeback. Tracking cost per successful run, not just total spend, also surfaces waste from failed and retried jobs that a simple total would hide.
What is the difference between Databricks system tables and information_schema? System tables and information_schema answer different questions. information_schema describes the structure of Unity Catalog objects, such as which columns a table has or which schemas exist. System tables describe activity, such as who ran a query, what a job cost, or how long a pipeline took. Treating the two as interchangeable is a common early mistake.
Can Databricks system tables be used for real-time monitoring? Not reliably. System tables are closer to a same-day operational data warehouse than a live telemetry stream, and update latency varies by domain without a documented SLA. Latency has improved substantially for some schemas, including Lakeflow tables, but time-critical alerting is better served by Databricks’ REST APIs or native monitoring alongside system tables rather than system tables alone.
How do I use Databricks system tables to monitor jobs and query performance? For jobs and pipelines, query system.lakeflow.job_run_timeline and job_task_run_timeline to track success rate, failure frequency, and duration regression at both the job and task level. For SQL performance, system.query.history ranks the slowest recurring statements and the highest total compute demand, and joining it back to billing usage shows which slow queries are also the most expensive ones.
