diff --git a/docs/docs/plugins/analytics.md b/docs/docs/plugins/analytics.md index 1ed102107..3696a86b3 100644 --- a/docs/docs/plugins/analytics.md +++ b/docs/docs/plugins/analytics.md @@ -7,6 +7,7 @@ sidebar_position: 3 Enables SQL query execution against Databricks SQL Warehouses. **Key features:** + - File-based SQL queries with automatic type generation - Parameterized queries with type-safe [SQL helpers](../api/appkit/Variable.sql.md) - JSON and Arrow format support @@ -35,7 +36,6 @@ await createApp({ The execution context is determined by the SQL file name, not by the hook call. - ## SQL parameters Use `:paramName` placeholders and optionally annotate parameter types using SQL comments: @@ -56,6 +56,7 @@ Annotate with `INT`, or use `sql.number()` (auto-infers `INT` for values in at the call site. **Supported `-- @param` types** (case-insensitive): + - `STRING`, `BOOLEAN`, `DATE`, `TIMESTAMP`, `BINARY` - `INT`, `BIGINT`, `TINYINT`, `SMALLINT` — bind via `sql.int()` / `sql.bigint()` - `FLOAT`, `DOUBLE` — bind via `sql.float()` / `sql.double()` @@ -103,12 +104,149 @@ The analytics plugin exposes these endpoints (mounted under `/api/analytics`): - `POST /api/analytics/query/:query_key` - `GET /api/analytics/arrow-result/:jobId` +- `POST /api/analytics/metric/:key` — measure a Unity Catalog Metric View (see [Metric views](#metric-views)) ## Format options - `format: "JSON"` (default) returns JSON rows - `format: "ARROW"` returns an Arrow "statement_id" payload over SSE, then the client fetches binary Arrow from `/api/analytics/arrow-result/:jobId` +## Metric views + +`POST /api/analytics/metric/:key` measures a [Unity Catalog Metric View](https://docs.databricks.com/en/metric-views/index.html) that you declared in `config/queries/metric-views.json`. Instead of writing SQL, the caller sends a structured request — which measures to aggregate, which dimensions to group by, and an optional filter — and the plugin builds and runs the `SELECT MEASURE(...) ... GROUP BY ALL` for you against the view. + +The route is **dormant until `metric-views.json` exists**: with no config file, every metric key returns `404`. Declaring the file (and generating types) is covered in [Metric-view types](../development/type-generation.md#metric-view-types); this section documents the runtime endpoint that config activates. + +### Request body + +``` +POST /api/analytics/metric/:key +Content-Type: application/json + +{ + "measures": ["arr", "revenue"], + "dimensions": ["region", "order_date"], + "timeGrain": "month", + "timeDimension": "order_date", + "filter": { "member": "region", "operator": "in", "values": ["EMEA", "APAC"] }, + "limit": 100 +} +``` + +`:key` is a metric key from `metric-views.json`. The body fields: + +| Field | Type | Required | Description | +| --------------- | ---------- | -------- | ------------------------------------------------------------------------------------------------------------ | +| `measures` | `string[]` | yes | Measures to aggregate. At least 1, at most 50. Each becomes `MEASURE() AS `. | +| `dimensions` | `string[]` | no | Dimensions to group by (max 20). Selected verbatim and grouped via `GROUP BY ALL`. | +| `filter` | object | no | Structured predicate tree translated into a parameterized `WHERE` clause (see [Filters](#filters)). | +| `timeGrain` | `string` | no | Bucket a time dimension via `date_trunc('', …)` — e.g. `day`, `month`. Requires `timeDimension`. | +| `timeDimension` | `string` | no | The single dimension `timeGrain` buckets. Must be one of `dimensions`. Required whenever `timeGrain` is set. | +| `limit` | `number` | no | Positive integer row cap (max 100000). | +| `format` | `string` | no | `JSON_ARRAY` (default). `JSON` is accepted as a deprecated alias for it; Arrow formats (`ARROW`, `ARROW_STREAM`) are rejected on this route. | + +Measures and dimensions must be unique across both lists — a name cannot repeat, nor appear as both a measure and a dimension. + +### How the request becomes SQL + +Given a view registered as `catalog.schema.revenue_metrics`, the request above produces (measures and dimensions are sorted for a deterministic SELECT list): + +```sql +SELECT MEASURE(`arr`) AS `arr`, MEASURE(`revenue`) AS `revenue`, + date_trunc('month', `order_date`) AS `order_date`, `region` +FROM `catalog`.`schema`.`revenue_metrics` +WHERE `region` IN (:f_0, :f_1) +GROUP BY ALL +LIMIT 100 +``` + +The metric view's FQN and every measure/dimension identifier are backtick-quoted; filter values are bound as parameters (`:f_0`, `:f_1`, …), never interpolated into the SQL string. + +### Filters + +`filter` is a recursive tree. A leaf is a single predicate: + +```json +{ "member": "region", "operator": "equals", "values": ["EMEA"] } +``` + +Predicates combine with `and` / `or` groups, which can nest: + +```json +{ + "and": [ + { "member": "region", "operator": "in", "values": ["EMEA", "APAC"] }, + { + "or": [ + { "member": "segment", "operator": "equals", "values": ["Enterprise"] }, + { "member": "deal_size", "operator": "gt", "values": [50000] } + ] + } + ] +} +``` + +The operator vocabulary: + +| Operator | SQL | Values | +| --------------------------- | ----------------------- | ------------------ | +| `equals` | `=` | exactly one | +| `notEquals` | `<>` | exactly one | +| `in` | `IN (…)` | one or more | +| `notIn` | `NOT IN (…)` | one or more | +| `gt` / `gte` / `lt` / `lte` | `>` / `>=` / `<` / `<=` | exactly one | +| `contains` | `LIKE :param` | exactly one string | +| `notContains` | `NOT LIKE :param` | exactly one string | +| `set` | `IS NOT NULL` | none | +| `notSet` | `IS NULL` | none | + +For `contains` / `notContains`, the `%…%` wildcards are applied to the *bound parameter value* (`%value%`), not written into the SQL text — so the value is never interpolated, consistent with every other operator. + +An empty `or` group (`{ "or": [] }`) is rejected with `400`; an empty `and` group (`{ "and": [] }`) is accepted and contributes no `WHERE` clause (a no-op). Keep this in mind when building filter trees programmatically. + +Filters are bounded to keep hostile input from exhausting the server: nesting depth ≤ 8, ≤ 100 children per `and` / `or` group, and ≤ 1000 values per predicate. A request that exceeds a cap is rejected with `400`. + +### Executors (cache scope) + +Each entry in `metric-views.json` names the executor the query runs as, which also sets the cache scope. This is fixed by config, not the request: + +| `executor` | Runs as | Cache | +| --------------------------------- | ---------------------------------- | ----------------------- | +| `app_service_principal` (default) | The app service principal | Shared across all users | +| `user` | The requesting user (on-behalf-of) | Per user | + +This mirrors the `.sql` vs `.obo.sql` distinction for [file-based queries](#execution-context). + +### Response + +The response is the same SSE stream as `POST /api/analytics/query/:query_key`. If the SQL warehouse is cold it first emits `warehouse_status` events (see [Warehouse readiness](#warehouse-readiness)), then a single `result` event with the rows as objects: + +```json +{ + "type": "result", + "data": [ + { + "region": "EMEA", + "order_date": "2025-01-01", + "arr": 1200000, + "revenue": 340000 + } + ] +} +``` + +On failure it emits an `error` event instead. + +### Errors and behavior + +| Status | Body | When | +| ------ | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| `404` | `{ "error": "Metric not found" }` | `:key` is not declared in `metric-views.json` (also the response for every key when the file is absent). | +| `400` | `{ "error": "Invalid metric request body (fields: …)", "code": … }` | The request body fails validation. The message names only the offending field paths, never the submitted values. | +| `503` | `{ "error": "Metric registry not available", "code": "METRIC_REGISTRY_LOAD_FAILED" }` | `metric-views.json` is present but malformed or unreadable. | + +Editing `metric-views.json` is picked up on the next request — no server restart is needed. A previously malformed file that you fix likewise starts working on the next request. + ## Frontend usage ### useAnalyticsQuery @@ -118,15 +256,19 @@ React hook that subscribes to an analytics query over SSE and returns its latest ```ts import { useAnalyticsQuery } from "@databricks/appkit-ui/react"; -const { data, loading, error } = useAnalyticsQuery(queryKey, parameters, options); +const { data, loading, error } = useAnalyticsQuery( + queryKey, + parameters, + options, +); ``` **Return type:** ```ts { - data: T | null; // query result (typed array for JSON, TypedArrowTable for ARROW) - loading: boolean; // true while the query is executing + data: T | null; // query result (typed array for JSON, TypedArrowTable for ARROW) + loading: boolean; // true while the query is executing error: string | null; // error message, or null on success warehouseStatus: WarehouseStatus | null; // see "Warehouse readiness" below } @@ -134,11 +276,11 @@ const { data, loading, error } = useAnalyticsQuery(queryKey, parameters, options **Options:** -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| `format` | `"JSON" \| "ARROW"` | `"JSON"` | Response format | -| `maxParametersSize` | `number` | `102400` | Max serialized parameters size in bytes | -| `autoStart` | `boolean` | `true` | Start query on mount | +| Option | Type | Default | Description | +| ------------------- | ------------------- | -------- | --------------------------------------- | +| `format` | `"JSON" \| "ARROW"` | `"JSON"` | Response format | +| `maxParametersSize` | `number` | `102400` | Max serialized parameters size in bytes | +| `autoStart` | `boolean` | `true` | Start query on mount | ### Warehouse readiness @@ -154,8 +296,10 @@ This means a cold start no longer freezes the UI on a stalled spinner. Render th import { useAnalyticsQuery } from "@databricks/appkit-ui/react"; function SpendTable() { - const { data, loading, error, warehouseStatus } = - useAnalyticsQuery("spend_summary", params); + const { data, loading, error, warehouseStatus } = useAnalyticsQuery( + "spend_summary", + params, + ); if (warehouseStatus && warehouseStatus.state !== "RUNNING") { return
Warehouse is {warehouseStatus.state.toLowerCase()}…
; @@ -197,10 +341,7 @@ export function AppShell({ children }) { If you already render your own `` for unrelated app toasts, drop the indicator and call `useResourceStatusToaster()` instead so resource-status toasts share that single Toaster: ```tsx -import { - useResourceStatusToaster, - Toaster, -} from "@databricks/appkit-ui/react"; +import { useResourceStatusToaster, Toaster } from "@databricks/appkit-ui/react"; function App() { useResourceStatusToaster(); @@ -219,7 +360,8 @@ For a fully custom toast body, pass `render` (rendered through `toast.custom`): (
- {agg.worst?.kind} {agg.worst?.state.toLowerCase()} ({agg.activeCount} waiting) + {agg.worst?.kind} {agg.worst?.state.toLowerCase()} ({agg.activeCount}{" "} + waiting)
)} /> @@ -232,8 +374,7 @@ To override copy for a specific kind without rewriting the whole UI, pass `rende renderers={{ warehouse: { title: () => "Spinning up your data", - description: (_s, agg) => - `${agg.affectedLabels.length} chart(s) waiting`, + description: (_s, agg) => `${agg.affectedLabels.length} chart(s) waiting`, }, }} /> @@ -263,11 +404,9 @@ import { useEffect, useId } from "react"; function useLakebaseReadiness() { const id = useId(); - const { publish, unpublish } = useResourceStatusPublisher( - id, - "lakebase", - { kindHint: "lakebase" }, - ); + const { publish, unpublish } = useResourceStatusPublisher(id, "lakebase", { + kindHint: "lakebase", + }); useEffect(() => { publish({ @@ -283,10 +422,10 @@ function useLakebaseReadiness() { **Server config (in `analytics({...})`):** -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| `warehouseStartupTimeoutMs` | `number` | `300000` (5 min) | Maximum time to wait for the warehouse to reach `RUNNING` before failing the request | -| `autoStartWarehouse` | `boolean` | `true` | When `true`, a `STOPPED` warehouse is auto-started on the first request. Set to `false` for cost-controlled deployments where billable warehouse starts must not be triggered by user requests; in that case `STOPPED` surfaces as a `ConfigurationError` | +| Option | Type | Default | Description | +| --------------------------- | --------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `warehouseStartupTimeoutMs` | `number` | `300000` (5 min) | Maximum time to wait for the warehouse to reach `RUNNING` before failing the request | +| `autoStartWarehouse` | `boolean` | `true` | When `true`, a `STOPPED` warehouse is auto-started on the first request. Set to `false` for cost-controlled deployments where billable warehouse starts must not be triggered by user requests; in that case `STOPPED` surfaces as a `ConfigurationError` | **Example with loading/error/empty handling:** @@ -296,21 +435,27 @@ import { sql } from "@databricks/appkit-ui/js"; import { Skeleton } from "@databricks/appkit-ui"; function SpendTable() { - const params = useMemo(() => ({ - startDate: sql.date("2025-01-01"), - endDate: sql.date("2025-12-31"), - }), []); + const params = useMemo( + () => ({ + startDate: sql.date("2025-01-01"), + endDate: sql.date("2025-12-31"), + }), + [], + ); const { data, loading, error } = useAnalyticsQuery("spend_summary", params); if (loading) return ; if (error) return
{error}
; - if (!data?.length) return
No results
; + if (!data?.length) + return
No results
; return (
    {data.map((row) => ( -
  • {row.name}: ${row.cost_usd}
    • +
  • + {row.name}: ${row.cost_usd} +
    • ))}
);