feat: full-text search and schema discovery tools for LangChain agents

[rohan-hotdata]

Exposes RuntimeDB's existing BM25 index as a LangChain tool, adds schema introspection, and rewrites the tool descriptions so a model can plan against the engine's real contract.

hotdata_search_text

Wraps bm25_search(...). The corpus is pinned when the tool is built rather than chosen by the model: nothing in the tool surface lets an agent discover which columns carry a BM25 index, and the engine errors outright instead of falling back to a scan when one is missing. Call make_hotdata_search_tool once per corpus to expose several — the agent then routes on the descriptions.

Three verified engine behaviours are encoded in the SQL it builds:

• Results arrive unsorted. Rows come back in rowid order, like SQLite FTS5, so ORDER BY score DESC is explicit. Confirmed live — without it the scores came back 8.788, 8.092, 8.034, 8.254, 8.496.
• k is passed as the fourth argument, not left to the trailing LIMIT. ORDER BY blocks limit pushdown, so a query relying on LIMIT alone lets the scan fall back to the engine's much larger default bound. Correctness is unaffected and the cost was not measurable at this corpus size, so this is a scan-bound difference rather than an observed slowdown — but passing it is free.
• The projection is narrow by default (searched column + score). The demo table is 85 columns wide; returning all of them would flood the model's context.

hotdata_describe_tables

Reads information_schema, listing tables with their column counts or one table's columns and types, capped so a wide table cannot flood the context. Registered by default, describe_tables=False to omit.

It closes a real gap: without it an agent guesses column names off the shape of data it has already seen. In an early run it produced AVG(review_scores_rating) for a column never shown to it — correct only because the fixture is a well-known public dataset. On proprietary data that guess fails.

Tool descriptions

The substantive change, and not originally on the roadmap. Descriptions now state the dialect, the supported constructs, where to look schema up, and the two constraints that silently produce wrong calls: SQL cannot rank text, and an aggregate query must reference at least one column (COUNT(*) alone is rejected, COUNT(<column>) works).

This is not cosmetic:

SQL tool description	System prompt	Outcome
12 tokens	explicit rules	wrote to_tsvector, query rejected, run aborted
160 tokens	none	search → SQL, correct

Every claim was checked against a live workspace, which corrected two of my own earlier assumptions — LIKE/ILIKE do work (as substring tests, without ranking), and the aggregate rule is "must reference a column", not "ungrouped aggregates fail" (MIN/MAX/AVG ungrouped are fine). tests/test_descriptions.py pins the load-bearing wording.

The search description deliberately names no mechanism, and a test enforces that it never says bm25/vector/hnsw, so hybrid retrieval can land underneath without changing the contract the model was given.

The database tools also steer callers towards ids, since names are non-unique display labels — groundwork for #38.

Also fixes, both already broken on main

• The README quickstart used create_tool_calling_agent/AgentExecutor, neither of which exists in LangChain v1.
• The quickstart and examples/langchain_basic.py ran SQL with no database scope, which the API now rejects with a database is required. The example now runs.

Dependency floors

Raises hotdata-framework>=0.9.0 / hotdata>=0.8.0. The lock already carried those after #35, but the floors still allowed framework 0.4.1 — which uploads through POST /v1/files, a route the API no longer serves, so load_managed_table fails with a bare Not Found. Supersedes #31 (a subset of the same bump, since dependabot has already closed it).

Verification

109 tests, ruff and format clean, no new mypy errors (the 18 pre-existing ones are untyped test functions, unchanged).

Verified end to end against a real workspace rather than only in CI: managed database creation, a 7,535-row load, BM25 index build, direct search, and a full agent loop with LangSmith tracing. That live path is how the framework 0.4.1 breakage was found — no test in the repo touches the upload route.

Repeating the agent step five times: the search tool was called 5/5 and the schema tool 4/5, so the routing behaviour is reliable. The final prose answer was right in 4/5 — one run aggregated over the matched listings instead of all listings in the matched neighbourhoods. That is a model property rather than something these tools control, and demo/README.md says so plainly instead of implying the output is always right.

demo/ and docs/

demo/ runs the whole path against a real workspace and is safe to re-run. It works with any tool-calling model — pass --model '<provider>:<model>'; steps 1–5 need no model at all.

docs/ carries the verified engine contract, the roadmap, and the VectorStore plan, checked in while the repo is small so the reasoning stays with the code. docs/engine-contract.md is the source of truth the descriptions encode — if engine behaviour changes, a description is now lying to the model.

Follow-ups

Filed as #36–#42, grouped by code surface. Nothing here blocks on them: #38 (address databases by id only) is the next intended change, and #23 is stale and can be dealt with separately.

[claude]

nit: max_rows=args.k also caps the SQL tool's rows (not blocking).

max_rows in make_hotdata_tools is shared by hotdata_execute_sql and hotdata_search_text, so with the default --k 5 the agent's aggregate query in step 6 silently comes back truncated to 5 rows — to_records(max_rows=...) drops the rest while metadata.row_count still reports the true count. That is precisely the failure mode demo/README.md attributes to the model ("one run aggregated over the handful of matched listings instead of all listings in those neighbourhoods"), so the demo may be undercutting its own headline result.

Search hits and SQL rows are different budgets; suggest decoupling them:

Suggested change

	max_rows=args.k,
	max_rows=100,

(and keep search_k=args.k as-is below).

[claude]

nit: truncated_at fires on an exact fit (not blocking).

A table with exactly max_columns columns returns exactly max_columns records and gets flagged as truncated, telling the model part of the schema is missing when it is complete — which is the one case where it might go guess a column name instead of trusting the list.

Fetching one extra row distinguishes the two:

result = client.execute_sql(table_columns_sql(table, limit=max_columns + 1), database=database)
records = result.to_records()
...
truncated = len(records) > max_columns
records = records[:max_columns]

[claude]

nit: the model-supplied k has no ceiling (not blocking).

k goes straight into bm25_search(..., k) and the trailing LIMIT — and per the docstring the fourth argument is exactly what bounds the scan. A model that emits k: 100000 (a plausible hallucination when told "ask for more with 'k'") makes the engine rank and ship 100k rows, which are then thrown away down to max_rows in Python. The construction-time k is caller-controlled; the invocation-time one is not.

Clamping to max_rows costs nothing and keeps the two bounds consistent:

    def hotdata_search_text(query: str, k: int | None = None) -> str:
        """Search indexed text by relevance and return ranked rows as JSON."""
        return bm25_search_json(
            client,
            table=table,
            column=column,
            query=query,
            k=default_k if k is None else min(k, max_rows),
            columns=columns,
            max_rows=max_rows,
            database=database,
        )

[claude]

nit: this recipe silently gives up the SQL description's search steer (not blocking).

make_hotdata_tools(client, database="sf_airbnb") here is called without search_table/search_column, so has_search is false and sql_tool_description falls back to the no-search branch — the SQL tool never mentions search_listings/search_reviews. The multi-corpus path documented here therefore loses exactly the behaviour the PR description shows to be load-bearing (12-token description → to_tsvector → aborted run).

There is currently no way to tell make_hotdata_tools "search tools exist under these names" without also pinning one corpus: search_tool_name is ignored unless search_table/search_column are set. Worth either accepting a search_tool_names: Sequence[str] (or letting search_tool_name stand alone) so sql_tool_description can name them, or at minimum noting in this section that callers taking this path should pass their own description= to the SQL tool.

[claude]

super nit: this hand-rolls schema discovery with SELECT * ... LIMIT 1 while the PR ships hl.describe_tables_json for exactly this (not blocking). Using the new helper here would both shorten step 4 and exercise the tool the demo is advertising.

[claude]

super nit: _IDENTIFIER_RE is defined identically in search.py:20 (not blocking). Two copies of the validator that stands between LLM-authored strings and a SQL literal is the kind of duplication that drifts — worth a shared _sql.py with _IDENTIFIER_RE / _validate_identifier.

[claude]

nit: the exact-fit fix opens an IndexError path for max_columns < 1 (not blocking).

make_hotdata_describe_tables_tool(client, max_columns=0) now queries LIMIT 1, gets one row, passes the if not records guard, sets truncated = 1 > 0, then slices records[:0] — and line 93's records[0]['table_schema'] raises IndexError. Before the max_columns + 1 change the same call emitted LIMIT 0, returned no rows and fell into the "no table named" branch: wrong, but not an exception.

make_hotdata_search_tool validates its analogous bound (max_rows must be >= 1); max_columns has no equivalent guard. Cheapest fix is to mirror it in the factory:

    if max_columns < 1:
        raise ValueError(f"max_columns must be >= 1, got {max_columns}")

(and/or clamp inside describe_tables_json, since it is also public API).

[claude]

super nit: the overview has no row cap while the column list does (not blocking).

DEFAULT_MAX_COLUMNS exists "so one wide table cannot flood the context", but the no-argument path — the one an agent calls first, unprompted — returns a row per table with no bound. A database with a few thousand tables floods exactly the context the other branch protects. A LIMIT here plus a truncated_at in the payload would make the two paths consistent.

[claude]

super nit: this five-run measurement predates the max_rows fix (not blocking).

The 5/5, 4/5 and "right in four of five" numbers were gathered while the demo still passed max_rows=args.k, so the SQL tool's aggregate was capped at five rows during those runs. The failure described here (aggregating over matched listings rather than the neighbourhoods) is a distinct, query-construction mistake, so the attribution to the model still reads as correct — but the numbers themselves are stale, and the fix commit only re-verified the row count, not the five-run pass rate. Either re-run the five and update, or note that the figures were taken before the row budget was decoupled.

[claude]

nit: a bare table name silently merges same-named tables across schemas (not blocking).

With no schema part the filter is WHERE table_name = 'listings' only, so if two schemas each hold a listings, describe_tables_json gets both schemas' rows back (ordered table_schema, table_name, ordinal_position) and flattens them into a single payload: line 97 labels it from records[0] — public.listings — while line 98 lists the union of both tables' columns.

The bare form is the one the tool description advertises first ("a table name ('listings' or 'public.listings')"), so the model will use it. The result is that it's told public.listings has columns that only exist in staging.listings — the exact "guess a column that isn't there" failure this tool was added to prevent, except now it looks authoritative. The max_columns + 1 cap makes it worse at the margin: with two 150-column listings tables and the default cap, the payload reads public.listings, 200 columns, truncated_at: 200.

Cheapest fix is to detect it rather than merge — the query already selects table_schema, so the ambiguity is visible in the records:

    schemas = {r["table_schema"] for r in records}
    if len(schemas) > 1:
        return json.dumps(
            {
                "table": table,
                "columns": [],
                "error": f"{table!r} is ambiguous: it exists in {sorted(schemas)}; "
                "qualify it as 'schema.table'",
            },
            indent=2,
        )

(placed after the if not records guard, before truncated is computed). Alternatively keep only records[0]'s schema and note the others.

[claude]

Prior findings all addressed: _sql.py shares the identifier/literal validators, describe_tables_json reads one row past the cap and max_columns < 1 is rejected in both the helper and the factory, model-supplied k is clamped to max_rows, the demo decouples SQL_MAX_ROWS from --k and discovers columns through describe_tables_json, the README multi-corpus recipe pins the first corpus with the reason inline, and demo/README.md re-measures five runs on the fixed budgets. One non-blocking nit left inline.

Note: I could not run pytest/ruff in this sandbox, so the "109 tests, clean" claim is unverified here — CI covers it.