feat: accept a resolved ManagedDatabase in managed-table ops (skip read probe) - #52
Conversation
…ad probe Add HotdataClient._as_managed_database and widen list_managed_tables, load_managed_table, add_managed_table, delete_managed_table, delete_managed_database, and execute_sql to accept str | ManagedDatabase. Passing an already-resolved ManagedDatabase (as returned by create_managed_database) skips the get_database/list_databases read probe, letting a create+load-scoped key bootstrap and load into a managed database in a single run without a forbidden read. The name/id string path is unchanged, and resolve_managed_database's 403->RuntimeError behavior is preserved. Additive change: minor version bump 0.8.0 -> 0.9.0. Refs hotdata-dlt-destination#55, hotdata-dlt-destination#39.
|
|
||
| def delete_managed_database(self, name_or_id: str) -> None: | ||
| db = self.resolve_managed_database(name_or_id) | ||
| def delete_managed_database(self, name_or_id: str | ManagedDatabase) -> None: |
There was a problem hiding this comment.
super nit: delete_managed_database keeps the name_or_id parameter name while now also accepting a ManagedDatabase. The other five ops in this change renamed their parameter to database. Renaming here would keep the signature naming consistent across the widened ops. (not blocking)
There was a problem hiding this comment.
Thanks! I left this as name_or_id on purpose: it mirrors the sibling database-level op resolve_managed_database(name_or_id), whereas database on the other five denotes which database a table/query lives in. The type widened here, but no op was actually renamed in this PR — the five table/query ops were already database, and the two identifier ops (resolve_/delete_managed_database) were already name_or_id.
There's also a small cost to renaming: name_or_id is public and keyword-callable, so delete_managed_database(name_or_id=...) would break — a backward-incompatible change in what's otherwise an additive 0.9.0. Happy to standardize the identifier name across all DB-level ops in a future major if we want the consistency. Leaving as-is for now.
What
Adds
HotdataClient._as_managed_databaseand widens six managed-table ops to acceptstr | ManagedDatabase:list_managed_tables,load_managed_table,add_managed_table,delete_managed_table,delete_managed_databaseexecute_sql(str | None→str | ManagedDatabase | None)Each op replaces its
resolve_managed_database(...)call with_as_managed_database(...). When passed an already-resolvedManagedDatabase(the valuecreate_managed_databasealready returns, complete withdefault_connection_id), the op skips theget_database/list_databasesread probe and uses the object directly.Why — unblocks hotdata-dlt-destination#55
A Hotdata API key can be scoped to create databases + upload + query + create schemas/tables but not read
/databases(a legitimate, server-advertised tier — the 403 body says so).Today every managed-table op resolves the DB by name first (
resolve_managed_database→ aget_databaseread, then alist_databasesscan). For a create-scoped key both reads return 403, whichresolve_managed_databasemaps to a hardRuntimeError. So:create_managed_database(...)succeeds (the key may create) and returns the id +default_connection_id.load_managed_table(name, …)— re-resolves the name → 403 → the run dies.The enabling fact:
createalready returns everything a load needs. The raw SDK'screate_database_responseexposesdefault_connection_idspecifically so callers can load viaPOST /v1/connections/{id}/schemas/{s}/tables/{t}/loadswithout a further read. The framework was discarding it and re-resolving by name.So the fix is not "make create return the id" (it already does) — it's "let the load/add/query ops accept the already-resolved
ManagedDatabaseand skip the read probe." A caller that caches theManagedDatabasefromcreatecan then drive load/add/query with zero reads, letting a create-scoped key bootstrap and load into a managed database in a single run.Scope / deliberate non-changes
resolve_managed_database's403 → RuntimeErrorbehavior is preserved on purpose: 403 means forbidden, not absent. The "403 → try create" policy stays in the destination, which discriminates viaRuntimeError.__cause__.status == 403(the framework raises... from e, preserving theApiException).Related: hotdata-dlt-destination#39 (name lookups are collision-prone) — the id/name string path is retained precisely for read-capable callers.
Versioning
Additive, backward-compatible → minor bump 0.8.0 → 0.9.0 (per CONTRACT.md's versioning policy).
CONTRACT.mdsemantic guarantees andCHANGELOG.mdupdated;ManagedDatabasewas already in the public export surface, so no surface change.Tests
_ForbiddenDatabasesApi(all reads raiseForbiddenException(403), writes succeed — mirrors a create-scoped key) provingload_managed_table/add_managed_table/execute_sqlwith aManagedDatabaseperform noget_database/list_databasescall and succeed.uv run pytest -q→ 86 passed. Contract-surface test green.Downstream follow-up (after release)
hotdata-dlt-destination bumps its
hotdata-frameworkpin, then inensure_managed_database: catches the 403 →create, caches the returnedManagedDatabase, and passes that object (not the name) intoload_managed_table/add_managed_table/execute_sql.