Skip to content

feat(managed): mode + key on managed-table loads - #43

Merged
shefeek-jinnah merged 3 commits into
mainfrom
feat/mode-and-key-on-managed-load
Jul 14, 2026
Merged

feat(managed): mode + key on managed-table loads#43
shefeek-jinnah merged 3 commits into
mainfrom
feat/mode-and-key-on-managed-load

Conversation

@shefeek-jinnah

@shefeek-jinnah shefeek-jinnah commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

What this adds

  1. load_managed_table(..., mode=...) — pick the load mode (replace (default), append, delete, update, upsert) instead of always replacing. replace/append apply the upload directly; delete/update/upsert match rows by the table's declared key.
  2. A table can declare its keycreate_managed_database(..., keys={table: [cols]}) and add_managed_table(..., key=[cols]). Declaring a key is what enables the key-based modes on that table; tables without one stay replace/append-only.

Backward compatibility

  • Omitting mode still replaces — existing callers are unchanged.
  • A hotdata client without key on its decl models silently declares the table key-less (previous behavior), so nothing breaks before the client is regenerated — the key just has no effect until then. The dependency floor bump enforces the key-capable client.

Testing

71 passed / 3 skipped. The mode tests run now; the key tests skipif until the installed client has the key field (they activate automatically once the regenerated client is in).

Before un-drafting

  • Set the exact hotdata floor (replace the TODO in pyproject.toml) to the released key-capable client version.
  • Run ruff + mypy.
  • Downstream: this unblocks the connector PR (hotdata-dlt-destination#38).

load_managed_table gains a `mode` kwarg (replace/append/delete/update/
upsert) instead of always replacing; create_managed_database(keys=) and
add_managed_table(key=) declare a table's row-identity key so the
key-based modes work on it.

The key path needs a hotdata client whose managed-table decl models carry
`key`; the dependency floor is bumped (exact tag TBD at release). Backward
compatible: omitting mode still replaces, and a client without `key`
silently declares the table keyless (previous behavior).
@shefeek-jinnah
shefeek-jinnah marked this pull request as ready for review July 14, 2026 12:31
*,
schema: str,
upload_id: str,
mode: str = "replace",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: load_managed_table here goes through _request_with_retry, which retries on transient errors. Previously the mode was hardcoded to replace (idempotent), so a retry after the server already committed was harmless. With append now reachable, a transient failure after the server commits but before the response is received will re-run the append on retry, duplicating the uploaded rows. replace/delete/update/upsert are idempotent, so append is the only exposed mode with this hazard. Worth confirming the server dedupes by upload_id, or otherwise not retrying append. (not blocking)

schema: str = DEFAULT_SCHEMA,
upload_id: str | None = None,
file: str | None = None,
mode: str = "replace",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super nit: mode is typed as a bare str, so a typo (e.g. mode="upser") passes through silently and only surfaces as a server-side error. Since the SDK enumerates exactly five valid modes, a Literal["replace", "append", "delete", "update", "upsert"] here (and on the managed_client wrapper) would catch that at type-check time and self-document the accepted values. (not blocking)

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only non-blocking nits (see inline comments). Changes are backward-compatible and covered by tests.

@shefeek-jinnah
shefeek-jinnah merged commit 099ce38 into main Jul 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant