Skip to content

fix(world-postgres): throw EntityConflictError on duplicate run_created#2983

Merged
VaguelySerious merged 1 commit into
vercel:mainfrom
joeyhotz:fix/postgres-run-created-conflict
Jul 17, 2026
Merged

fix(world-postgres): throw EntityConflictError on duplicate run_created#2983
VaguelySerious merged 1 commit into
vercel:mainfrom
joeyhotz:fix/postgres-run-created-conflict

Conversation

@joeyhotz

@joeyhotz joeyhotz commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Description

events.create with a run_created event inserts the run with onConflictDoNothing(), so when the run already exists the insert returns no row and create resolves with { run: undefined } instead of signalling the conflict. start() asserts on that field and throws Missing 'run' in server response for 'run_created' event — even though the run is healthy and already running.

This is the race start() documents: it fires events.create(run_created) and world.queue() in parallel, and when the worker dequeues first, run_started on the not-yet-existent run takes the resilient start path and creates the run itself. start() already treats EntityConflictError here as benign ("we can safely return", expected in <=1% of cases) — but the Postgres world never gives it one. world-local throws it for this same race and world-vercel surfaces a 409 that maps to it; world-postgres is the only world that resolves silently.

Throwing before the event insert also stops a duplicate run_created landing in the event log.

Observed on a self-hosted Postgres deployment on 3 of 261 runs (~1%, matching the estimate above) of our highest-frequency workflow over 7 days: each affected run has two run_created events and completed normally, while the trigger that started it returned a 500.

How did you test your changes?

Two cases added to packages/world-postgres/test/storage.test.ts (runscreate):

  • rejects a duplicate run_created with EntityConflictError — the direct conflict.
  • rejects run_created when resilient start already created the run — reproduces the production race (run_started on a non-existent run → resilient start → late run_created), asserting exactly one run_created remains in the event log.

Both resolve instead of rejecting against the current insert, and pass with the fix.

  • pnpm --filter @workflow/world-postgres exec vitest run test/storage.test.ts (119 tests)
  • pnpm --filter @workflow/world-postgres typecheck
  • biome check packages/world-postgres/src/storage.ts packages/world-postgres/test/storage.test.ts

PR Checklist - Required to merge

  • 📦 pnpm changeset was run to create a changelog for this PR
  • 🔒 DCO sign-off passes (run git commit --signoff on your commits)
  • 📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete

`events.create` with a `run_created` event inserts the run with
`onConflictDoNothing()`, so a run that already exists resolves with
`{ run: undefined }` rather than signalling the conflict. `start()`
asserts on that field and throws "Missing 'run' in server response for
'run_created' event", even though the run is healthy and already
running.

This is the race `start()` documents: it fires `events.create` and
`world.queue()` in parallel, and when the worker dequeues first,
`run_started` on the not-yet-existent run takes the resilient start
path and creates the run itself. `start()` already treats
`EntityConflictError` as benign, but the Postgres world never gives it
one — world-local throws it, and world-vercel surfaces a 409 that maps
to it.

Throwing before the event insert also stops a duplicate `run_created`
landing in the event log.

Signed-off-by: Joey Hotz <joeyhotz1@gmail.com>
@joeyhotz
joeyhotz requested review from a team and ijjk as code owners July 17, 2026 09:12
@changeset-bot

changeset-bot Bot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 9e0319e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@workflow/world-postgres Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@joeyhotz is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

@joeyhotz

Copy link
Copy Markdown
Contributor Author

Pinging @vercel/workflow

@VaguelySerious
VaguelySerious merged commit 3ddf42e into vercel:main Jul 17, 2026
66 of 102 checks passed
github-actions Bot added a commit that referenced this pull request Jul 17, 2026
…ed (#2983)

Signed-off-by: Joey Hotz <joeyhotz1@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport PR opened against stable: #2992. Merge conflicts were resolved by AI — please review carefully. (backport job run)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants