Skip to content

fix(compilers/openapi): refuse a $ref resolving through itself - #229

Merged
OmarAlJarrah merged 3 commits into
mainfrom
fix/openapi-reentrant-ref-scan
Aug 2, 2026
Merged

fix(compilers/openapi): refuse a $ref resolving through itself#229
OmarAlJarrah merged 3 commits into
mainfrom
fix/openapi-reentrant-ref-scan

Conversation

@OmarAlJarrah

@OmarAlJarrah OmarAlJarrah commented Aug 2, 2026

Copy link
Copy Markdown
Member

Summary

The pre-parse cycle scan let two shapes reach speakeasy's resolver that hang the process instead of producing a diagnostic. Both are reachable from under a hundred bytes.

Both are also validation-clean — zero errors from the parser's validation and from Validate(). $ref is a legal Path Item Object field in 3.1, and a pointer that fails to resolve is a resolution failure rather than a schema violation, so a caller that validates before compiling is not protected.

A pointer that passes through the reference being resolved deadlocks

openapi: 3.1.0
info: {title: t, version: '1'}
paths:
  /a: {$ref: '#/paths/~1a/t'}

Reference.resolve holds that reference's own write lock across the pointer walk, and the walk read-locks every reference it traverses, so re-entering one waits on a lock its own goroutine holds. sync.RWMutex is not reentrant; no concurrency is involved.

The scan resolved the pointer and asked only where it landed. There is no t key, so it read this as a dangling reference and let it through — but the resolver never evaluates the last segment. It deadlocks walking the #/paths/~1a prefix.

The same shape reaches the in-flight reference through GetObject's cache delegation, so it is not only self-reference:

paths:
  /a: {$ref: '#/paths/~1b'}
  /b: {$ref: '#/paths/~1a/t'}

A pointer the scan read differently from the resolver stack-overflows

The resolver trims and percent-decodes the pointer half of a $ref, so '#/paths/~1a ' — one trailing space — names /a there, while a scan reading the raw value called it dangling. That self-reference left the resolution cache pointing at its own reference, and GetObject's delegation recursed until the stack was exhausted.

Any divergence between how this package reads a pointer and how the resolver reads one is a reference the scan cannot see. One space was enough to be one.

Changes

followRefChain reports a three-valued verdict (terminates / cycles / reenters) instead of (cyclic, leftComponents), which also retires the "only meaningful alongside cyclic=true" footgun the old signature documented against itself.

The components carve-out is re-cut along what the resolver can see, not where the pointer points. Its guard extends its reference chain only once a hop completes, so a closed cycle is caught there — and the all-components spelling carries the better message, naming the chain, so those stay its to report. A hop that re-enters a reference never completes, so nothing upstream can report it, and the components spelling deadlocks exactly like the document-position one. Re-entrancy is refused whatever the pointer names.

The safe-memo is narrowed. Re-entrancy is a property of a pointer and the chain reading it, not of a node alone, so a chain is recorded terminating only when no hop passed through a reference. Without that the refusal depends on declaration order. Every hop in a real document passes through mappings like components and schemas and no $ref node at all, so the memo stays fully in force where it earns its keep.

nodeview.InternalPointer mirrors the resolver's normalization, and PointerPath walks the normalized form while keeping every node it passes through — a target alone cannot express a danger that is not at the destination. ResolvePointer had no callers left and is removed.

Test plan

Six reproducers under testdata/openapi/, wired into cycleReproducers so they are also fuzz seeds, and into knownInvalid for the corpus sweep.

Four negative controls pin the shapes that must stay accepted — a prefix naming an off-chain reference, both hops dangling, and the two schema-position shapes. Each was measured against the real resolver before being written down: refusing any of them would be refusing a document that compiles today.

cycle_path_item_prefix_chain.yaml declares its paths in the order that hides the bug, which is the order the memo narrowing exists for.

Planted-defect checks rather than assertions that merely look right, run against the branch as it will merge — in both directions, since a rule like this fails by being too wide as readily as too narrow:

planted defect reddens
re-entrancy detection removed all five prefix reproducers
memo narrowing removed only path-item-prefix-chain, the fixture whose declaration order exists for it
refuse on any reference prefix (too wide) three negative controls
extend the refusal to schema positions two negative controls

The over-refusal pair is what caught a bad control: legal-prefix-names-offchain-ref claimed to cover a prefix naming an off-chain reference, but its /b carried a get and so was not a reference at all. It passed the too-wide mutation. Fixed to match its name, renamed legal-prefix-through-offchain-ref, and it now reddens.

TestCompile_ReentrantPrefixRefusedInBothDeclarationOrders covers both orders at the compile boundary.

Gate: gofmt clean, go vet clean, golangci-lint 0 issues, go build ./..., coverage gate passed at 4221/4221 statements.

FuzzCycleDetector ran 10 minutes / 22.8M executions clean. Before the fix it failed at 29s, 12.6s, 87s and 116s across four runs.

Notes

Reported upstream as speakeasy-api/openapi#231, with a fix in speakeasy-api/openapi#230. The scan stays regardless of whether that lands: a replace directive is ignored when this module is consumed as a dependency, so it would protect cmd/morphic and CI while doing nothing for anyone importing morphic as a library. This refusal is what covers them.

The refusal is now measured, not argued

internal/harness/resolver_oracle_test.go is a differential oracle over 35 generated shapes (5 reference positions x 7 pointer forms). For each it compares what the compiler does against what the resolver actually does, holding the compiler to both directions a model can be wrong in: a shape the resolver cannot survive must be refused, and a shape it resolves cleanly must not be.

Two things it taught us that hand-written controls could not:

  • "The resolver returned" is not "the document is fine." Its first run reported four failures, all of them the oracle's own premise being wrong: an exact self-reference at a document position returns, reporting a circular reference. Refusing it early is a choice about which message is better, not an over-refusal. The helper now reports what the resolver said rather than only that it returned, and a shape the resolver survives-with-an-error binds neither way.
  • An oracle measured in-process would hang on the very regression it exists for. Both halves run in a subprocess, so a deadlock, a stack overflow, and a regressed refusal are all observable. Planting the regression reddens six shapes in ten seconds, each naming the spec.

It lives entirely in a _test.go file, so neither the coverage gate nor the architecture test is affected by a harness that talks to the dependency directly.

check-coverage.sh now sets an explicit -timeout, since a regression in this refusal is a test that never returns rather than one that fails.

The pre-parse cycle scan let two shapes reach speakeasy's resolver that hang
the process rather than producing a diagnostic. Both are reachable from under
a hundred bytes, and both are validation-clean: $ref is a legal Path Item
field in 3.1 and a pointer that fails to resolve is a resolution problem, not
a schema one, so a caller that validates first is not protected.

A pointer that passes through the reference being resolved deadlocks.
Reference.resolve holds that reference's own write lock across the pointer
walk, and the walk read-locks every reference it traverses, so re-entering one
waits on a lock its own goroutine holds. sync.RWMutex is not reentrant, and no
concurrency is involved. The scan resolved the pointer and asked only where it
landed, so it read

    paths:
      /a: {$ref: '#/paths/~1a/t'}

as a dangling reference and let it through -- but the resolver never evaluates
the last segment; it deadlocks walking the '#/paths/~1a' prefix.

followRefChain now reports a three-valued verdict instead of (cyclic,
leftComponents), which also retires the "only meaningful alongside
cyclic=true" footgun. The split between them is what the resolver can see, not
where the pointer points: its guard extends its reference chain only once a
hop completes, so a closed cycle is caught there -- and the all-components
spelling of one carries the better message, so those stay its to report. A hop
that re-enters a reference never completes, so nothing upstream can report it,
and the components spelling deadlocks exactly like the document-position one.
Re-entrancy is therefore refused whatever the pointer names.

Re-entrancy is a property of a pointer and the chain reading it, not of a node
alone, so the safe-memo is narrowed: a chain is recorded terminating only when
no hop passed through a reference. Without that the refusal depends on
declaration order -- declaring the dangling reference first records it as safe
and the later chain short-circuits past the re-entrant hop. Every hop in a
real document passes through mappings like components and schemas and no $ref
node at all, so the memo stays in force where it earns its keep.

A pointer the scan read differently from the resolver stack-overflows. The
resolver trims and percent-decodes the pointer half of a $ref
(references/reference.go GetURI and GetJSONPointer), so '#/paths/~1a ' names
/a there while a scan reading the raw value called it dangling. That
self-reference left the resolution cache pointing at its own reference, and
GetObject's delegation recursed until the stack was exhausted. Reading the
pointer the way the resolver reads it is not a nicety here: any divergence is
a reference the scan cannot see, and one trailing space was enough to be one.

nodeview.InternalPointer now mirrors that normalization and PointerPath walks
the normalized form, keeping every node it passes through -- a target alone
cannot express a danger that is not at the destination. ResolvePointer had no
callers left and is gone.

Six reproducers are added and seeded into the fuzz corpus, with four negative
controls pinning the shapes that must stay accepted -- each measured against
the resolver first, since refusing one would be refusing a document that
compiles. cycle_path_item_prefix_chain.yaml declares its paths in the order
that hides the bug, which is the order the memo narrowing exists for.

Both defects were found by FuzzCycleDetector, where they presented only as
workers dying with "EOF": go test -fuzz wires worker stdio to /dev/null, so
the worker's 10s watchdog panic is discarded and the input recorded as the
crasher is whichever one the worker happened to hold, not the one at fault.

Reported upstream as speakeasy-api/openapi#231, with a fix in
speakeasy-api/openapi#230; the scan stays regardless, since a replace
directive would not reach anyone consuming this module as a library.
Review follow-ups on the re-entrant-reference refusal.

legal-prefix-names-offchain-ref claimed to cover a pointer whose prefix names
a reference off the chain reading it, but its /b carried a `get` and so was
not a reference at all. The fixture exercised a weaker shape than its name,
and a mutation that refused on any reference prefix -- the natural way to get
this rule too wide -- passed it. /b now references a component path item, which
is what the name always said, and the mutation reddens it. Renamed to
legal-prefix-through-offchain-ref, since what matters is the traversal, not
the naming.

legal-schema-prefix-into-path-item read backwards: the chain it covers starts
at the schema and re-enters it through a path item, not the other way round.
Renamed to legal-schema-chain-reentry, and the block comment now says why each
control is safe -- two pass through a reference that is not on the chain, two
re-enter from a schema position, which resolves through jsonschema rather than
the openapi Reference wrapper and never takes the lock.

The reproducer list described cycle_pointer_whitespace_self as a deadlock
under the shared re-entrancy comment. It is a stack overflow, and its cause is
the pointer normalization rather than a re-entrant hop, so it carries its own
note.

The package comment, Cycles, refCycles and diag.CyclicRef all described the
refusals as they were before this change. Each now covers the re-entrant one,
and the counts ("two refusals", "three classes") are gone rather than
incremented -- a maintained count is one more thing to get wrong, and neither
comment needed the number.
The refusal in compilers/openapi/internal/scan models one version of
speakeasy's behavior. Until now the evidence that the model was right was four
hand-written negative controls and a set of reproducers -- an argument, not a
measurement, and the same kind of argument that produced the components
carve-out this branch had to correct.

So ask the resolver. For each of thirty-five generated shapes (five reference
positions x seven pointer forms) the oracle compares what the compiler does
with what the resolver actually does, and holds the compiler to both
directions a model can be wrong in: a shape the resolver cannot survive must
be refused, and a shape it resolves cleanly must not be.

A shape it survives but reports an error on binds neither way, and getting
that distinction wrong is not hypothetical -- the oracle's first run "failed"
on four shapes because it read "the resolver returned" as "the document is
fine". An exact self-reference at a document position returns, reporting a
circular reference. Refusing it early is a choice about which message is
better, not an over-refusal, so the helper reports what the resolver said
rather than only that it returned.

Both halves run in a subprocess. A deadlock and a stack overflow cannot be
observed from inside the process they happen to, and the compiler is subject
to both if its refusal regresses -- measured in-process, the oracle would hang
on exactly the regression it exists to catch rather than failing. Planting that
regression now reddens six shapes in ten seconds, each naming the spec.

It lives entirely in a _test.go file, so neither the coverage gate nor the
architecture test is affected by a harness that talks to the dependency
directly.

Alongside it, check-coverage.sh sets an explicit -timeout. A regression in this
refusal is a test that never returns rather than one that fails, and go test's
ten-minute default is a long time to wait to be told which goroutine is stuck.
@OmarAlJarrah
OmarAlJarrah merged commit 3fd1615 into main Aug 2, 2026
1 check passed
@OmarAlJarrah
OmarAlJarrah deleted the fix/openapi-reentrant-ref-scan branch August 2, 2026 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant