Skip to content

v5: reject out-of-range array indices instead of padding with nulls - #222

Open
hperl wants to merge 1 commit into
evanphx:masterfrom
hperl:fix-unbounded-array-padding-on-add
Open

v5: reject out-of-range array indices instead of padding with nulls#222
hperl wants to merge 1 commit into
evanphx:masterfrom
hperl:fix-unbounded-array-padding-on-add

Conversation

@hperl

@hperl hperl commented Jul 31, 2026

Copy link
Copy Markdown

The problem

With EnsurePathExistsOnAdd, an add whose path names an array index beyond
the end of the array is satisfied by padding the array with nulls, one element
at a time, until the index is reachable. ensurePathExists does this in two
places: once for an array that already exists, and once for an array it has just
created.

Each add copies the array, so the cost grows with the square of the index
while the patch stays the same size. This 46 byte patch is enough:

[{"op":"add","path":"/a/2000000000","value":1}]

Nothing bounds it. ApplyOptions has no limit that applies —
AccumulatedCopySizeLimit is consulted only by copy — the loop checks no
context or deadline, and a caller cannot know the index in advance without
walking the patch itself, which means reimplementing path parsing outside the
library.

For anything that accepts JSON Patch documents from its users, that is a denial
of service from a request body too small to look suspicious. Measuring it on the
current release: a 100 KB body of such operations ran for 97 seconds and
allocated 71 GB before it was stopped; a single ~60 byte operation pins a
core and allocates gigabytes. We hit this in an identity-management service
where the patch body comes straight from an API client.

The fix

RFC 6902, section 4.1
already says the index

MUST NOT be greater than the number of elements in the array

so the padding was outside the specification to begin with. Both loops now
return ErrInvalidIndex instead of padding.

Everything the array actually reaches keeps working, and there are tests for
each: an existing element, the array's length (an append), -, negative indices
where SupportNegativeIndices is on, and creating a fresh array for index 0.

Behaviour change

This is a deliberate behaviour change for patches that were relying on the
padding, and two entries in Cases asserted exactly that:

  • {} + add /a/b/3{"a":{"b":[null,null,null,"hello"]}}
  • {"a":[{"b":"whatever"}]} + add /a/2/b/c{"a":[{"b":"whatever"},null,{"b":{"c":"hello"}}]}

Both now assert the rejection, in TestEnsurePathExistsOnAddRejectsOutOfRangeIndex
alongside the shapes that still succeed. I removed them from Cases rather than
leaving the table asserting two different things.

If you would rather not change the default, I am happy to rework this as an
ApplyOptions field — something like MaxArrayPadding, defaulting to 0 so
it is opt-in to pad. Say which you prefer and I will push it. My reasoning for
proposing the straight fix is that the current behaviour is unbounded work
chosen by untrusted input, so anyone who has not thought about it is exposed by
default, and the RFC is on the side of rejecting.

The full suite passes.

EnsurePathExistsOnAdd padded an array with nulls, one element at a time,
to reach any index a patch named. Each append copies the array, so a
~60 byte operation such as

    [{"op":"add","path":"/a/2000000000","value":1}]

costs O(index) allocations and O(index²) copies. There is no option to
bound it, the loop checks no context, and a caller cannot see the index
in advance without walking the patch itself. In a server that accepts
patches from its clients this is a denial of service: 100 KB of such
operations allocated tens of gigabytes and ran for minutes in our
testing.

RFC 6902, section 4.1 already requires that an array index be at most
the number of elements in the array, so the padding was outside the
specification to begin with. Both padding loops now return
ErrInvalidIndex instead.

Every index the array actually reaches still works: an existing element,
the array's length (an append), "-", negative indices where they are
enabled, and creating a fresh array for index 0. Two cases in Cases
asserted the padding and now assert the rejection in a test of their
own.
hperl added a commit to ory/json-patch that referenced this pull request Jul 31, 2026
Only for use as a go.mod replace target while
evanphx#222 is open. Not part of that
pull request. Import paths deliberately keep the upstream prefix: under a
replace directive the module's packages are still provided under the
path being replaced.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant