v5: reject out-of-range array indices instead of padding with nulls - #222
Open
hperl wants to merge 1 commit into
Open
v5: reject out-of-range array indices instead of padding with nulls#222hperl wants to merge 1 commit into
hperl wants to merge 1 commit into
Conversation
EnsurePathExistsOnAdd padded an array with nulls, one element at a time,
to reach any index a patch named. Each append copies the array, so a
~60 byte operation such as
[{"op":"add","path":"/a/2000000000","value":1}]
costs O(index) allocations and O(index²) copies. There is no option to
bound it, the loop checks no context, and a caller cannot see the index
in advance without walking the patch itself. In a server that accepts
patches from its clients this is a denial of service: 100 KB of such
operations allocated tens of gigabytes and ran for minutes in our
testing.
RFC 6902, section 4.1 already requires that an array index be at most
the number of elements in the array, so the padding was outside the
specification to begin with. Both padding loops now return
ErrInvalidIndex instead.
Every index the array actually reaches still works: an existing element,
the array's length (an append), "-", negative indices where they are
enabled, and creating a fresh array for index 0. Two cases in Cases
asserted the padding and now assert the rejection in a test of their
own.
hperl
added a commit
to ory/json-patch
that referenced
this pull request
Jul 31, 2026
Only for use as a go.mod replace target while evanphx#222 is open. Not part of that pull request. Import paths deliberately keep the upstream prefix: under a replace directive the module's packages are still provided under the path being replaced.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
With
EnsurePathExistsOnAdd, anaddwhose path names an array index beyondthe end of the array is satisfied by padding the array with nulls, one element
at a time, until the index is reachable.
ensurePathExistsdoes this in twoplaces: once for an array that already exists, and once for an array it has just
created.
Each
addcopies the array, so the cost grows with the square of the indexwhile the patch stays the same size. This 46 byte patch is enough:
[{"op":"add","path":"/a/2000000000","value":1}]Nothing bounds it.
ApplyOptionshas no limit that applies —AccumulatedCopySizeLimitis consulted only bycopy— the loop checks nocontext or deadline, and a caller cannot know the index in advance without
walking the patch itself, which means reimplementing path parsing outside the
library.
For anything that accepts JSON Patch documents from its users, that is a denial
of service from a request body too small to look suspicious. Measuring it on the
current release: a 100 KB body of such operations ran for 97 seconds and
allocated 71 GB before it was stopped; a single ~60 byte operation pins a
core and allocates gigabytes. We hit this in an identity-management service
where the patch body comes straight from an API client.
The fix
RFC 6902, section 4.1
already says the index
so the padding was outside the specification to begin with. Both loops now
return
ErrInvalidIndexinstead of padding.Everything the array actually reaches keeps working, and there are tests for
each: an existing element, the array's length (an append),
-, negative indiceswhere
SupportNegativeIndicesis on, and creating a fresh array for index0.Behaviour change
This is a deliberate behaviour change for patches that were relying on the
padding, and two entries in
Casesasserted exactly that:{}+add /a/b/3→{"a":{"b":[null,null,null,"hello"]}}{"a":[{"b":"whatever"}]}+add /a/2/b/c→{"a":[{"b":"whatever"},null,{"b":{"c":"hello"}}]}Both now assert the rejection, in
TestEnsurePathExistsOnAddRejectsOutOfRangeIndexalongside the shapes that still succeed. I removed them from
Casesrather thanleaving the table asserting two different things.
If you would rather not change the default, I am happy to rework this as an
ApplyOptionsfield — something likeMaxArrayPadding, defaulting to0soit is opt-in to pad. Say which you prefer and I will push it. My reasoning for
proposing the straight fix is that the current behaviour is unbounded work
chosen by untrusted input, so anyone who has not thought about it is exposed by
default, and the RFC is on the side of rejecting.
The full suite passes.