fix: skip byte arrays in shadow mismatch JSON - #116
Closed
lan17 wants to merge 1 commit into
Closed
Conversation
Native JSON expands a Buffer or other Uint8Array view into one decimal element per byte, so an 8 KiB-clamped mismatch preview became a run of digits with no diagnostic value, and a 200 KB view produced roughly 2 MB of intermediate JSON to do it. Replace byte arrays with a "<binary N bytes>" marker at any depth. Plain views are intercepted by the replacer before expansion; Buffer runs its own toJSON first, so it is recognized from that result instead.
lan17
force-pushed
the
fix/shadow-log-skip-binary
branch
from
July 31, 2026 22:39
6cee172 to
8c3115b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #113. Byte arrays are the one shape left where the mismatch JSON preview is actively counterproductive.
Native JSON expands a
Bufferor otherUint8Arrayview into one decimal element per byte, so an 8 KiB-clamped preview becomes a run of digits:That is ~6 characters of JSON per byte of payload: the worst cost-per-byte of any shape, and it crowds every other field out of the capped preview. Measured before this change, a record with a 200 KB
Bufferfield produced ~2 MB of intermediate JSON to emit 8192 characters of the digit7.After, at any depth:
Implementation
Buffer.prototype.toJSONruns before aJSON.stringifyreplacer, which splits the handling:Uint8Arrayview reaches the replacer intact and is swapped for the marker before expansion, so no per-byte array is built.Bufferhas already been converted to{ type: "Buffer", data: [...] }by the time the replacer sees it, so it is recognized from that result. This keeps the array out of the intermediate JSON string, but thetoJSONallocation itself is still paid — the README notes that limit rather than implying otherwise.JSON.stringifyso a bare binary cached value skipstoJSONentirely.A plain record already shaped like
{ type: "Buffer", data: [...] }is reduced the same way. That is deliberate: native JSON renders a realBufferidentically, so collapsing the shape loses no information the previous output preserved.Scope
No public API change.
shadow-log-json.tsis internal and not exported from the root entry, so this alters only the contents of an opt-inlogMismatcheswarning. No new config, metric, label, or dependency. The 2 KiB key cap, 8 KiB value caps,...[truncated]marker, andnull-on-unserializable behavior are untouched.Validation
Node 26.5.0, from
main@34bd022:pnpm typecheck— cleanpnpm test— 409 passed (4 new cases covering nested and top-level views, offset/pooled views, large-buffer crowding, and the look-alike record)pnpm build— clean; verified the marker is present indist/index.jsand the CommonJS bundle loadspnpm test:package— not run locally (corepackunavailable in my environment); it exercises the packed public surface, which this change does not touch. Worth confirming in CI.🤖 Generated with Claude Code