perf(array): gather dict byte values straight into the offset builder

[robert3005]

Rationale for this change

Follow-up to #8902. That PR originally had a Dict fast path, which was removed in review rather than fixed — reasonably, since the version being removed did three full copies of the expanded data, bypassed TakeExecute::precondition, and dropped the is_empty / definitely_all_null guards the neighbouring path has.

The gap it left is worth closing: dictionary encoding is the common representation for low-cardinality string columns, so a pa.string() export of such a column falls all the way through to the canonical route. That route takes the values to full logical length via take_canonical, allocating an intermediate array which is then copied into the builder a second time. The intermediate is proportional to the logical length, while the dictionary is usually far smaller.

Note the description of #8902 still claims Dict as a direct-export encoding and its benchmark list includes three Dict cases — worth correcting separately, since the headline "500,000 repeated URL strings" figure was likely measuring a Dict column.

What changes are included in this PR?

When the target is a DynVarBinBuilder and the dtype is Utf8 or Binary, materialize only the dictionary values and gather by code. Resolving a value is then an O(1) read out of the views slice and data buffers, both hoisted out of the loop. Runs of null codes are filled with for_each_set_index plus append_nulls.

Net effect: one fewer full-length allocation and copy per append.

Deliberately keeping the guards the removed version dropped:

• Same conditions as the generic path it precedes — non-empty, primitive codes, not definitely-all-null codes — so anything not handled falls through unchanged.
• An explicit matches!(array.dtype(), DType::Utf8(_) | DType::Binary(_)) check, rather than relying on the builder downcast alone to imply the source is byte-typed.
• TakeExecute is not called at all, so there is no precondition to bypass.

A code pointing at a null dictionary entry appends a null, matching the canonical route. This is the case worth reviewing most closely: null-ness can come from either the codes or the values, and the test asserts against execute::<Canonical> rather than a hand-written expectation, so the two routes are compared directly.

dict_byte_gather_matches_canonical covers all-valid, null codes, null values, both together, heap values, mixed inlined/heap, leading and trailing null codes, and a single-value dictionary — each across both offset widths and both sliced and unsliced.

On the benchmark

Shapes are 4096 rows across three cardinalities (16, 256, 2048), reduced 16x at review request to fit the CI time budget. At that size the change is within run-to-run noise, so the benchmark is a regression guard, not evidence of a speedup. Performance validation is happening separately, outside this CI environment.

Reviewing this on structural grounds: one fewer full-length allocation and copy, and the guards the earlier attempt dropped are back.

What APIs are changed? Are there any user-facing changes?

No API changes. No behaviour change — output matches the canonical route, which the tests assert directly.

Note for reviewers

The view-resolution closure here overlaps with #8994, which hoists the same resolution in append_varbinview. The two are kept separate so the PRs stay independent — this one gathers by arbitrary code (random access) while #8994 scans sequentially. If both land, factoring the resolution into one shared helper would be a tidy follow-up.

Verification

• cargo test -p vortex-array — 3177 + 72 passed, 0 failed.
• cargo test -p vortex-arrow -p vortex-fsst -p vortex-onpair -p vortex-sparse -p vortex-runend -p vortex-zstd — 529 passed, 0 failed.
• cargo clippy -p vortex-array --all-targets --all-features — clean.
• cargo +nightly fmt --all, git diff --check — clean.

Independent of #8993 and #8994; all three branch from develop and touch disjoint code.

[codspeed-hq]

Merging this PR will not alter performance

✅ 1849 untouched benchmarks
🆕 6 new benchmarks
⏩ 46 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
🆕	Simulation	all_valid[(4096, 16)]	N/A	640.4 µs	N/A
🆕	Simulation	all_valid[(4096, 2048)]	N/A	700.7 µs	N/A
🆕	Simulation	all_valid[(4096, 256)]	N/A	630.3 µs	N/A
🆕	Simulation	with_null_codes[(4096, 16)]	N/A	538.1 µs	N/A
🆕	Simulation	with_null_codes[(4096, 2048)]	N/A	590.1 µs	N/A
🆕	Simulation	with_null_codes[(4096, 256)]	N/A	673.2 µs	N/A

---

_{Comparing claude/dict-varbin-append (3e48f58) with develop (b4f8d34)}

Footnotes

1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[robert3005]

Local before/after benchmark

Compared this commit (3e48f587d) with its parent/shared merge base (452ef6a7d9). The benchmark harness was held constant while only the production implementation was restored for the before run.

Machine: Apple M4 Max, 16 cores, 128 GB RAM, macOS 26.5.2, Rust 1.91.0. Divan reported 100 samples × 100 iterations per case; values below are medians.

Cardinality	Null codes	Before	After	Speedup
16	no	50.99 µs	35.91 µs	1.42×
256	no	50.24 µs	34.24 µs	1.47×
2,048	no	49.06 µs	35.12 µs	1.40×
16	yes	41.99 µs	31.62 µs	1.33×
256	yes	44.97 µs	32.54 µs	1.38×
2,048	yes	44.91 µs	30.54 µs	1.47×

Command:

NO_COLOR=1 CARGO_TARGET_DIR=/private/tmp/vortex-append-bench-target cargo bench -p vortex-array --bench dict_append_varbin