perf(array): hoist per-row work out of append_varbinview

[robert3005]

Rationale for this change

Follow-up to #8902. DynVarBinBuilder::append_varbinview is the fallback every string encoding without a direct branch reaches through canonicalization: Sparse, RunEnd, Dict, and anything on the default VTable::append_to_builder. It was written per-row.

AGENTS.md names both anti-patterns it hit. Converting views to offsets does have to visit each value — that part is irreducible, and AGENTS.md explicitly carves it out ("Gather over arbitrary indices → you cannot amortize a per-element decode, but you can still materialize the backing buffer once"). The work around the walk is what this removes.

What changes are included in this PR?

• Resolve the backing storage once. bytes_at re-resolved the views slice and returned an owned ByteBuffer on every call. The views slice and data buffer slices are now hoisted and borrowed from. Inlined values (≤ 12 bytes, the common short-string case) come straight out of the view rather than round-tripping the views handle through into_byte_buffer().slice_ref(..).
• Walk validity a word at a time. bit_buffer().iter().enumerate() paid a branch per element. Now for_each_set_index, with gaps filled by append_n_nulls — the same shape the Zstd path uses.
• Size the data buffer once. The byte total is a sum over the fixed-width views and needs no value access, so a new private reserve_data sizes the buffer up front rather than letting it reallocate as it grows. reserve_exact takes a row count and so cannot serve this purpose, hence the separate method.

Net effect: the function no longer allocates per row, and the data buffer no longer reallocates during an append.

append_varbinview_matches_source covers inlined-only, heap-only, mixed, all-null, leading/trailing nulls, and empty, each across both offset widths and both sliced and unsliced. Slicing matters because it shifts the validity bit offset, which the word-at-a-time walk must respect. for_each_set_index needs no bounds check here: iter_padded zero-pads the remainder word, so it never yields an index past len.

On the benchmark

Sizes are 4096 and 16384, chosen to keep every case well under 1ms (medians ~110–180µs). At those sizes the change is within run-to-run noise, so the benchmark is a regression guard, not evidence of a speedup. Performance validation is happening separately, outside this CI environment — the run-to-run spread on the runner used here is wide enough that small deltas are not defensible either way.

Reviewing this on structural grounds: fewer allocations per append, and conformance with the hot-loop guidance in AGENTS.md.

What APIs are changed? Are there any user-facing changes?

No public API changes. reserve_data is private to the module. Output is unchanged.

Verification

• cargo test -p vortex-array -p vortex-arrow -p vortex-fsst -p vortex-onpair -p vortex-sparse -p vortex-runend -p vortex-zstd — 3177 + 201 + 85 + 36 + 70 + 109 + 28 + 72 passed, 0 failed.
• cargo clippy -p vortex-array --all-targets --all-features — clean.
• cargo +nightly fmt --all, git diff --check — clean.

Independent of #8993 and #8995; all three branch from develop and touch disjoint code.

[codspeed-hq]

Merging this PR will degrade performance by 20.83%

❌ 1 regressed benchmark
✅ 1848 untouched benchmarks
🆕 8 new benchmarks
⏩ 46 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	decode_varbin[(1000, 8)]	27 µs	34.1 µs	-20.83%
🆕	Simulation	heap_all_valid[4096]	N/A	395.5 µs	N/A
🆕	Simulation	heap_with_nulls[4096]	N/A	416.3 µs	N/A
🆕	Simulation	inlined_all_valid[4096]	N/A	236.5 µs	N/A
🆕	Simulation	inlined_with_nulls[4096]	N/A	304.9 µs	N/A
🆕	Simulation	heap_all_valid[16384]	N/A	1.5 ms	N/A
🆕	Simulation	heap_with_nulls[16384]	N/A	1.6 ms	N/A
🆕	Simulation	inlined_all_valid[16384]	N/A	905 µs	N/A
🆕	Simulation	inlined_with_nulls[16384]	N/A	1.1 ms	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing claude/varbin-builder-bulk-append (8110b99) with develop (b4f8d34)}

Footnotes

1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[robert3005]

Local before/after benchmark

Compared this commit (8110b9926) with its parent/shared merge base (452ef6a7d9). The benchmark harness was held constant while only the production implementation was restored for the before run.

Machine: Apple M4 Max, 16 cores, 128 GB RAM, macOS 26.5.2, Rust 1.91.0. Divan reported 100 samples × 100 iterations per case; values below are medians.

Case	Size	Before	After	Speedup
heap, all valid	4,096	46.58 µs	24.29 µs	1.92×
heap, all valid	16,384	184.7 µs	93.87 µs	1.97×
heap, nulls	4,096	35.89 µs	25.12 µs	1.43×
heap, nulls	16,384	140.3 µs	101.8 µs	1.38×
inline, all valid	4,096	46.70 µs	19.83 µs	2.36×
inline, all valid	16,384	184.4 µs	76.68 µs	2.41×
inline, nulls	4,096	35.83 µs	22.04 µs	1.63×
inline, nulls	16,384	141.2 µs	88.70 µs	1.59×

Command:

NO_COLOR=1 CARGO_TARGET_DIR=/private/tmp/vortex-append-bench-target cargo bench -p vortex-array --bench varbin_append_varbinview