docs(examples): VolumeCache warm-cache example + transport benchmarks - #48
Open
deanq wants to merge 9 commits into
Open
docs(examples): VolumeCache warm-cache example + transport benchmarks#48deanq wants to merge 9 commits into
deanq wants to merge 9 commits into
Conversation
|
Capy auto-review is paused for this organization because the usage-cycle auto-review limit has been reached. Increase the limit or turn it off in billing settings to resume automatic reviews. |
deanq
changed the title
deanq
changed the title
…tive transport Serverless CPU-endpoint benchmark sweeping file-shape quadrants (few/many x small/large + mixed HF) across serial/parallel/tar transport in both mirror and hydrate directions. Provides the empirical basis for the 256 KiB size-bucketed adaptive transport. Includes recorded results.
deanq
force-pushed
the
deanq/sls-367-volume-cache-example
branch
from
July 24, 2026 19:58
b058f01 to
3692334
Compare
Repo has no committed uv.lock, so CI's uv sync floats ruff>=0.8.0 to 0.16.0, which formats Python inside Markdown code blocks by default and flags pre-existing repo-wide files. Cap at <0.16 (resolves 0.15.x) to restore the passing behavior CI had before the float.
PR #531 was closed as superseded by #538, which is now merged and ships VolumeCache in runpod 1.12.0. The example pinned runpod to #531's branch, which still resolves but carries the pre-adaptive-transport implementation. - requirements.txt and both rigorous/ install commands now use runpod>=1.12.0 instead of the superseded git branch pin. - Status notes across the example, benchmark, quadrant and rigorous READMEs now cite #538 and name the one remaining gate: a flash-worker image built against runpod>=1.12.0, since the model load runs in the remote worker.
results.json is a run artifact written by benchmark.py, not source, so it should never have been committed. Removed and added to benchmark/.gitignore so a re-run does not re-add it. The measured numbers were already tabulated in quadrant/README.md, which stays as the committed record of the 2026-07-09 run. Reworded the two README pointers and the benchmark.py docstring that referred to the raw file; the hydrate reference now points at the summary matrix, which carries the hydrate winners the raw file was being cited for.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
05_data_workflows/05_volume_warm_cache/example forrunpod.serverless.VolumeCacheplus a benchmark suite, including the quadrant transport benchmark that motivated the adaptive-transport work in runpod-python (#538, merged).Contents:
gpu_worker.py+ README): a Flash@Endpointthat warm-caches its model via VolumeCache (HF cache on local disk, mirrored to the network volume) instead of pointingHF_HOMEat the mount.benchmark/): direct-on-volume vs VolumeCache cold-start A/B (serverless), arigorous/pod-based harness with page-cache eviction and breakeven analysis, and aquadrant/serverless benchmark.benchmark/quadrant/): a CPU serverless worker that sweeps file-shape quadrants (few/many × small/large + a mixed HF-like tree) across serial/parallel/tar transport in both mirror and hydrate directions, with cold page cache and a byte-for-byte correctness gate. Measured results are tabulated inbenchmark/quadrant/README.md.Findings (why adaptive transport)
Mirror (local → volume), mean seconds:
Mean file size decides the winner: many-small-file trees favor tar (metadata collapse; 72× over serial), large files favor parallel (tar's single stream is dead weight). This is the empirical basis for the 256 KiB size-bucketed transport in the runpod-python VolumeCache follow-up.
Dependency status
VolumeCachelanded in runpod-python via #538 (merged) and ships in runpod 1.12.0. This PR pinsrunpod>=1.12.0; the earlier pin pointed at #531's branch, which was closed as superseded and carries the pre-adaptive-transport implementation.Merge order: hold this until runpod 1.12.0 publishes (release-please runpod-python#556).
One gate remains before the example runs end-to-end: the deployed flash-worker image must be built against
runpod>=1.12.0. The model load happens inside the remote GPU worker, so it is that image'srunpodthat must haveVolumeCache— it cannot be added at runtime viadependencies, because the worker'srunpodis imported before the handler runs. Until then the example is labeled illustrative in its README, and the benchmark's direct arm runs standalone via--skip-volumecache.Linear: SLS-367
Test plan
make quality-check(ruff format + lint) passes on the added files.benchmark/quadrant/README.md(the rawresults.jsonis a run artifact and is gitignored, not committed); the rigorous harness was run on a GPU pod.gpu_worker.pyitself has not been run end-to-end — it is blocked on the flash-worker image gate above.