docs: Document Serverless network-volume warm cache (VolumeCache) - #693
docs: Document Serverless network-volume warm cache (VolumeCache)#693promptless[bot] wants to merge 2 commits into
Conversation
Add a new page documenting the runpod-python volume cache: its built-in auto-hydrate/sync behavior, env-var configuration (RUNPOD_VOLUME_CACHE, RUNPOD_VOLUME_CACHE_MAX_GB, RUNPOD_CACHE_DIRS), the VolumeCache class and warm() context manager, and limitations. Wire it into the nav and the optimization checklist. Source: runpod/runpod-python#531 (SLS-367)
![promptless[bot]](https://avatars.githubusercontent.com/in/979718?s=60&v=4){kind=small}
|
|
||
| | Variable | Default | Description | | ||
| |----------|---------|-------------| | ||
| | `RUNPOD_VOLUME_CACHE` | Enabled | Controls whether the built-in volume cache runs. Set to `0`, `false`, or `no` to turn it off. | |
There was a problem hiding this comment.
Env-var names, defaults, and gating come from runpod/serverless/utils/rp_volume_cache.py in PR #531: RUNPOD_VOLUME_CACHE defaults on (disabled only for 0/false/no), RUNPOD_VOLUME_CACHE_MAX_GB defaults to 50, and RUNPOD_CACHE_DIRS is split on os.pathsep (: on Linux). Used to write the configuration table.
Source: https://github.com/runpod/runpod-python/pull/531/files
|
|
||
| ### Directories that are cached | ||
|
|
||
| By default, the volume cache targets the directories where Hugging Face and PyTorch store downloaded model weights: |
There was a problem hiding this comment.
The auto-discovered cache directories (HF_HOME or ~/.cache/huggingface, plus HF_HUB_CACHE and TORCH_HOME when set) come from _discover_model_dirs() in rp_volume_cache.py. Used to write the "Directories that are cached" section.
Source: https://github.com/runpod/runpod-python/pull/531/files
| `VolumeCache` accepts the following arguments: | ||
|
|
||
| | Argument | Default | Description | | ||
| |----------|---------|-------------| |
There was a problem hiding this comment.
The VolumeCache constructor signature and defaults (dirs, namespace defaulting to the endpoint ID, volume_path="/runpod-volume", max_size_gb=None, best_effort=True) and the public export runpod.serverless.VolumeCache come from rp_volume_cache.py and runpod/serverless/__init__.py in PR #531. Used to write the arguments table and code sample.
Source: https://github.com/runpod/runpod-python/pull/531/files
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
…docs Folds in runpod/runpod-python#538 (SLS-367): documents the additive max_workers constructor argument and the size-bucketed transport (small.tar + parallel big/ copy + versioned manifest). Also reconciles the page to the shipping VolumeCache API: the earlier env-var toggles (RUNPOD_VOLUME_CACHE/_MAX_GB/CACHE_DIRS), the warm() method, max_size_gb, and automatic worker-loop wiring were removed upstream in favor of the single explicit opt-in context manager.
Open this suggestion in Promptless to view citations and reasoning process
Adds a new Serverless page documenting
VolumeCache, the runpod-python network-volume warm cache introduced for SLS-367.VolumeCacheis an explicit opt-in context manager (with VolumeCache(dirs=[...])) that hydrates cached files from an attached network volume on cold start and syncs newly written files back, turning a repeated multi-GB model download into a one-time cost per endpoint. The page covers thehydrate()/sync()phases, the constructor arguments (dirs,namespace,volume_path,best_effort,max_workers), the size-bucketed adaptive transport (files below 256 KiB packed intosmall.tar, larger files parallel-copied intobig/, versionedmanifest.jsonwritten last as the commit marker), and its limitations. Adds the page to the Serverless nav and the optimization checklist.The documented API reflects the merged, shipping surface: the earlier env-var toggles (
RUNPOD_VOLUME_CACHE/_MAX_GB/RUNPOD_CACHE_DIRS), thewarm()method,max_size_gb, and automatic worker-loop wiring were dropped upstream in favor of the single explicit primitive.Trigger Events
Tip: Filter the Dashboard by labels or assignees to focus on what matters to you 🔎