Promote llama.cpp Windows CUDA release in README#3250
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the English and Chinese READMEs, as well as the llama.cpp runtime documentation, to announce the release of llama.cpp runtime v0.1.7 and document the new prebuilt Windows CUDA package for SenseVoiceSmall. The review feedback focuses on improving the technical accuracy of the CUDA-related documentation. Specifically, it suggests clarifying that the prebuilt package is compatible with RTX 30-class and newer GPUs due to backward compatibility, and refining the terminology from 'architecture 86' to 'compute capability 8.6' (or 'Ampere architecture').
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .\gguf | ||
| hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf | ||
| .\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav | ||
| # Use the windows-x64-cuda package on RTX 30-class GPUs: |
There was a problem hiding this comment.
The prebuilt CUDA binary can also run on newer NVIDIA GPUs (such as RTX 40-class / Ada Lovelace) due to backward compatibility. Specifying "RTX 30-class (or newer)" would be more accurate and prevent users with newer GPUs from thinking they cannot use this package.
| # Use the windows-x64-cuda package on RTX 30-class GPUs: | |
| # Use the windows-x64-cuda package on RTX 30-class (or newer) GPUs: |
| hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .\gguf | ||
| hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf | ||
| .\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav | ||
| # RTX 30 系列等架构 86 GPU 可使用 windows-x64-cuda 包: |
There was a problem hiding this comment.
The term "架构 86" (architecture 86) is non-standard in CUDA terminology. It is more accurate to refer to it as "Ampere 架构 (算力 8.6)" or "sm_86 架构". Also, mentioning "or newer" (及以上) is helpful since newer GPUs (like RTX 40-series) are backward-compatible.
| # RTX 30 系列等架构 86 GPU 可使用 windows-x64-cuda 包: | |
| # RTX 30 系列及以上(Ampere 架构,算力 8.6+)GPU 可使用 windows-x64-cuda 包: |
| enabled and select the backend at runtime: | ||
| The CPU release ZIPs are portable packages. Tagged releases also publish | ||
| `funasr-llamacpp-windows-x64-cuda.zip` for SenseVoiceSmall graph execution on | ||
| NVIDIA GPUs that match CUDA architecture 86. Download the CUDA ZIP from |
There was a problem hiding this comment.
In CUDA terminology, "CUDA architecture 86" is more precisely referred to as "CUDA compute capability 8.6" or "sm_86 architecture" (which corresponds to the Ampere architecture). Clarifying this will make the technical requirements clearer to users.
| NVIDIA GPUs that match CUDA architecture 86. Download the CUDA ZIP from | |
| NVIDIA GPUs with CUDA compute capability 8.6 (Ampere architecture). Download the CUDA ZIP from |
Summary
Promotes the published llama.cpp runtime v0.1.7 Windows CUDA asset in the main README surfaces.
runtime-llamacpp-v0.1.7release andfunasr-llamacpp-windows-x64-cuda.zipfrom English and Chinese READMEs--backend cudaquickstart lines for Windows usersruntime/llama.cpp/README.mdfrom future-tense packaging language to the published v0.1.7 download pathValidation
runtime-llamacpp-v0.1.7,funasr-llamacpp-windows-x64-cuda.zip, and--backend cudagit diff --checkThis is documentation-only and follows the successful release workflow at https://github.com/modelscope/FunASR/actions/runs/29657743507.