Skip to content

feat(gpu): offer the CUDA pack to Pascal cards (0.0.9 kernel floor) - #1585

Merged
gabrielste1n merged 1 commit into
mainfrom
feat/cuda-pascal-gate
Aug 12, 2026
Merged

feat(gpu): offer the CUDA pack to Pascal cards (0.0.9 kernel floor)#1585
gabrielste1n merged 1 commit into
mainfrom
feat/cuda-pascal-gate

Conversation

@gabrielste1n

Copy link
Copy Markdown
Collaborator

⏸ HOLD — do not merge until the Pascal smoke test below passes on real hardware.

With #1584 pinning the packs to whisper.cpp 0.0.9 (which ships sm_61 kernels), this lowers MIN_CUDA_COMPUTE_CAP from 7.5 to 6.1 so GTX 10-series cards are offered the CUDA pack.

Exactly who this affects

  • Cards ≥ 7.5, AMD/Intel, macOS, cloud users: no change — they already passed the gate or never consult it.
  • Cards < 6.1 (Maxwell/Kepler): no change — still routed to Vulkan (new below-floor test added).
  • Pascal/Volta (6.1–7.4): newly offered the CUDA pack. Volta runs via the embedded compute_61 PTX (driver JIT).

Prefer-installed-backend fix included

Without it, a Pascal user already running the Vulkan pack would see the "Enable GPU" CUDA banner reappear over a working setup the moment this gate opens. The picker now prefers whichever pack is installed — mirroring the runtime resolver, which only prefers CUDA once it's actually downloaded. Their dictation never changes out from under them either way.

Bounded worst case (already shipped)

If a Pascal kernel still fails on some card: the dictation completes via the CPU fallback retry, the failure is persisted (WHISPER_GPU_FAILED) so it's never re-attempted silently, and the chip shows "GPU could not be activated — Retry". Nobody loses words; the cost is one wasted download.

Go / no-go: Pascal smoke test

On a GTX 10-series machine running this build:

  1. Settings → Speech-to-Text → Local → Whisper → Enable GPU (should offer CUDA, ~772 MB).
  2. Wait for the chip to reach "GPU acceleration active" (poll reflects the live server, not the download).
  3. Dictate a long sentence; confirm it pastes, no "using CPU instead" toast appears, and whisper-server-status reports gpuBackend: "cuda".
  4. Compare latency against the Vulkan pack on the same box (both should beat CPU; CUDA should beat or match Vulkan).

Full suite: 1923 pass / 0 fail, typecheck + prettier clean.

whisper.cpp release 0.0.9 ships sm_61 kernels (and compute_61 PTX that
covers Volta via driver JIT), so the offering gate drops from 7.5 to
6.1 — GTX 10-series cards get the CUDA pack instead of being routed to
Vulkan. Maxwell and older stay on Vulkan.

The picker now prefers whichever backend is already installed: a Pascal
user running the Vulkan pack keeps their working chip instead of being
re-prompted with a 772 MB CUDA download the moment the gate opens. This
matches the runtime resolver, which only prefers CUDA once it is
actually downloaded.

HOLD FOR HARDWARE SMOKE TEST on a Pascal card before merging (CI proves
sm_61 compiles; only a real GPU proves the kernels run).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant