fix(cuda): address review feedback on NVIDIA Blackwell/PRIME detection#2073
Closed
kenvandine with Copilot wants to merge 6 commits into
Closed
fix(cuda): address review feedback on NVIDIA Blackwell/PRIME detection#2073kenvandine with Copilot wants to merge 6 commits into
kenvandine with Copilot wants to merge 6 commits into
Conversation
UUID-based CUDA_VISIBLE_DEVICES filtering causes "no CUDA-capable device is detected" on some single-architecture systems (e.g. RTX 50 Blackwell, sm_120) because the CUDA runtime fails to match the nvidia-smi UUID. The restriction is only necessary when there are mixed GPU architectures to hide from an arch-specific binary. If all available NVIDIA GPUs share the same architecture as the installed binary, return empty string so CUDA_VISIBLE_DEVICES is left unset and the runtime enumerates freely. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…well GPUs The /proc/driver/nvidia fallback returns the full marketing name (e.g. "NVIDIA RTX PRO 3000 Blackwell Generation Laptop GPU") but identify_cuda_arch_from_name() only matched consumer RTX 50xx model numbers, leaving professional and non-standard Blackwell SKUs undetected. Add "blackwell" as a top-level keyword for sm_120 — any GPU with "Blackwell" in its name is sm_120. Also enumerate RTX PRO Blackwell model numbers (1000–6000) as explicit fallbacks in case the name doesn't include the architecture generation string. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…fails
On Optimus laptops (e.g. ThinkPad P16 Gen 3) nvidia-smi can fail silently
when the GPU is in power-save mode, causing the CUDA backend to be marked
as "Unsupported GPU". The lspci fallback that follows only has a PCI-ID
name (e.g. "NVIDIA Corporation Device 2c02") for brand-new Blackwell cards
not yet in the lspci database, so identify_cuda_arch_from_name() returns ""
and CUDA stays unsupported even though the driver is loaded.
Add /proc/driver/nvidia/gpus/*/information as an intermediate fallback
between nvidia-smi and lspci. Those files are maintained by the nvidia
kernel module and include the full model name ("NVIDIA GeForce RTX 5090
Laptop GPU") and GPU UUID, which is sufficient for arch detection and
survives Optimus power-save scenarios.
Also add a WARNING log when nvidia-smi detection fails so users and
maintainers can see that the fallback path was taken, and expand the
ModelManager "Backend availability" log to include detected NVIDIA GPUs
(name, compute capability, sm family) so detection failures are visible
at startup without requiring debug log level.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…fload systems On NVIDIA Optimus laptops in On-Demand/PRIME Offload mode, CUDA applications report "no CUDA-capable device is detected" unless __NV_PRIME_RENDER_OFFLOAD=1 is set in the subprocess environment. NVIDIA documents this requirement for CUDA compute workloads on PRIME systems, not just OpenGL/Vulkan. Without this, the dGPU stays in power-save state and the CUDA runtime cannot enumerate any devices, even though /proc/driver/nvidia/gpus shows the kernel module is loaded. Setting the variable is a no-op on non-Optimus (single-GPU) systems. Respects any existing __NV_PRIME_RENDER_OFFLOAD already set by the user. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot
AI
changed the title
[WIP] Fix code based on review comments
fix(cuda): address review feedback on NVIDIA Blackwell/PRIME detection
Jun 1, 2026
Member
|
wrong target branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Applies four fixes from the code review on the NVIDIA Blackwell/PRIME detection changes, covering log correctness, spurious warnings, incorrect device indexing, and unsafe env-var overrides.
Changes
model_manager.cpp— singleLOGcall per GPU entry: The NVIDIA GPU startup log was built across 3–4 chainedLOG(...)calls, risking interleaved output under concurrent access. Suffix string is now assembled first and emitted in one call.system_info.cpp— conditional fallback warning: Thenvidia-smi detection failedwarning was emitted unconditionally, even on machines with no NVIDIA driver. Moved inside thefs::exists("/proc/driver/nvidia/gpus")check so it only fires when the kernel module is actually loaded and the fallback is genuinely attempted.system_info.cpp— drop syntheticgpu.indexin/procfallback:GPUInfo::indexdocuments the physical nvidia-smi device index. The/procpath was assigning a filesystem-iteration ordinal, which could cause wrong device selection.indexis now left at-1;uuidis the identifier for this path.llamacpp_server.cpp— respect existing__NV_PRIME_RENDER_OFFLOAD_PROVIDER: The provider was unconditionally hard-coded toNVIDIA-G0whenever__NV_PRIME_RENDER_OFFLOADwas unset, silently overriding any user-set value. Now gated on the provider also being unset: