Skip to content

fix(cuda): address review feedback on NVIDIA Blackwell/PRIME detection#2073

Closed
kenvandine with Copilot wants to merge 6 commits into
mainfrom
copilot/fix-comments-in-review-thread
Closed

fix(cuda): address review feedback on NVIDIA Blackwell/PRIME detection#2073
kenvandine with Copilot wants to merge 6 commits into
mainfrom
copilot/fix-comments-in-review-thread

Conversation

Copilot AI commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Applies four fixes from the code review on the NVIDIA Blackwell/PRIME detection changes, covering log correctness, spurious warnings, incorrect device indexing, and unsafe env-var overrides.

Changes

  • model_manager.cpp — single LOG call per GPU entry: The NVIDIA GPU startup log was built across 3–4 chained LOG(...) calls, risking interleaved output under concurrent access. Suffix string is now assembled first and emitted in one call.

  • system_info.cpp — conditional fallback warning: The nvidia-smi detection failed warning was emitted unconditionally, even on machines with no NVIDIA driver. Moved inside the fs::exists("/proc/driver/nvidia/gpus") check so it only fires when the kernel module is actually loaded and the fallback is genuinely attempted.

  • system_info.cpp — drop synthetic gpu.index in /proc fallback: GPUInfo::index documents the physical nvidia-smi device index. The /proc path was assigning a filesystem-iteration ordinal, which could cause wrong device selection. index is now left at -1; uuid is the identifier for this path.

  • llamacpp_server.cpp — respect existing __NV_PRIME_RENDER_OFFLOAD_PROVIDER: The provider was unconditionally hard-coded to NVIDIA-G0 whenever __NV_PRIME_RENDER_OFFLOAD was unset, silently overriding any user-set value. Now gated on the provider also being unset:

const char* existing_provider = std::getenv("__NV_PRIME_RENDER_OFFLOAD_PROVIDER");
if (!existing_provider || existing_provider[0] == '\0') {
    env_vars.push_back({"__NV_PRIME_RENDER_OFFLOAD_PROVIDER", "NVIDIA-G0"});
}

kenvandine and others added 5 commits June 1, 2026 22:18
UUID-based CUDA_VISIBLE_DEVICES filtering causes "no CUDA-capable device
is detected" on some single-architecture systems (e.g. RTX 50 Blackwell,
sm_120) because the CUDA runtime fails to match the nvidia-smi UUID.

The restriction is only necessary when there are mixed GPU architectures
to hide from an arch-specific binary. If all available NVIDIA GPUs share
the same architecture as the installed binary, return empty string so
CUDA_VISIBLE_DEVICES is left unset and the runtime enumerates freely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…well GPUs

The /proc/driver/nvidia fallback returns the full marketing name (e.g.
"NVIDIA RTX PRO 3000 Blackwell Generation Laptop GPU") but
identify_cuda_arch_from_name() only matched consumer RTX 50xx model
numbers, leaving professional and non-standard Blackwell SKUs undetected.

Add "blackwell" as a top-level keyword for sm_120 — any GPU with
"Blackwell" in its name is sm_120. Also enumerate RTX PRO Blackwell
model numbers (1000–6000) as explicit fallbacks in case the name
doesn't include the architecture generation string.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…fails

On Optimus laptops (e.g. ThinkPad P16 Gen 3) nvidia-smi can fail silently
when the GPU is in power-save mode, causing the CUDA backend to be marked
as "Unsupported GPU". The lspci fallback that follows only has a PCI-ID
name (e.g. "NVIDIA Corporation Device 2c02") for brand-new Blackwell cards
not yet in the lspci database, so identify_cuda_arch_from_name() returns ""
and CUDA stays unsupported even though the driver is loaded.

Add /proc/driver/nvidia/gpus/*/information as an intermediate fallback
between nvidia-smi and lspci. Those files are maintained by the nvidia
kernel module and include the full model name ("NVIDIA GeForce RTX 5090
Laptop GPU") and GPU UUID, which is sufficient for arch detection and
survives Optimus power-save scenarios.

Also add a WARNING log when nvidia-smi detection fails so users and
maintainers can see that the fallback path was taken, and expand the
ModelManager "Backend availability" log to include detected NVIDIA GPUs
(name, compute capability, sm family) so detection failures are visible
at startup without requiring debug log level.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…fload systems

On NVIDIA Optimus laptops in On-Demand/PRIME Offload mode, CUDA applications
report "no CUDA-capable device is detected" unless __NV_PRIME_RENDER_OFFLOAD=1
is set in the subprocess environment. NVIDIA documents this requirement for
CUDA compute workloads on PRIME systems, not just OpenGL/Vulkan.

Without this, the dGPU stays in power-save state and the CUDA runtime
cannot enumerate any devices, even though /proc/driver/nvidia/gpus shows
the kernel module is loaded.

Setting the variable is a no-op on non-Optimus (single-GPU) systems.
Respects any existing __NV_PRIME_RENDER_OFFLOAD already set by the user.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot AI changed the title [WIP] Fix code based on review comments fix(cuda): address review feedback on NVIDIA Blackwell/PRIME detection Jun 1, 2026
Copilot AI requested a review from kenvandine June 1, 2026 22:23
@kenvandine

Copy link
Copy Markdown
Member

wrong target branch

@kenvandine kenvandine closed this Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants