Skip to content

docs(p5en): add the NCCL NVLS fix that blocks every multi-GPU serve - #173

Merged
aarora79 merged 1 commit into
mainfrom
docs/p5en-nccl-nvls
Sep 8, 2026
Merged

aarora79 merged 1 commit into
mainfrom
docs/p5en-nccl-nvls

Conversation

@aarora79

@aarora79 aarora79 commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Why

Serving any TP=8 model on a p5en.48xlarge failed at engine init, before any weights loaded:

RuntimeError: NCCL error: unhandled cuda error (run with NCCL_DEBUG=INFO for details)
RuntimeError: Engine core initialization failed. See root cause above.

Neither line names the real cause. Under NCCL_DEBUG=INFO it appears:

transport/nvls.cc:287 NCCL WARN Failed to bind NVLink SHARP (NVLS) Multicast memory
  of size 2097152 : CUDA error 401 'the operation cannot be performed in the present state'.
Disable NVLS (NCCL_NVLS_ENABLE=0) if you wish to avoid this error in the future.

The existing note covers three CUDA JIT failures on this node but not this one, which is the first thing a multi-GPU serve hits and the only one that fails before weight loading.

What changed

.claude/skills/vllm-setup/p5en-h200-cuda-fixes.md gains Fix 4 (NCCL_NVLS_ENABLE=0), a row in the quick failure reference, the export in the combined launch block, and a corrected Model -> which fixes line: Fix 4 applies to every multi-GPU model whatever its precision, because it fails before a model is loaded.

The section preamble is rewritten. It claimed startup fails "in two places" and described all of them as CUDA JIT problems; there are now four, and Fix 4 is neither a JIT nor a link-time failure.

Three things that cost time, now recorded

  • A clean fabric does not rule this out. nvidia-smi -q reported State: Completed / Status: Success on all 8 GPUs, nvidia-fabricmanager was active, and NVLink showed 26.562 GB/s per link. Section 0 was already satisfied. The multicast bind still failed.
  • The aws-ofi-nccl warnings alongside it are a red herring. NCCL_NET_PLUGIN=none does not fix it, and a single-node TP job never uses the network plugin.
  • It costs performance. Disabling NVLS drops in-network reduction, so TP=8 collectives are slower. Cost per task derived from wall clock on such a run is not comparable to a run with NVLS working, and a results doc built from one should say so.

The note also adds the bare 8-GPU torch.distributed all_reduce that reproduces the failure with no vLLM involved, which separates a node fault from a serving bug in about a minute.

Verification

Measured on a p5en.48xlarge: driver 595.91.07, NCCL 2.29.7, torch 2.13.0+cu130, vLLM 0.28.0.

  • Without the switch: every rank raises ncclUnhandledCudaError, both inside vLLM and in the bare all_reduce.
  • With it: NCCL ALL_REDUCE OK, world=8, sum=8.0, and deepseek-v3.2 serves at TP=8 with a 131072 window, through KV-cache init and CUDA graph capture.

Documentation only.

Serving any TP=8 model on a p5en.48xlarge failed at engine init with
"NCCL error: unhandled cuda error" and "Engine core initialization
failed", before any weights loaded. The real cause only appears under
NCCL_DEBUG=INFO: NCCL cannot bind NVLink SHARP (NVLS) multicast memory
and the driver returns CUDA 401, which NCCL treats as fatal.

NCCL names the switch itself. Setting NCCL_NVLS_ENABLE=0 takes a bare
8-GPU all_reduce from every rank failing to world=8.

Record it as Fix 4, and rewrite the section preamble, which claimed two
failure points and described them all as CUDA JIT problems. Fix 4 is
neither: it fails at distributed init rather than at kernel link time,
and it applies to every multi-GPU model regardless of precision.

Three things cost time here and are now written down:

- A clean fabric does not rule this out. Fabric Manager was active and
  reported Completed/Success on all 8 GPUs, with NVLink at 26.562 GB/s,
  while the multicast bind still failed.
- The aws-ofi-nccl warnings printed alongside it are unrelated;
  NCCL_NET_PLUGIN=none does not help, and a single-node TP job never
  uses the network plugin.
- Disabling NVLS drops in-network reduction, so TP=8 collectives are
  slower. Cost per task derived from wall clock on such a run is not
  comparable to a run with NVLS working, and results docs should say so.

Also add the bare torch.distributed all_reduce that reproduces it
without vLLM, which separates a node fault from a serving bug in about
a minute.
@aarora79 aarora79 added the documentation Improvements or additions to documentation label Sep 7, 2026
@aarora79
aarora79 merged commit 0a72281 into main Sep 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant