Skip to content

Windows nvcc: Cannot cache (multiple input files) triggered by -Xcompiler "/openmp:llvm" #2726

Closed

Description

@gsakhnovsky-nvidia

Summary

When sccache wraps nvcc on Windows, passing -Xcompiler "/openmp:llvm" through to
the host MSVC compiler causes sccache to log
Cannot cache(multiple input files, [...]) and return exit code 1, instead of
caching or falling through to a direct compile. The cl.exe argument list shown
in the error contains exactly one source file (the x_0.cudafe1.cpp produced
by nvcc), so the "multiple input files" classification appears wrong.

Likely cause: sccache's MSVC argument parser splitting /openmp:llvm on : and
counting llvm as a second positional (input file) argument.

Environment

  • sccache: 0.15.0
  • OS: Windows Server Core LTSC2019 (inside a Docker container)
  • MSVC: 14.43.34808 (Visual Studio 2022 BuildTools)
  • nvcc: CUDA 13.0 (sm_75 used for repro; not arch-specific)
  • Host arch: x86_64

Minimal reproducer

echo __global__ void k() {} > hello.cu
echo int main() { k^<^<^<1,1^>^>^>(); return 0; } >> hello.cu

set SCCACHE_DIR=C:\Temp\sc
set SCCACHE_LOG=debug
sccache --stop-server >NUL 2>&1

sccache nvcc -forward-unknown-to-host-compiler ^
  -Xcompiler "/openmp:llvm" ^
  --generate-code arch=compute_75,code=sm_75 ^
  -std=c++17 -c hello.cu -o hello.obj

Exit code: 1. Removing -Xcompiler "/openmp:llvm" makes the same invocation
succeed (exit 0) and cache.

Verbatim error

  "C:\\BuildTools\\VC\\Tools\\MSVC\\14.43.34808\\bin\\HostX64\\x64\\cl.exe"
  ["/std:c++17", "-Fohello.obj", "-D__CUDA_ARCH__=750", "-D__CUDA_ARCH_LIST__=750",
   "-nologo", "-c", "-TP", "-DCUDA_DOUBLE_MATH_FUNCTIONS", "-EHsc", "-MT",
   "/openmp:llvm", "-Ic:/tools/cuda/...", ...,
   "C:/Users/.../sccache_nvccXXXXX\\x_0.cudafe1.cpp"]

The bracketed file list contains exactly one source file.

Bisection

Four -Xcompiler flag combos against the same hello-world .cu:

  ┌──────┬──────────────────────────────┬──────┬──────────────────────┐
  │ Case │       -Xcompiler flags       │ Exit │ Cannot cache emitted │
  ├──────┼──────────────────────────────┼──────┼──────────────────────┤
  │ 1    │ -MT, "/openmp:llvm", /bigobj │ 1    │ yes                  │
  ├──────┼──────────────────────────────┼──────┼──────────────────────┤
  │ 2    │ -MT, "/openmp:llvm"          │ 1    │ yes                  │
  ├──────┼──────────────────────────────┼──────┼──────────────────────┤
  │ 3    │ -MT, /bigobj                 │ 0    │ no                   │
  ├──────┼──────────────────────────────┼──────┼──────────────────────┤
  │ 4    │ -MT only (control)           │ 0    │ no                   │
  └──────┴──────────────────────────────┴──────┴──────────────────────┘

/openmp:llvm alone is sufficient to trigger; /bigobj is innocent.

Hypothesis

The MSVC argument parser appears to split /openmp:llvm on ":" and treat llvm
as a positional argument.

Impact

CUDA projects on Windows that pass -Xcompiler /openmp:llvm cannot be built
through sccache today. Not sure whether Linux is impacted.

Activity

  1. gsakhnovsky-nvidia commented on Jun 15, 2026

    @gsakhnovsky-nvidia
    Author

    For anyone hitting this: the rapidsai/sccache fork fixes it in 2b02f81 (released in v0.14.0-rapids.17). One-line summary: the MSVC arg parser in src/compiler/msvc.rs only had a special case for /openmp:experimental — the fix generalizes it to any /openmp:, including llvm.

    I've moved to the fork for my use case, but leaving this open since the bug still reproduces against upstream. The fork's commit isn't cross-linked back here, so flagging the connection for whoever picks this up.

  2. AJIOB commented on Aug 29, 2026

    @AJIOB
    Contributor

    I've tried to backport some MSVC flags support from rapidsai/sccache in #2831 - it should help you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions