PyTorch: fix configuration of CUDA and cuDNN #9860
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
please test |
Contributor
|
A new Pull Request was created by @fwyzard for branch IB/CMSSW_15_1_X/master. @iarspider, @smuzaffar can you please review it and eventually sign? Thanks. |
Contributor
|
cms-bot internal usage |
Contributor
|
-1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d9ee6b/46108/summary.html External BuildI found compilation error when building: [1842/1852] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/bin/nvcc -forward-unknown-to-host-compiler -DAT_PER_OPERATOR_HEADERS -DFLASHATTENTION_DISABLE_ALIBI -DFMT_HEADER_ONLY=1 -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DPROTOBUF_USE_DLLS -DTORCH_CUDA_BUILD_MAIN_LIB -DTORCH_CUDA_USE_NVTX3 -DUSE_CUDA -DUSE_EXTERNAL_MZCRC -DUSE_FLASH_ATTENTION -DUSE_MEM_EFF_ATTENTION -D_FILE_OFFSET_BITS=64 -Dtorch_cuda_EXPORTS -DTORCH_ASSERT_NO_OPERATORS -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0 -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/onnx -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/third_party/onnx -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/nlohmann -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/THC -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/cuda -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/../../../third_party/cutlass/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/../../../third_party/cutlass/tools/util/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/caffe2/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/c10/cuda/../.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/c10/.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/torch/csrc/api -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/torch/csrc/api/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/protobuf/3.21.9-1126508a53768c90e66f6bf1821ac03a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/OpenBLAS/0.3.27-70a9dd2c9f309171934f13e3003b0540/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/ittapi/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/eigen/3bb6a48d8c171cf20b5f8e48bfb4e424fbd4f79e-5d91c922e771c0dc4f6bc00f61f3e2c5/include/eigen3 -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/INTERFACE -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/nlohmann/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/fmt/10.2.1-e35fd1db5eb3abc8ac0452e8ee427196/include -isystem /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02889/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cudnn/9.6.0.74-f14af0c4fad77bbae92ce5fee74a8d55/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/cmake/../third_party/cudnn_frontend/include -DLIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS -D_GLIBCXX_USE_CXX11_ABI=1 -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch -gencode arch=compute_60,code=sm_60 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_90,code=compute_90 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -O3 -DNDEBUG -std=c++17 -Xcompiler=-fPIC -Xcompiler -Wall -Wextra -Wdeprecated -Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds -Wno-unknown-pragmas -Wno-strict-overflow -Wno-strict-aliasing -Wunused-function -Wunused-variable -Wunused-but-set-variable -Wno-maybe-uninitialized -MD -MT caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassF_f32_aligned.cu.o -MF caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassF_f32_aligned.cu.o.d -x cu -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassF_f32_aligned.cu -o caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassF_f32_aligned.cu.o [1843/1852] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/bin/nvcc -forward-unknown-to-host-compiler -DAT_PER_OPERATOR_HEADERS -DFLASHATTENTION_DISABLE_ALIBI -DFMT_HEADER_ONLY=1 -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DPROTOBUF_USE_DLLS -DTORCH_CUDA_BUILD_MAIN_LIB -DTORCH_CUDA_USE_NVTX3 -DUSE_CUDA -DUSE_EXTERNAL_MZCRC -DUSE_FLASH_ATTENTION -DUSE_MEM_EFF_ATTENTION -D_FILE_OFFSET_BITS=64 -Dtorch_cuda_EXPORTS -DTORCH_ASSERT_NO_OPERATORS -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0 -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/onnx -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/third_party/onnx -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/nlohmann -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/THC -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/cuda -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/../../../third_party/cutlass/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/../../../third_party/cutlass/tools/util/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/caffe2/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/c10/cuda/../.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/c10/.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/torch/csrc/api -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/torch/csrc/api/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/protobuf/3.21.9-1126508a53768c90e66f6bf1821ac03a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/OpenBLAS/0.3.27-70a9dd2c9f309171934f13e3003b0540/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/ittapi/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/eigen/3bb6a48d8c171cf20b5f8e48bfb4e424fbd4f79e-5d91c922e771c0dc4f6bc00f61f3e2c5/include/eigen3 -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/INTERFACE -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/nlohmann/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/fmt/10.2.1-e35fd1db5eb3abc8ac0452e8ee427196/include -isystem /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02889/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cudnn/9.6.0.74-f14af0c4fad77bbae92ce5fee74a8d55/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/cmake/../third_party/cudnn_frontend/include -DLIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS -D_GLIBCXX_USE_CXX11_ABI=1 -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch -gencode arch=compute_60,code=sm_60 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_90,code=compute_90 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -O3 -DNDEBUG -std=c++17 -Xcompiler=-fPIC -Xcompiler -Wall -Wextra -Wdeprecated -Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds -Wno-unknown-pragmas -Wno-strict-overflow -Wno-strict-aliasing -Wunused-function -Wunused-variable -Wunused-but-set-variable -Wno-maybe-uninitialized -MD -MT caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/flash_attn/kernels/flash_fwd_split_hdim64_fp16_sm80.cu.o -MF caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/flash_attn/kernels/flash_fwd_split_hdim64_fp16_sm80.cu.o.d -x cu -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/native/transformers/cuda/flash_attn/kernels/flash_fwd_split_hdim64_fp16_sm80.cu -o caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/flash_attn/kernels/flash_fwd_split_hdim64_fp16_sm80.cu.o [1844/1852] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -DAT_PER_OPERATOR_HEADERS -DFLASHATTENTION_DISABLE_ALIBI -DFMT_HEADER_ONLY=1 -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DPROTOBUF_USE_DLLS -DTORCH_CUDA_BUILD_MAIN_LIB -DTORCH_CUDA_USE_NVTX3 -DUSE_CUDA -DUSE_EXTERNAL_MZCRC -DUSE_FLASH_ATTENTION -DUSE_MEM_EFF_ATTENTION -D_FILE_OFFSET_BITS=64 -Dtorch_cuda_EXPORTS -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0 -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/onnx -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/third_party/onnx -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/nlohmann -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/THC -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/cuda -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/../../../third_party/cutlass/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/../../../third_party/cutlass/tools/util/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/caffe2/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/c10/cuda/../.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/c10/.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/torch/csrc/api -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/torch/csrc/api/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/protobuf/3.21.9-1126508a53768c90e66f6bf1821ac03a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/OpenBLAS/0.3.27-70a9dd2c9f309171934f13e3003b0540/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/ittapi/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/eigen/3bb6a48d8c171cf20b5f8e48bfb4e424fbd4f79e-5d91c922e771c0dc4f6bc00f61f3e2c5/include/eigen3 -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/INTERFACE -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/nlohmann/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/fmt/10.2.1-e35fd1db5eb3abc8ac0452e8ee427196/include -isystem /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02889/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cudnn/9.6.0.74-f14af0c4fad77bbae92ce5fee74a8d55/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/cmake/../third_party/cudnn_frontend/include -DEIGEN_DONT_PARALLELIZE -DEIGEN_MAX_ALIGN_BYTES=64 -march=x86-64-v2 -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DNDEBUG -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow -DHAVE_AVX512_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -std=gnu++17 -fPIC -Wall -Wextra -Wdeprecated -Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds -Wno-unknown-pragmas -Wno-strict-overflow -Wno-strict-aliasing -Wunused-function -Wunused-variable -Wunused-but-set-variable -Wno-maybe-uninitialized -fvisibility=hidden -O2 -MD -MT caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/RegisterCUDA.cpp.o -MF caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/RegisterCUDA.cpp.o.d -o caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/RegisterCUDA.cpp.o -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/aten/src/ATen/RegisterCUDA.cpp [1845/1852] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/bin/nvcc -forward-unknown-to-host-compiler -DAT_PER_OPERATOR_HEADERS -DFLASHATTENTION_DISABLE_ALIBI -DFMT_HEADER_ONLY=1 -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DPROTOBUF_USE_DLLS -DTORCH_CUDA_BUILD_MAIN_LIB -DTORCH_CUDA_USE_NVTX3 -DUSE_CUDA -DUSE_EXTERNAL_MZCRC -DUSE_FLASH_ATTENTION -DUSE_MEM_EFF_ATTENTION -D_FILE_OFFSET_BITS=64 -Dtorch_cuda_EXPORTS -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0 -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/onnx -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/third_party/onnx -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/nlohmann -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/THC -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/cuda -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/../../../third_party/cutlass/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/../../../third_party/cutlass/tools/util/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/caffe2/aten/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/aten/src/ATen/.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/c10/cuda/../.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/c10/.. -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/torch/csrc/api -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/torch/csrc/api/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/protobuf/3.21.9-1126508a53768c90e66f6bf1821ac03a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/OpenBLAS/0.3.27-70a9dd2c9f309171934f13e3003b0540/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/ittapi/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/eigen/3bb6a48d8c171cf20b5f8e48bfb4e424fbd4f79e-5d91c922e771c0dc4f6bc00f61f3e2c5/include/eigen3 -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/INTERFACE -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/third_party/nlohmann/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/fmt/10.2.1-e35fd1db5eb3abc8ac0452e8ee427196/include -isystem /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02889/el8_amd64_gcc12/external/cuda/12.8.1-f1c01abd08373a07ceeffab8d5f1930a/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cudnn/9.6.0.74-f14af0c4fad77bbae92ce5fee74a8d55/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/pytorch-2.6.0/cmake/../third_party/cudnn_frontend/include -DLIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS -D_GLIBCXX_USE_CXX11_ABI=1 -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch -gencode arch=compute_60,code=sm_60 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_90,code=compute_90 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -O3 -DNDEBUG -std=c++17 -Xcompiler=-fPIC -Xcompiler -Wall -Wextra -Wdeprecated -Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds -Wno-unknown-pragmas -Wno-strict-overflow -Wno-strict-aliasing -Wunused-function -Wunused-variable -Wunused-but-set-variable -Wno-maybe-uninitialized -MD -MT caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/UfuncCUDA_add.cu.o -MF caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/UfuncCUDA_add.cu.o.d -x cu -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/pytorch_x86-64-v2/2.6.0-14ff12785445087bf4d9b2e688b0b1cf/build/aten/src/ATen/UfuncCUDA_add.cu -o caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/UfuncCUDA_add.cu.o ninja: build stopped: subcommand failed. error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.0K3DRo (%build) RPM build errors: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.0K3DRo (%build) |
fwyzard
force-pushed
the
IB/CMSSW_15_1_X/master_fix_pytorch
branch
from
7dabbc1 to
ff1373a
Compare
Contributor
Author
|
please test |
fwyzard
changed the title
Contributor
|
Pull request #9860 was updated. |
fwyzard
changed the title
fwyzard
force-pushed
the
IB/CMSSW_15_1_X/master_fix_pytorch
branch
from
ff1373a to
1e4fd12
Compare
Contributor
|
Pull request #9860 was updated. |
Contributor
Author
|
please test |
Contributor
|
-1 Failed Tests: Build The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: You can see more details here: BuildI found compilation error when building: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_15_1_X_2025-05-14-1100/external/el8_amd64_gcc12/lib/libtorch_cuda.so: undefined reference to `cudnnConvolutionBiasActivationForward' /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_15_1_X_2025-05-14-1100/external/el8_amd64_gcc12/lib/libtorch_cuda.so: undefined reference to `cudnnSpatialTfGridGeneratorBackward' /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_15_1_X_2025-05-14-1100/external/el8_amd64_gcc12/lib/libtorch_cuda.so: undefined reference to `cudnnBackendSetAttribute' /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_15_1_X_2025-05-14-1100/external/el8_amd64_gcc12/lib/libtorch_cuda.so: undefined reference to `cudnnSetConvolutionMathType' /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_15_1_X_2025-05-14-1100/external/el8_amd64_gcc12/lib/libtorch_cuda.so: undefined reference to `cudnnDestroySpatialTransformerDescriptor' collect2: error: ld returned 1 exit status >> Deleted: tmp/el8_amd64_gcc12/src/PhysicsTools/PyTorch/test/testTorch/testTorch gmake: *** [tmp/el8_amd64_gcc12/src/PhysicsTools/PyTorch/test/testTorch/testTorch] Error 1 >> Compiling src/PhysicsTools/PyTorch/test/testRunner.cc /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -c -DCMS_MICRO_ARCH='' -DGNU_GCC -D_GNU_SOURCE -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DBOOST_MPL_IGNORE_PARENTHESES_WARNING -DCMSSW_GIT_HASH='CMSSW_15_1_X_2025-05-14-1100' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_15_1_X_2025-05-14-1100' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02889/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-05-14-1100/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/pytorch/2.6.0-e04cb88bc71c326dd461b8844cd82ccc/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/pytorch/2.6.0-e04cb88bc71c326dd461b8844cd82ccc/include/torch/csrc/api/include -isystem/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/boost/1.80.0-cebec1e56ea7fc5fc916811ecd058739/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/cppunit/1.15.x-25a760f1303b0fca73df75b14e1358bc/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/protobuf/3.21.9-1126508a53768c90e66f6bf1821ac03a/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/zlib/1.2.13-d217cdbdd8d586e845e05946de2796be/include -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++20 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v3 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el8_amd64_gcc12/src/PhysicsTools/PyTorch/test/testTorchSimpleDnn/testRunner.cc.d src/PhysicsTools/PyTorch/test/testRunner.cc -o tmp/el8_amd64_gcc12/src/PhysicsTools/PyTorch/test/testTorchSimpleDnn/testRunner.cc.o >> Compiling src/PhysicsTools/PyTorch/test/testTorchSimpleDnn.cc |
fwyzard
force-pushed
the
IB/CMSSW_15_1_X/master_fix_pytorch
branch
from
1e4fd12 to
014c03d
Compare
Contributor
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d9ee6b/46134/summary.html The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: You can see more details here: Comparison SummarySummary:
|
smuzaffar
reviewed
May 15, 2025
| sed -i -e 's|CMAKE_CXX_STANDARD *14|CMAKE_CXX_STANDARD %{cms_cxx_standard}|' CMakeLists.txt | ||
|
|
||
| %build | ||
|
|
Contributor
There was a problem hiding this comment.
@fwyzard , thanks for this fix. yes this section should have been part of %build otherwise USE_CUDA is not set
Contributor
|
enable gpu |
Contributor
|
please test for el8_amd64_gcc14 |
Contributor
|
please test for el8_aarch64_gcc12 |
|
+1 |
Contributor
|
-1 Failed Tests: RelVals The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: You can see more details here: RelVals |
Contributor
|
+externals |
smuzaffar
merged commit 707ee32
into
cms-sw:IB/CMSSW_15_1_X/master
16 of 22 checks passed
cmsbuild
added
externals-approved
fully-signed
and removed
externals-pending
pending-signatures
labels
Contributor
|
This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_15_1_X/master IBs (tests are also fine). This pull request will be automatically merged. |
This was referenced May 15, 2025
Contributor
|
-1 Failed Tests: rocmUnitTests The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: ROCm Unit TestsI found 2 errors in the following unit tests: ---> test testRocmSoALayoutAndView_t had ERRORS ---> test alpakaTestBufferROCmAsync had ERRORS Comparison SummarySummary:
|
Contributor
Author
|
type bugfix |
This was referenced May 16, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix the configuration of CUDA and cuDNN in PyTorch and related tools.