Skip to content

Conversation

@davidlange6
Copy link
Contributor

These proposed changes are to move towards a more standardized llvm build.

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @davidlange6 (David Lange) for branch IB/CMSSW_12_6_X/master.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

@davidlange6
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e82419/27174/summary.html
COMMIT: 3e72ee8
CMSSW: CMSSW_12_6_X_2022-08-29-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8050/27174/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e82419/27174/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e82419/27174/git-merge-result

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 3695708
  • DQMHistoTests: Total failures: 25
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 3695660
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 1476.3739999999998 KiB( 50 files compared)
  • DQMHistoSizes: changed ( 11634.0,... ): 147.637 KiB L1T/L1TStage2uGT
  • DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
  • Checked 212 log files, 49 edm output root files, 51 DQM output files
  • TriggerResults: no differences found

@smuzaffar
Copy link
Contributor

test parameters:

  • full_cmssw = true

@smuzaffar
Copy link
Contributor

please test for CMSSW_12_6_CLANG_X

# Avoid dependency on /usr/bin/python, Darwin + Xcode specific
rm -f %{i}/bin/set-xcode-analyzer
# Remove all static libs except libomptarget-nvptx.a
find %{i}/lib64 -name "*.a" -and -not -name "libomptarget-*.a" -delete
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidlange6 , note that this adds additional 900MB to distribution

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e82419/27197/summary.html
COMMIT: 3e72ee8
CMSSW: CMSSW_12_6_CLANG_X_2022-08-29-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8050/27197/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e82419/27197/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e82419/27197/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test EcalTPG_updateWeightIdMap_test had ERRORS
---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beamhlt_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS
and more ...

Comparison Summary

Summary:

  • You potentially added 88190 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 24951 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 3595258
  • DQMHistoTests: Total failures: 107299
  • DQMHistoTests: Total nulls: 2
  • DQMHistoTests: Total successes: 3487935
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -80769.815 KiB( 50 files compared)
  • DQMHistoSizes: changed ( 23234.0,... ): -10466.235 KiB HGCAL/HGCalValidator
  • DQMHistoSizes: changed ( 39434.0,... ): -9874.222 KiB HGCAL/HGCalValidator
  • Checked 212 log files, 49 edm output root files, 51 DQM output files
  • TriggerResults: found differences in 1 / 50 workflows

@smuzaffar smuzaffar merged commit c0e60d8 into cms-sw:IB/CMSSW_12_6_X/master Sep 1, 2022
@smuzaffar
Copy link
Contributor

smuzaffar commented Sep 1, 2022

+externals

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 1, 2022

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_12_6_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)


cmake %{_builddir}/llvm-%{realversion}-%{llvmCommit}/llvm \
-G Ninja \
-DGCC_INSTALL_PREFIX="${GCC_ROOT}" \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fwyzard , looks like building llvm with GCC_INSTALL_PREFIX fixed the llvm not finding our gcc header issue. After this change now I see that clang++ shows the correct gcc paths

>  clang++ -E -x c++ - -v
clang version 14.0.6
Target: x86_64-redhat-linux-gnu
Thread model: posix
InstalledDir: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/bin
Found candidate GCC installation: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0
Selected GCC installation: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
Found CUDA installation: /cvmfs/cms-ib.cern.ch/nweek-02748/el8_amd64_gcc10/external/cuda/11.5.2-c927b7e765e06433950d8a7eab9eddb4, version 11.5
 (in-process)
 "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/bin/clang-14" -cc1 -triple x86_64-redhat-linux-gnu -E -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -v -fcoverage-compilation-dir=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_6_CLANG_X_2022-08-31-2300 -resource-dir /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/lib64/clang/14.0.6 -internal-isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0/../../../../include/c++/10.3.0 -internal-isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0/../../../../include/c++/10.3.0/x86_64-redhat-linux-gnu -internal-isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0/../../../../include/c++/10.3.0/backward -internal-isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/lib64/clang/14.0.6/include -internal-isystem /usr/local/include -internal-isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0/../../../../x86_64-redhat-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_6_CLANG_X_2022-08-31-2300 -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o - -x c++ -
clang -cc1 version 14.0.6 based upon LLVM 14.0.6 default target x86_64-redhat-linux-gnu
ignoring nonexistent directory "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0/../../../../x86_64-redhat-linux-gnu/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0/../../../../include/c++/10.3.0
 /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0/../../../../include/c++/10.3.0/x86_64-redhat-linux-gnu
 /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/lib/gcc/x86_64-redhat-linux-gnu/10.3.0/../../../../include/c++/10.3.0/backward
 /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/lib64/clang/14.0.6/include
 /usr/local/include
 /usr/include
End of search list.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still need to see if it works properly when it is installed on cvmfs

@davidlange6
Copy link
Contributor Author

davidlange6 commented Sep 1, 2022 via email

@smuzaffar
Copy link
Contributor

humm, not really. @davidlange6 it works in the build area ( where gcc install path and GCC_INSTALL_PREFIX are same) but once things are installed on cvmfs then clang does not find our gcc ( specially in llvm 14)

Singularity> which clang++
/cvmfs/cms-ci.cern.ch/week0/PR_334f39cf/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/bin/clang++

Singularity> which gcc
/cvmfs/cms-ci.cern.ch/week0/PR_334f39cf/el8_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/bin/gcc

Singularity> echo '' | clang++ -E -x c++ - -v
clang version 14.0.6
Target: x86_64-redhat-linux-gnu
Thread model: posix
InstalledDir: /cvmfs/cms-ci.cern.ch/week0/PR_334f39cf/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/bin
Found CUDA installation: /cvmfs/cms-ib.cern.ch/nweek-02748/el8_amd64_gcc10/external/cuda/11.5.2-c927b7e765e06433950d8a7eab9eddb4, version 11.5
 (in-process)
 "/cvmfs/cms-ci.cern.ch/week0/PR_334f39cf/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/bin/clang-14" -cc1 -triple x86_64-redhat-linux-gnu -E -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -v -fcoverage-compilation-dir=/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7729/27252/CMSSW_12_6_CLANG_X_2022-08-31-2300 -resource-dir /cvmfs/cms-ci.cern.ch/week0/PR_334f39cf/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/lib64/clang/14.0.6 -internal-isystem /cvmfs/cms-ci.cern.ch/week0/PR_334f39cf/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/lib64/clang/14.0.6/include -internal-isystem /usr/local/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7729/27252/CMSSW_12_6_CLANG_X_2022-08-31-2300 -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o - -x c++ -
clang -cc1 version 14.0.6 based upon LLVM 14.0.6 default target x86_64-redhat-linux-gnu
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /cvmfs/cms-ci.cern.ch/week0/PR_334f39cf/el8_amd64_gcc10/external/llvm/14.0.6-7e877bac6aa04ac511ab1c515ec417ab/lib64/clang/14.0.6/include
 /usr/local/include
 /usr/include
End of search list.

@davidlange6
Copy link
Contributor Author

davidlange6 commented Sep 1, 2022 via email

@smuzaffar
Copy link
Contributor

looks like llvm 13 and above ignore the COMPILER_PATH env see llvm/llvm-project#51351

@smuzaffar
Copy link
Contributor

we can either patch llvm and add back SmallVector<std::string, 8> Prefixes(D.PrefixDirs.begin(),D.PrefixDirs.end()); or start using --gcc-toolchain=$GCC_ROOT

@fwyzard
Copy link
Contributor

fwyzard commented Sep 1, 2022

For the external version of llvm/clang coming with HIP, I'm passing --gcc-toolchain=$COMPILER_PATH explicitly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants