Skip to content

Nano Comparison: Keep WF full name and forward it to .json files #2487

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
cmsbuild merged 1 commit into from
May 6, 2025
Merged

Nano Comparison: Keep WF full name and forward it to .json files #2487

cmsbuild merged 1 commit into from
May 6, 2025

Conversation

@ftorrresd
Copy link
Contributor

@ftorrresd ftorrresd commented Apr 23, 2025

Currently the nano comparison report will shows only the WF number. The goal is to have the the full WF name available.

The table:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.238 2.595 2.501 0.094 ( +3.8% ) 13.76 13.71 +0.4% 2.751 2.745

Will become:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.238_ScoutingNANOwithPromptdata140Xrun3 2.595 2.501 0.094 ( +3.8% ) 13.76 13.71 +0.4% 2.751 2.745

Since I don´t know how to test this proposal in this repo, I'm placing this PR and hoping to test during review.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 23, 2025

A new Pull Request was created by @ftorrresd for branch master.

@cmsbuild, @iarspider, @smuzaffar can you please review it and eventually sign? Thanks.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 23, 2025 •
edited
Loading

cms-bot internal usage

@ftorrresd
Copy link
Contributor Author

ftorrresd commented Apr 23, 2025

enable nano

(do I have rights in this repo?)

@ftorrresd
Copy link
Contributor Author

ftorrresd commented Apr 23, 2025

please test with cms-sw/cmssw#47934

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 24, 2025

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7a0733/45691/summary.html
COMMIT: 9d73e92
CMSSW: CMSSW_15_1_X_2025-04-23-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/2487/45691/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 1 lines to the logs
  • Reco comparison results: 12 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3908633
  • DQMHistoTests: Total failures: 85
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3908528
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially added 1822 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 23
  • DQMHistoTests: Total histograms compared: 90509
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 90509
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 22 files compared)
  • Checked 113 log files, 65 edm output root files, 23 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.001_NANOmc106Xul16v2 3.115 3.115 0.000 ( +0.0% ) 6.71 6.32 +6.1% 2.638 2.631
2500.002_NANOmc106Xul17v2 3.231 3.231 0.000 ( +0.0% ) 5.95 5.67 +4.9% 3.109 3.099
2500.003_NANOmc106Xul18v2 3.172 3.172 0.000 ( +0.0% ) 6.09 5.73 +6.3% 3.098 3.082
2500.011_NANOdata106Xul16v2 1.647 1.647 0.000 ( +0.0% ) 11.34 10.52 +7.8% 2.745 2.743
2500.012_NANOdata106Xul17v2 2.185 2.185 0.000 ( +0.0% ) 6.28 6.00 +4.7% 2.932 2.921
2500.013_NANOdata106Xul18v2 2.002 2.002 0.000 ( +0.0% ) 8.91 8.38 +6.4% 2.847 2.843
2500.021_NANOmcUL16APVreMINI 0.022 0.022 0.000 ( +0.0% ) 1.85 1.45 +27.6% 2.984 2.976
2500.022_NANOmcUL16reMINI 0.022 0.022 0.000 ( +0.0% ) 1.80 1.58 +13.6% 2.983 2.975
2500.023_NANOmcUL17reMINI 0.022 0.022 0.000 ( +0.0% ) 1.58 1.21 +30.6% 2.694 2.702
2500.024_NANOmcUL18reMINI 0.022 0.022 0.000 ( +0.0% ) 1.42 1.27 +11.7% 3.105 3.106
2500.031_NANOdataUL16APVreMINI 0.035 0.035 0.000 ( +0.0% ) 1.59 1.38 +15.1% 2.925 2.939
2500.032_NANOdataUL16reMINI 0.036 0.036 0.000 ( +0.0% ) 1.58 1.35 +17.1% 2.924 2.923
2500.033_NANOdataUL17reMINI 0.037 0.037 0.000 ( +0.0% ) 1.55 1.37 +13.0% 2.982 2.974
2500.034_NANOdataUL18reMINI 0.036 0.036 0.000 ( +0.0% ) 1.55 1.20 +28.9% 2.954 2.955
2500.101_NANOmc130X 2.872 2.872 0.000 ( +0.0% ) 15.29 14.08 +8.6% 2.736 2.736
2500.111_NANOdata130Xrun3 1.474 1.474 0.000 ( +0.0% ) 29.00 25.46 +13.9% 2.435 2.443
2500.112_NANOdata130Xrun3 1.896 1.896 0.000 ( +0.0% ) 24.14 22.60 +6.8% 2.518 2.515
2500.131_ScoutingNANOdata130Xrun3 0.771 0.758 0.014 ( +1.8% ) 33.36 31.98 +4.3% 1.695 1.692
2500.201_NANOmc140X 2.706 2.706 0.000 ( +0.0% ) 13.59 12.69 +7.1% 2.276 2.271
2500.211_NANOdata140Xrun3 1.845 1.845 0.000 ( +0.0% ) 25.32 23.63 +7.2% 2.539 2.541
2500.212_NANOdata140Xrun3 2.243 2.243 0.000 ( +0.0% ) 21.59 19.84 +8.8% 2.648 2.642
2500.221_muPOGNANOmc140X 2.141 2.141 0.000 ( +0.0% ) 13.42 12.57 +6.7% 2.135 2.131
2500.222_EGMNANOmc140X 3.516 3.516 0.000 ( +0.0% ) 12.44 11.90 +4.6% 2.225 2.224
2500.223_BTVNANOmc140X 10.328 10.328 0.000 ( +0.0% ) 4.58 4.30 +6.4% 2.393 2.393
2500.224_jmeNANOmc140X 6.622 6.622 0.000 ( +0.0% ) 1.24 1.18 +4.4% 2.357 2.360
2500.225_jmeNANOrePuppimc140X 6.671 6.671 0.000 ( +0.0% ) 1.16 1.10 +5.1% 2.586 2.571
2500.226_lepTrackInfoNANOmc140X 3.210 3.210 0.000 ( +0.0% ) 13.22 12.58 +5.1% 2.218 2.223
2500.227_ScoutingNANOmc140X 1.486 1.463 0.023 ( +1.6% ) 15.79 15.02 +5.2% 1.961 1.962
2500.228_ScoutingNANOwithPromptmc140X 4.105 4.075 0.030 ( +0.7% ) 8.36 8.00 +4.5% 2.512 2.509
2500.231_muPOGNANO140Xrun3 1.516 1.516 0.000 ( +0.0% ) 21.30 20.09 +6.0% 2.460 2.463
2500.232_EGMNANOdata140Xrun3 2.502 2.502 0.000 ( +0.0% ) 19.89 18.95 +5.0% 2.545 2.544
2500.233_BTVNANOdata140Xrun3 5.422 5.422 0.000 ( +0.0% ) 6.89 6.55 +5.3% 2.711 2.706
2500.234_jmeNANOdata140Xrun3 3.928 3.928 0.000 ( +0.0% ) 1.58 1.50 +5.1% 2.650 2.650
2500.235_jmeNANOrePuppidata140Xrun3 3.960 3.960 0.000 ( +0.0% ) 1.47 1.43 +2.8% 2.860 2.859
2500.236_lepTrackInfoNANOdata140Xrun3 2.292 2.292 0.000 ( +0.0% ) 21.16 20.71 +2.2% 2.545 2.540
2500.237_ScoutingNANOdata140Xrun3 1.101 1.030 0.071 ( +6.9% ) 29.49 28.09 +5.0% 1.690 1.673
2500.238_ScoutingNANOwithPromptdata140Xrun3 2.595 2.501 0.094 ( +3.8% ) 14.48 13.71 +5.6% 2.748 2.745
2500.241_l1DPGNANO140Xrun3 9.404 9.404 0.000 ( +0.0% ) 6.68 6.74 -0.9% 1.970 1.961
2500.242_muDPGNANO140Xrun3 10.331 10.331 0.000 ( +0.0% ) 1.67 1.54 +8.2% 1.758 1.760
2500.243_muDPGNANOBkg140Xrun3 2.712 2.712 0.000 ( +0.0% ) 15.50 14.28 +8.6% 1.096 1.095
2500.244_hcalDPGNANO140Xrun3 486.016 486.016 0.000 ( +0.0% ) 1.14 1.12 +1.7% 1.733 1.725
2500.245_hcalDPGCalibNANO140Xrun3 826.413 826.413 0.000 ( +0.0% ) 1.52 1.51 +1.0% 1.723 1.704
2500.251_hcalDPGMCNANO140Xrun3 645.333 645.333 0.000 ( +0.0% ) 1.66 1.61 +3.1% 1.821 1.816
2500.301_NANOmc2024reMINI 0.021 0.021 0.000 ( +0.0% ) 1.72 1.63 +5.9% 3.150 3.128
2500.311_NANOdata2024reMINI 0.036 0.036 0.000 ( +0.0% ) 1.65 1.58 +4.6% 2.926 2.927
2500.901_TTbarMINIAOD10.6_UL18v2 1.819 1.819 0.000 ( +0.0% ) 32.22 32.34 -0.4% 1.906 1.917
2500.902_TTbarMINIAOD14.0 1.665 1.665 0.000 ( +0.0% ) 32.20 31.65 +1.7% 1.499 1.496
2500.911_DYToLL_M-50_13TeV_pythia8 14.345 14.345 0.000 ( +0.0% ) 8.43 7.27 +16.0% 1.123 1.124
2500.912_DYToll01234Jets_5f_LO_MLM_Madgraph_LHE_13TeV 0.310 0.240 0.070 ( +29.3% ) 2.10 2.81 -25.3% 0.880 0.880
2500.913_TTbar_Pow_LHE_13TeV 0.110 0.110 0.000 ( +0.0% ) 2.63 2.50 +5.0% 0.883 0.882

@ftorrresd ftorrresd mentioned this pull request Apr 24, 2025
Merged
@ftorrresd
Copy link
Contributor Author

ftorrresd commented Apr 24, 2025

please test with cms-sw/cmssw#47943

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 24, 2025

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7a0733/45706/summary.html
COMMIT: 9d73e92
CMSSW: CMSSW_15_1_X_2025-04-24-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/2487/45706/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 2 lines to the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3913297
  • DQMHistoTests: Total failures: 11
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3913266
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially removed 319 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 23
  • DQMHistoTests: Total histograms compared: 90509
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 90509
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 22 files compared)
  • Checked 113 log files, 65 edm output root files, 23 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

workflow id kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
NANOmc106Xul16v2 2500.001 3.115 3.115 0.000 ( +0.0% ) 5.76 5.58 +3.2% 2.644 2.643
NANOmc106Xul17v2 2500.002 3.231 3.231 0.000 ( +0.0% ) 5.27 4.95 +6.6% 3.097 3.090
NANOmc106Xul18v2 2500.003 3.172 3.172 0.000 ( +0.0% ) 5.42 5.15 +5.3% 3.097 3.088
NANOdata106Xul16v2 2500.011 1.647 1.647 0.000 ( +0.0% ) 9.42 8.93 +5.6% 2.735 2.718
NANOdata106Xul17v2 2500.012 2.185 2.185 0.000 ( +0.0% ) 5.62 5.28 +6.3% 2.927 2.922
NANOdata106Xul18v2 2500.013 2.002 2.002 0.000 ( +0.0% ) 7.48 7.04 +6.2% 2.855 2.843
NANOmcUL16APVreMINI 2500.021 0.022 0.022 0.000 ( +0.0% ) 1.40 1.40 +0.0% 2.977 2.951
NANOmcUL16reMINI 2500.022 0.022 0.022 0.000 ( +0.0% ) 1.44 1.40 +2.5% 2.990 2.788
NANOmcUL17reMINI 2500.023 0.022 0.022 0.000 ( +0.0% ) 1.22 1.14 +7.0% 2.705 2.663
NANOmcUL18reMINI 2500.024 0.022 0.022 0.000 ( +0.0% ) 1.14 1.05 +8.1% 3.091 3.064
NANOdataUL16APVreMINI 2500.031 0.035 0.035 0.000 ( +0.0% ) 1.32 1.26 +5.3% 2.928 2.895
NANOdataUL16reMINI 2500.032 0.036 0.036 0.000 ( +0.0% ) 1.27 1.21 +4.6% 2.929 2.898
NANOdataUL17reMINI 2500.033 0.037 0.037 0.000 ( +0.0% ) 1.28 1.19 +7.0% 2.971 2.935
NANOdataUL18reMINI 2500.034 0.036 0.036 0.000 ( +0.0% ) 1.35 1.16 +15.9% 2.948 2.918
NANOmc130X 2500.101 2.872 2.872 0.000 ( +0.0% ) 12.55 11.32 +10.8% 2.749 2.703
NANOdata130Xrun3 2500.111 1.474 1.474 0.000 ( +0.0% ) 23.72 22.28 +6.5% 2.442 2.439
NANOdata130Xrun3 2500.112 1.896 1.896 0.000 ( +0.0% ) 20.92 19.61 +6.7% 2.523 2.510
ScoutingNANOdata130Xrun3 2500.131 0.758 0.758 0.000 ( +0.0% ) 27.93 25.46 +9.7% 1.689 1.669
NANOmc140X 2500.201 2.706 2.706 0.000 ( +0.0% ) 10.69 10.08 +6.1% 2.272 2.218
NANOdata140Xrun3 2500.211 1.845 1.845 0.000 ( +0.0% ) 20.45 18.73 +9.2% 2.540 2.545
NANOdata140Xrun3 2500.212 2.243 2.243 0.000 ( +0.0% ) 17.47 16.72 +4.5% 2.635 2.639
muPOGNANOmc140X 2500.221 2.141 2.141 0.000 ( +0.0% ) 10.60 10.96 -3.3% 2.130 2.133
EGMNANOmc140X 2500.222 3.516 3.516 0.000 ( +0.0% ) 10.60 10.14 +4.5% 2.225 2.231
BTVNANOmc140X 2500.223 10.328 10.328 0.000 ( +0.0% ) 3.60 3.28 +9.7% 2.392 2.370
jmeNANOmc140X 2500.224 6.622 6.622 0.000 ( +0.0% ) 1.02 0.96 +7.1% 2.367 2.348
jmeNANOrePuppimc140X 2500.225 6.671 6.671 0.000 ( +0.0% ) 0.97 0.92 +5.3% 2.585 2.542
lepTrackInfoNANOmc140X 2500.226 3.210 3.210 0.000 ( +0.0% ) 10.37 8.14 +27.3% 2.224 2.202
ScoutingNANOmc140X 2500.227 1.463 1.463 0.000 ( +0.0% ) 12.95 12.22 +6.0% 1.968 1.857
ScoutingNANOwithPromptmc140X 2500.228 4.075 4.075 0.000 ( +0.0% ) 7.23 6.91 +4.6% 2.513 2.513
muPOGNANO140Xrun3 2500.231 1.516 1.516 0.000 ( +0.0% ) 17.21 17.29 -0.5% 2.461 2.463
EGMNANOdata140Xrun3 2500.232 2.502 2.502 0.000 ( +0.0% ) 14.78 14.17 +4.3% 2.543 2.545
BTVNANOdata140Xrun3 2500.233 5.422 5.422 0.000 ( +0.0% ) 5.48 5.29 +3.6% 2.703 2.704
jmeNANOdata140Xrun3 2500.234 3.928 3.928 0.000 ( +0.0% ) 1.30 1.27 +2.4% 2.656 2.657
jmeNANOrePuppidata140Xrun3 2500.235 3.960 3.960 0.000 ( +0.0% ) 1.29 1.25 +3.7% 2.871 2.862
lepTrackInfoNANOdata140Xrun3 2500.236 2.292 2.292 0.000 ( +0.0% ) 16.10 16.51 -2.5% 2.541 2.536
ScoutingNANOdata140Xrun3 2500.237 1.030 1.030 0.000 ( +0.0% ) 25.32 26.12 -3.0% 1.674 1.676
ScoutingNANOwithPromptdata140Xrun3 2500.238 2.501 2.501 0.000 ( +0.0% ) 12.84 12.78 +0.5% 2.759 2.755
l1DPGNANO140Xrun3 2500.241 9.404 9.404 0.000 ( +0.0% ) 4.78 5.48 -12.7% 1.966 1.959
muDPGNANO140Xrun3 2500.242 10.331 10.331 0.000 ( +0.0% ) 1.28 1.47 -13.3% 1.760 1.758
muDPGNANOBkg140Xrun3 2500.243 2.712 2.712 0.000 ( +0.0% ) 12.17 14.10 -13.7% 1.096 1.095
hcalDPGNANO140Xrun3 2500.244 486.016 486.016 0.000 ( +0.0% ) 0.88 0.95 -7.5% 1.714 1.724
hcalDPGCalibNANO140Xrun3 2500.245 826.413 826.413 0.000 ( +0.0% ) 1.19 1.37 -12.7% 1.733 1.709
hcalDPGMCNANO140Xrun3 2500.251 645.333 645.333 0.000 ( +0.0% ) 1.28 1.39 -7.7% 1.817 1.818
NANOmc2024reMINI 2500.301 0.021 0.021 0.000 ( +0.0% ) 1.52 1.37 +10.8% 3.144 3.130
NANOdata2024reMINI 2500.311 0.036 0.036 0.000 ( +0.0% ) 1.50 1.29 +16.5% 2.921 2.916
TTbarMINIAOD10.6_UL18v2 2500.901 1.819 1.819 0.000 ( +0.0% ) 23.03 24.93 -7.6% 1.916 1.915
TTbarMINIAOD14.0 2500.902 1.665 1.665 0.000 ( +0.0% ) 27.90 24.06 +16.0% 1.497 1.496
DYToLL_M-50_13TeV_pythia8 2500.911 14.345 14.345 0.000 ( +0.0% ) 5.90 5.29 +11.6% 1.127 1.125
DYToll01234Jets_5f_LO_MLM_Madgraph_LHE_13TeV 2500.912 0.729 0.240 0.490 ( +204.0% ) 2.10 1.95 +7.9% 0.881 0.880
TTbar_Pow_LHE_13TeV 2500.913 0.110 0.110 0.000 ( +0.0% ) 2.52 2.15 +16.9% 0.882 0.882

@ftorrresd ftorrresd mentioned this pull request Apr 24, 2025
Merged
@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 28, 2025

REMINDER @rappoccio, @antoniovilela, @sextonkennedy, @mandrenguyen: This PR was tested with cms-sw/cmssw#47943, please check if they should be merged together

@mandrenguyen
Copy link
Contributor

mandrenguyen commented Apr 30, 2025

+1
@cms-sw/externals-l2 I let you merge this when ready

@smuzaffar
Copy link
Contributor

smuzaffar commented May 6, 2025

+externals
@ftorrresd , is this ready to go in? and I hope this should work for all release cycles/IBs ( with or without cms-sw/cmssw#47943 change) ... right?

@cmsbuild
Copy link
Contributor

cmsbuild commented May 6, 2025

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.
Notice This PR was tested with additional Pull Request(s), please also merge them if necessary: cms-sw/cmssw#47943

@cmsbuild cmsbuild merged commit 8200c97 into cms-sw:master May 6, 2025
13 checks passed
@ftorrresd
Copy link
Contributor Author

ftorrresd commented May 6, 2025

+externals @ftorrresd , is this ready to go in? and I hope this should work for all release cycles/IBs ( with or without cms-sw/cmssw#47943 change) ... right?

Its is ready. Should also work without cms-sw/cmssw#47943 .

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

externals-approved fully-signed orp-approved requires-external tests-approved

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ftorrresd @cmsbuild @mandrenguyen @smuzaffar