Skip to content

Conversation

@alintulu
Copy link
Contributor

@alintulu alintulu commented Aug 7, 2023

PR description:

This PR is to address the concern brought up in #40438 (comment). It creates a new module for gtStage2Digis, as well as every module needed to update the configuration given the initial change. The new modules are named after the original, with a suffix of Scouting (e.g. gtStage2Digis -> gtStage2DigisScouting), however please let me know if there is a more appropriate naming strategy.

The only difference between this and PR #40438, are the changes described above made to PhysicsTools/NanoAOD/python/custom_run3scouting_cff.py.

PR validation:

The PR passed the runTheMatrix tests affiliated with #40438. The Cannot unpack: no FEDRawDataCollection found messages related to gmtStage2Digis and caloStage2Digis are gone, however No HLT information produced is still present (an example displayed below).

Begin processing the 1st record. Run 357735, Event 501441569, LumiSection 300 on stream 0 at 07-Aug-2023 09:18:58.116 CEST
%MSG-e triggerEventValid:  PATTriggerProducer:patTriggerScouting  07-Aug-2023 09:18:58 CEST Run: 357735 Event: 501441569
trigger::TriggerEvent product with InputTag 'hltTriggerSummaryAOD::HLT' not in event
No HLT information produced

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2023

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42484/36490

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2023

A new Pull Request was created by @alintulu (Adelina Lintuluoto) for master.

It involves the following packages:

  • CommonTools/RecoAlgos (reconstruction)
  • Configuration/PyReleaseValidation (pdmv, upgrade)
  • PhysicsTools/NanoAOD (xpog)
  • PhysicsTools/Scouting (xpog)
  • RecoBTag/FeatureTools (reconstruction)
  • RecoBTag/ONNXRuntime (reconstruction)

@vlimant, @bbilin, @clacaputo, @cmsbuild, @AdrianoDee, @srimanob, @simonepigazzini, @sunilUIET, @mandrenguyen can you please review it and eventually sign? Thanks.
@AlexDeMoor, @rappoccio, @jdolen, @missirol, @makortel, @AnnikaStein, @abbiendi, @emilbols, @jhgoh, @Martin-Grunewald, @JyothsnaKomaragiri, @gkasieczka, @hqucms, @hatakeyamak, @gpetruc, @ahinzmann, @andrzejnovak, @demuller, @fabiocos, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Copy link
Contributor

@perrotta perrotta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alintulu this PR is identical to previous #40438 but the fixes applied to PhysicsTools/NanoAOD/python/custom_run3scouting_cff.py

Making a new PR will loose all the discussions you had in that thread (181 entries since January!), or at least it will make them not so easy to retrieve.

I think that for documentation purposes it would be better to modify the original PR and finally merge it, together with its whole story.

Regardless of that, the updates implemented here look correct, at the first sight. Let test them here; but as I said it would be preferable to update and finally merge the original PR.

'--datatier':'NANOAOD',
'--eventcontent':'NANOAOD'}])

steps['NANO_mcRun3ScoutingPF13.X']=merge([{'-s':'NANO:PhysicsTools/NanoAOD/custom_run3scouting_cff.nanoSequenceMC',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copying here the comment of @vlimant in the original PR (that could even had been addressed here, since you are making a new PR):

I believe that the .nanoSequenceMC is not necessary, since --mc will have cmsDriver pick that one up automatically. We'll deal with this at a later stage during the NANO:@Scouting integration

@perrotta
Copy link
Contributor

perrotta commented Aug 7, 2023

enable nano

@perrotta
Copy link
Contributor

perrotta commented Aug 7, 2023

please test

@alintulu
Copy link
Contributor Author

alintulu commented Aug 7, 2023

@alintulu this PR is identical to previous #40438 but the fixes applied to PhysicsTools/NanoAOD/python/custom_run3scouting_cff.py

Making a new PR will loose all the discussions you had in that thread (181 entries since January!), or at least it will make them not so easy to retrieve.

I think that for documentation purposes it would be better to modify the original PR and finally merge it, together with its whole story.

Regardless of that, the updates implemented here look correct, at the first sight. Let test them here; but as I said it would be preferable to update and finally merge the original PR.

@perrotta Sorry I misunderstood you, I thought you preferred a new PR. I can of course add the changes to the original, however in order to avoid recollecting all the signatures on the last PR, maybe the following could be an option?

  1. Validate this PR
  2. Merge Custom Run 3 PFScouting NanoAOD #40438
  3. Merge this PR (which at that point would only include the change to PhysicsTools/NanoAOD/python/custom_run3scouting_cff.py)

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2023

-1

Failed Tests: RelVals-INPUT RelVals-NANO
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0da083/34117/summary.html
COMMIT: c16b2bf
CMSSW: CMSSW_13_3_X_2023-08-06-2300/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42484/34117/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

  • 2500.52500.5_ScoutingNanodata13X/step2_ScoutingNanodata13X.log

RelVals-NANO

  • 2500.512500.51_ScoutingNanomc13X/step1_ScoutingNanomc13X.log
  • 2500.52500.5_ScoutingNanodata13X/step2_ScoutingNanodata13X.log

Comparison Summary

Summary:

  • You potentially removed 179 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 45 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3150947
  • DQMHistoTests: Total failures: 278
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3150647
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 207 log files, 159 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

@alintulu
Copy link
Contributor Author

alintulu commented Aug 7, 2023

Looking at the log, I think the test failed because of the missing PR to RecoBTag-Combined (cms-data/RecoBTag-Combined#49). It was added to the other PR by #40438 (comment).

@perrotta
Copy link
Contributor

perrotta commented Aug 7, 2023

test parameters:

@perrotta
Copy link
Contributor

perrotta commented Aug 7, 2023

please test

@perrotta
Copy link
Contributor

perrotta commented Aug 7, 2023

@alintulu #40438 cannot be merged as such, because it is bugged. It must be merged together with this one, which implies recollecting all signatures once again.

Either you make a new PR with only the updated PhysicsTools/NanoAOD/python/custom_run3scouting_cff.py, which will likely require only the xpog signature, and the two PRs can therefore get tested and merged together. Or we merge the overall PR with the bug fixed, as it is this one now. If we decide for the complete PR, better to stick on the old one, adding the fix to it, so that the whole set of comments can be saved for reference

@alintulu
Copy link
Contributor Author

alintulu commented Aug 7, 2023

@alintulu #40438 cannot be merged as such, because it is bugged. It must be merged together with this one, which implies recollecting all signatures once again.

Either you make a new PR with only the updated PhysicsTools/NanoAOD/python/custom_run3scouting_cff.py, which will likely require only the xpog signature, and the two PRs can therefore get tested and merged together. Or we merge the overall PR with the bug fixed, as it is this one now. If we decide for the complete PR, better to stick on the old one, adding the fix to it, so that the whole set of comments can be saved for reference

Ah I see, I didn't realise I could make a PR with just one file change which could then be tested together with the old one. Thanks! I don't have a preference, but maybe that would be the easiest? If you agree, I'll go ahead and make the PR with only the PhysicsTools/NanoAOD/python/custom_run3scouting_cff.py change.

@alintulu
Copy link
Contributor Author

alintulu commented Aug 7, 2023

@perrotta as you gave my message a thumbs up, I've gone ahead and created the new PR. You can find it at #42490. Cheers!

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2023

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0da083/34124/summary.html
COMMIT: c16b2bf
CMSSW: CMSSW_13_3_X_2023-08-07-1100/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42484/34124/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0da083/34124/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0da083/34124/git-merge-result

Comparison Summary

Summary:

  • You potentially added 179 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 51 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3150947
  • DQMHistoTests: Total failures: 281
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3150644
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 207 log files, 159 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially removed 6 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 15
  • DQMHistoTests: Total histograms compared: 15526
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 15526
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
  • Checked 31 log files, 14 edm output root files, 15 DQM output files

Nano size comparison Summary:

  • Nano ERROR: Missing ref/2500.5-size.json
  • Nano ERROR: Missing ref/2500.5-size.json
    | Sample | kb/ev | ref kb/ev | diff kb/ev | ev/s/thd | ref ev/s/thd | diff rate | mem/thd | ref mem/thd |
    | --- | --- | --- | --- | --- | --- | --- | --- | --- |
    | 2500.0 | 2.450 | 2.447 | 0.003 ( +0.1% ) | 5.13 | 5.02 | +2.2% | 2.056 | 2.104 |
    | 2500.001 | 2.578 | 2.553 | 0.024 ( +1.0% ) | 4.55 | 4.53 | +0.5% | 2.401 | 2.403 |
    | 2500.002 | 2.502 | 2.498 | 0.004 ( +0.2% ) | 4.79 | 4.69 | +2.1% | 2.387 | 2.402 |
    | 2500.01 | 1.243 | 1.241 | 0.001 ( +0.1% ) | 9.45 | 9.31 | +1.5% | 2.141 | 2.196 |
    | 2500.011 | 1.606 | 1.605 | 0.002 ( +0.1% ) | 5.14 | 5.14 | +0.0% | 2.304 | 2.290 |
    | 2500.012 | 1.488 | 1.486 | 0.002 ( +0.1% ) | 7.44 | 7.39 | +0.7% | 2.205 | 2.264 |
    | 2500.1 | 2.161 | 2.161 | 0.000 ( +0.0% ) | 5.20 | 5.21 | -0.3% | 1.924 | 1.930 |
    | 2500.2 | 2.272 | 2.271 | 0.000 ( +0.0% ) | 5.99 | 5.98 | +0.2% | 1.818 | 1.831 |
    | 2500.21 | 1.148 | 1.147 | 0.001 ( +0.0% ) | 4.31 | 4.27 | +0.9% | 2.113 | 2.133 |
    | 2500.211 | 1.506 | 1.505 | 0.001 ( +0.1% ) | 3.78 | 3.78 | -0.0% | 2.122 | 2.118 |
    | 2500.3 | 2.010 | 2.009 | 0.001 ( +0.1% ) | 12.17 | 12.23 | -0.5% | 1.619 | 1.706 |
    | 2500.31 | 1.211 | 1.209 | 0.002 ( +0.1% ) | 19.23 | 19.23 | -0.0% | 2.132 | 2.125 |
    | 2500.311 | 1.588 | 1.586 | 0.002 ( +0.1% ) | 13.98 | 14.25 | -1.9% | 2.011 | 2.002 |
    | 2500.4 | 2.010 | 2.009 | 0.001 ( +0.1% ) | 12.19 | 12.27 | -0.6% | 1.624 | 1.707 |

@perrotta
Copy link
Contributor

perrotta commented Aug 8, 2023

please test
(try to test on a clean IB and see if the quite several differences in outputs remain there)

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 8, 2023

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0da083/34150/summary.html
COMMIT: c16b2bf
CMSSW: CMSSW_13_3_X_2023-08-07-2300/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42484/34150/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 14 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 14 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3150947
  • DQMHistoTests: Total failures: 9
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3150916
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 207 log files, 159 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially added 4 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 15
  • DQMHistoTests: Total histograms compared: 15526
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 15526
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
  • Checked 31 log files, 14 edm output root files, 15 DQM output files

Nano size comparison Summary:

  • Nano ERROR: Missing ref/2500.5-size.json
  • Nano ERROR: Missing ref/2500.5-size.json
    | Sample | kb/ev | ref kb/ev | diff kb/ev | ev/s/thd | ref ev/s/thd | diff rate | mem/thd | ref mem/thd |
    | --- | --- | --- | --- | --- | --- | --- | --- | --- |
    | 2500.0 | 2.450 | 2.447 | 0.003 ( +0.1% ) | 5.16 | 5.27 | -2.0% | 1.977 | 1.785 |
    | 2500.001 | 2.578 | 2.553 | 0.024 ( +1.0% ) | 4.63 | 4.68 | -1.1% | 2.376 | 2.120 |
    | 2500.002 | 2.502 | 2.498 | 0.004 ( +0.2% ) | 4.83 | 4.89 | -1.1% | 2.368 | 2.117 |
    | 2500.01 | 1.243 | 1.241 | 0.001 ( +0.1% ) | 9.41 | 9.93 | -5.2% | 2.062 | 1.899 |
    | 2500.011 | 1.606 | 1.605 | 0.002 ( +0.1% ) | 5.15 | 5.29 | -2.7% | 2.283 | 2.012 |
    | 2500.012 | 1.488 | 1.486 | 0.002 ( +0.1% ) | 7.37 | 7.64 | -3.5% | 2.163 | 1.998 |
    | 2500.1 | 2.161 | 2.161 | 0.000 ( +0.0% ) | 5.19 | 5.42 | -4.2% | 1.868 | 1.660 |
    | 2500.2 | 2.272 | 2.271 | 0.000 ( +0.0% ) | 5.95 | 6.08 | -2.3% | 1.720 | 1.576 |
    | 2500.21 | 1.148 | 1.147 | 0.001 ( +0.0% ) | 4.34 | 4.34 | +0.0% | 2.088 | 1.843 |
    | 2500.211 | 1.506 | 1.505 | 0.001 ( +0.1% ) | 3.80 | 3.93 | -3.2% | 1.992 | 1.880 |
    | 2500.3 | 2.010 | 2.009 | 0.001 ( +0.1% ) | 12.22 | 12.88 | -5.1% | 1.611 | 1.380 |
    | 2500.31 | 1.211 | 1.209 | 0.002 ( +0.1% ) | 19.40 | 20.97 | -7.5% | 2.033 | 1.885 |
    | 2500.311 | 1.588 | 1.586 | 0.002 ( +0.1% ) | 14.30 | 15.04 | -4.9% | 2.009 | 1.773 |
    | 2500.4 | 2.010 | 2.009 | 0.001 ( +0.1% ) | 12.39 | 12.88 | -3.8% | 1.604 | 1.379 |

@perrotta
Copy link
Contributor

perrotta commented Aug 8, 2023

@missirol @Martin-Grunewald do you have any clue about the configuration error reported in the PR description?

Begin processing the 1st record. Run 357735, Event 501441569, LumiSection 300 on stream 0 at 07-Aug-2023 09:18:58.116 CEST
%MSG-e triggerEventValid:  PATTriggerProducer:patTriggerScouting  07-Aug-2023 09:18:58 CEST Run: 357735 Event: 501441569
trigger::TriggerEvent product with InputTag 'hltTriggerSummaryAOD::HLT' not in event
No HLT information produced

@missirol
Copy link
Contributor

missirol commented Aug 8, 2023

Looks like the message comes from PATTriggerProducer (not maintained by HLT, #37157 (comment)), and it's because the wf runs (a modified version of) NANO directly on RAW, where hltTriggerSummaryAOD::HLT does not exist.

https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0da083/34150/runTheMatrixINPUT-results/2500.5_ScoutingNanodata13X/step2_ScoutingNanodata13X.log

@rappoccio
Copy link
Contributor

We would like to close this PR and have you apply the changes in the original PR #40438, please.

@alintulu
Copy link
Contributor Author

We would like to close this PR and have you apply the changes in the original PR #40438, please.

Okay, pushing the changes to #40438 now!

@perrotta
Copy link
Contributor

This is now a duplicate of #40438, and it can be closed: correct @alintulu ?

@alintulu
Copy link
Contributor Author

We would like to close this PR and have you apply the changes in the original PR #40438, please.

Okay, pushing the changes to #40438 now!

Hi @perrotta, yes that is correct!

@perrotta
Copy link
Contributor

Replaced by #40438

@perrotta perrotta closed this Aug 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants