[12_4_X] New HB thresholds for 2022 data rereco

[swagata87]

PR description:

Run3 2022 data has issues with HB noise which affects HCAL related quantities of leptons and photons.
For example, here are the plots of electron's PF neutral hadron isolation and H/E in 2022 EraE and EraG respectively, in prompt reco, which shows that the noise in data is not modelled in MC.

In this PR, new HB thresholds are introduced via a new era called Run3_2022_rereco.
The thresholds are obtained from HCAL DPG.
The plan is to do a rereco of [some parts of] 2022 data using a new 12_4_X release that includes this PR.
This was discussed in Cross-coordination meeting of 5th May: https://indico.cern.ch/event/1283310/
It was agreed that, these new HB thresholds will be validated using relVals, to check if data/MC agreement improves or not. The validation/sign-off should be concluded by May 18th. If a resolution cannot be achieved by this time, the rereco will proceed w/o addressing the HB issue. So, this PR is urgent.

FYI @cms-sw/hcal-dpg-l2 @cms-sw/pf-l2 @cms-sw/egamma-pog-l2 @cms-sw/jetmet-pog-l2

PR validation:

Checked with 11834.0 and with --era Run3_2022_rereco.

This PR is needed only for 12_4_X because 2022 data rereco will be done in 12_4_X only.

[cmsbuild]

A new Pull Request was created by @swagata87 (Swagata Mukherjee) for CMSSW_12_4_X.

It involves the following packages:

• Configuration/Eras (operations)
• Configuration/StandardSequences (operations)
• RecoEgamma/EgammaIsolationAlgos (reconstruction)
• RecoParticleFlow/PFClusterProducer (reconstruction)

@perrotta, @rappoccio, @clacaputo, @cmsbuild, @mandrenguyen, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@VourMa, @felicepantaleo, @jainshilpi, @Martin-Grunewald, @a-kapoor, @lgray, @mmusich, @seemasharmafnal, @mmarionncern, @makortel, @JanFSchulte, @dgulhan, @missirol, @Prasant1993, @slomeo, @Sam-Harper, @GiacomoSguazzoni, @rovere, @VinInn, @hatakeyamak, @ebrondol, @mtosi, @fabiocos, @AnnikaStein, @varuns23, @valsdav, @sobhatta, @afiqaize, @wrtabb, @sameasy, @ram1123 this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[swagata87]

urgent

• need a new 12_4_X release with this PR in for 2022 data rereco, which will start asap.

[swagata87]

@cmsbuild please test

[swagata87]

Similar problem in 2023 data is expected to be addressed by
#41271 [master]
#41368 [13_0_X]
The HB thresholds values for 2023 are higher than 2022-rereco. That's why different Era was needed.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9ec717/32454/summary.html
COMMIT: 51067fb
CMSSW: CMSSW_12_4_X_2023-05-07-0000/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41581/32454/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 5 lines to the logs
• Reco comparison results: 6 differences found in the comparisons
• DQMHistoTests: Total files compared: 51
• DQMHistoTests: Total histograms compared: 3766083
• DQMHistoTests: Total failures: 13
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3766047
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.004 KiB( 50 files compared)
• DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
• Checked 212 log files, 167 edm output root files, 51 DQM output files
• TriggerResults: no differences found

[mandrenguyen]

+reconstruction
Basically backporting the thresholds to 12_4_X but protecting it with a dedicated era.
I have not directly validated the changes, since there's no corresponding workflow.
I trust that @swagata87 checked that the thresholds are correctly loaded privately.

[swagata87]

Yes, I had run this:

cmsDriver.py step3 -s RAW2DIGI,L1Reco,RECO,RECOSIM,PAT,NANO,VALIDATION:@standardValidation+@miniAODValidation,DQM:@standardDQM+@ExtraHLT+@miniAODDQM+@nanoAODDQM --conditions auto:phase1_2022_realistic --datatier GEN-SIM-RECO,MINIAODSIM,NANOAODSIM,DQMIO -n 10 --eventcontent RECOSIM,MINIAODSIM,NANOEDMAODSIM,DQM --geometry DB:Extended --era Run3_2022_rereco --pileup Run3_Flat55To75_PoissonOOTPU --pileup_input das:/RelValMinBias_14TeV/CMSSW_12_4_9_patch1-124X_mcRun3_2022_realistic_v10_BS2022-v1/GEN-SIM --filein file:step2.root --fileout file:step3.root

as a result I got step3_RAW2DIGI_L1Reco_RECO_RECOSIM_PAT_NANO_VALIDATION_DQM_PU.py ,

then:

edmConfigDump step3_RAW2DIGI_L1Reco_RECO_RECOSIM_PAT_NANO_VALIDATION_DQM_PU.py > dump.py

Then, grep recHitEThresholdHB dump.py, shows that
recHitEThresholdHB = cms.vdouble(0.25, 0.25, 0.3, 0.3), everywhere.

[rappoccio]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will be automatically merged.

[srimanob]

Hi @swagata87 @rappoccio

Sorry to come back, just to confirm the statement in this PR that it needs only 12_4. What will happen in UL of Run-3? Will the issue be treated somehow?

I am going to finish the Era splitting as discussed in one PR, so I would like to understand the situation. Thanks.

[swagata87]

What will happen in UL of Run-3

As far as I can tell, this is not known yet. The evolving HB noise in Run3 data is still a puzzle, and trying to tame it by increasing HCAL PF recHit thresholds is our current best attempt. Whether these thresholds will stay in Run3 UL or not is really not known yet.

[malbouis]

Hi @srimanob , it is my understanding that the HCalPFCuts can be included in the GTs for releases >= 131X as from PR 41128 . I think it is the best way to deal with it in the future.

[srimanob]

But then it is not better to have this PR and #41591 in the master.

In addition, having them in the master will keep the workflow numbers the same for newcoming workflow without invention. With introduction of #41591, the wf number between 12_4 and master are not the same anymore,

https://github.com/cms-sw/cmssw/blob/CMSSW_12_4_X/Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py#L29-L32

https://github.com/cms-sw/cmssw/blob/master/Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py#L29-L32

[srimanob]

Hi @srimanob , it is my understanding that the HCalPFCuts can be included in the GTs for releases >= 131X as from PR 41128 . I think it is the best way to deal with it in the future.

Hi @malbouis
I missed your comment when I replied. Thanks for clarification.
OK, so what should be done then is to fix the workflow ID as implemented in #41591 and maybe agree if 2022 data to be reprocessed in future, the new PFCuts will be used by default.