[15.0.X] Follow-up to add HLT tracking monitoring in cosmics

[mmusich]

backport of #47663

PR description:

This is a follow-up to PR #47568 and integrates the (tracking) HLT monitoring during cosmics in the DQM Offline sequences used at Tier0 when the cosmics scenario is selected.
This resolves issue #47662.
In addition wf 7.25 that was introduced at #47542 is enhanced in order to actually test the same setup that is foreseen to be used at Tier0.

PR validation:

Run the command:

runTheMatrix.py -l 7.25 -t 4 -j 8 --nEvents 100000

successfully and observed filled DQM plots in the output file.
Also run the following commands:

cmsDriver.py step3 -s RAW2DIGI,L1Reco,RECO,ALCA:MuAlGlobalCosmics,DQM:@HLTMon --conditions 150X_dataRun3_Express_v1 --datatier GEN-SIM-RECO,DQMIO --eventcontent RECOSIM,DQM -n 1000 --era Run3_2025 --scenario cosmics --filein root://eoscms.cern.ch://eos/cms/tier0/store/data/Commissioning2025/CosmicHLTMonitor/RAW/v1/000/389/873/00000/3ea592ff-5490-47ed-8242-fff796875a01.root

cmsDriver.py step5 -s HARVESTING:@HLTMon --conditions 150X_dataRun3_Express_v1 --filein file:step3_RAW2DIGI_L1Reco_RECO_ALCA_DQM_inDQM.root --scenario cosmics --filetype DQM --era Run3_2025 -n 1000

and obtained the following DQM output file: https://cernbox.cern.ch/s/ez9ceBBuCEkjFr2

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Verbatim backport #47663 of for 2025 data-taking purposes.

[cmsbuild]

A new Pull Request was created by @mmusich for CMSSW_15_0_X.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv, upgrade)
• DQMOffline/Configuration (dqm)
• DQMOffline/Trigger (dqm)

@AdrianoDee, @Moanwar, @antoniovagnerini, @cmsbuild, @DickyChant, @miquork, @rseidita, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@Fedespring, @HuguesBrun, @Martin-Grunewald, @cericeci, @fabiocos, @jhgoh, @makortel, @missirol, @mtosi, @rociovilar, @slomeo, @threus, @trocino this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

• Backported from Follow-up to add HLT tracking monitoring in cosmics #47663

[cmsbuild]

cms-bot internal usage

[mmusich]

test parameters:

• workflow = 7.25

[mmusich]

@cmsbuild, please test

[cmsbuild]

+1

Size: This PR adds an extra 76KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e809b0/45151/summary.html
COMMIT: 4150d9e
CMSSW: CMSSW_15_0_X_2025-03-22-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47664/45151/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 138 lines from the logs
• Reco comparison results: 12 differences found in the comparisons
• DQMHistoTests: Total files compared: 51
• DQMHistoTests: Total histograms compared: 4019794
• DQMHistoTests: Total failures: 105
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 4019669
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -85662.679 KiB( 50 files compared)
• DQMHistoSizes: changed ( 4.22,... ): 1.668 KiB HLT/Tracking
• DQMHistoSizes: changed ( 7.25 ): -10138.093 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 7.25 ): -6057.653 KiB PixelPhase1/Phase1_MechanicalView
• DQMHistoSizes: changed ( 7.25 ): -4579.888 KiB PixelPhase1/Tracks
• DQMHistoSizes: changed ( 7.25 ): -4512.607 KiB Hcal/DigiTask
• DQMHistoSizes: changed ( 7.25 ): -4211.243 KiB L1T/L1TStage2EMTF
• DQMHistoSizes: changed ( 7.25 ): -3592.058 KiB L1T/L1TStage2BMTF
• DQMHistoSizes: changed ( 7.25 ): -2923.854 KiB CSC/CSCOfflineMonitor
• DQMHistoSizes: changed ( 7.25 ): -2849.033 KiB EcalBarrel/EBTimingTask
• DQMHistoSizes: changed ( 7.25 ): -2629.476 KiB L1TEMU/L1TStage2CaloLayer2
• DQMHistoSizes: changed ( 7.25 ): ...
• Checked 223 log files, 192 edm output root files, 51 DQM output files
• TriggerResults: no differences found

[cmsbuild]

Pull request #47664 was updated. @AdrianoDee, @Moanwar, @antoniovagnerini, @cmsbuild, @DickyChant, @miquork, @rseidita, @srimanob, @subirsarkar can you please check and sign again.

[mmusich]

@cmsbuild, please test

[cmsbuild]

+1

Size: This PR adds an extra 84KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e809b0/45154/summary.html
COMMIT: 5d14611
CMSSW: CMSSW_15_0_X_2025-03-22-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47664/45154/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 137 lines from the logs
• Reco comparison results: 358 differences found in the comparisons
• DQMHistoTests: Total files compared: 52
• DQMHistoTests: Total histograms compared: 4133825
• DQMHistoTests: Total failures: 5863
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 4127942
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -85661.43 KiB( 51 files compared)
• DQMHistoSizes: changed ( 4.22,... ): 1.980 KiB HLT/Tracking
• DQMHistoSizes: changed ( 7.25 ): -10138.093 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 7.25 ): -6057.653 KiB PixelPhase1/Phase1_MechanicalView
• DQMHistoSizes: changed ( 7.25 ): -4579.888 KiB PixelPhase1/Tracks
• DQMHistoSizes: changed ( 7.25 ): -4512.607 KiB Hcal/DigiTask
• DQMHistoSizes: changed ( 7.25 ): -4211.243 KiB L1T/L1TStage2EMTF
• DQMHistoSizes: changed ( 7.25 ): -3592.058 KiB L1T/L1TStage2BMTF
• DQMHistoSizes: changed ( 7.25 ): -2923.854 KiB CSC/CSCOfflineMonitor
• DQMHistoSizes: changed ( 7.25 ): -2849.033 KiB EcalBarrel/EBTimingTask
• DQMHistoSizes: changed ( 7.25 ): -2629.476 KiB L1TEMU/L1TStage2CaloLayer2
• DQMHistoSizes: changed ( 7.25 ): ...
• Checked 227 log files, 197 edm output root files, 52 DQM output files
• TriggerResults: no differences found

[Moanwar]

+Upgrade

[AdrianoDee]

+pdmv

[rseidita]

+dqm

• spurious differences seen from unstable WF 12834.7 (open issue spurious differences in outputs of Run-3 wfs *.7 #39803)
• spurious differences seen from unstable WF 29634.911 (open issue Non-reproducibility in 29634.911 #45505 )
• spurious differences from unstable WF 29834.999(open issue Non-reproducibility in TrackerPhase2OTL1Track #47071)
• P5 tests OK

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_15_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_15_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @mandrenguyen, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[antoniovilela]

+1