Put back SiPixel Rechits GPU-vs-CPU validation in online DQM and GPU relvals

[mmusich]

PR description:

Title says it all, possible after the merge of #49928 (see CMSHLT-3710 for more details) in which we have (re-)started to populate the DQMGPUvsCPU stream with the SiPixelRecHits SoA data-structures, thanks to the resolution of #49349 via #49432

PR validation:

scram b runtests_TestDQMOnlineClient-pixelgpu_dqm_sourceclient runs fine after the updated.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport, it will be backported to CMSSW_16_0_X for 2026 data-taking operations.

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49937/47687

• There are other open Pull requests which might conflict with changes you have proposed:
  • File DQM/Integration/python/clients/pixelgpu_dqm_sourceclient-live_cfg.py modified in PR(s): Clean-up DQM/SiPixelHeterogeneous plugins #49936
  • File DQMOffline/Trigger/python/HeterogeneousMonitoring_cff.py modified in PR(s): Clean-up DQM/SiPixelHeterogeneous plugins #49936

[cmsbuild]

A new Pull Request was created by @mmusich for master.

It involves the following packages:

• DQM/Integration (dqm)
• DQMOffline/Trigger (dqm)

@cmsbuild, @ctarricone, @gabrielmscampos, @nothingface0, @rseidita can you please review it and eventually sign? Thanks.
@Fedespring, @HuguesBrun, @batinkov, @cericeci, @francescobrivio, @jhgoh, @missirol, @mtosi, @rociovilar, @threus, @trocino this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[mmusich]

enable gpu

[mmusich]

test parameters:

• addpkg = DQM/Integration

[mmusich]

@cmsbuild, please test

[cmsbuild]

-1

Failed Tests: RelVals-NVIDIA_T4
Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5a0c86/50886/summary.html
COMMIT: f77be6d
CMSSW: CMSSW_16_1_X_2026-01-25-2300/el8_amd64_gcc13
Additional Tests: GPU,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/49937/50886/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed RelVals-NVIDIA_T4

• 34634.40334634.403_TTbar_14TeV+Run4D121PU_Patatrack_PixelOnlyAlpaka_Validation/step2_TTbar_14TeV+Run4D121PU_Patatrack_PixelOnlyAlpaka_Validation.log
• 34634.40234634.402_TTbar_14TeV+Run4D121PU_Patatrack_PixelOnlyAlpaka/step2_TTbar_14TeV+Run4D121PU_Patatrack_PixelOnlyAlpaka.log
• 34634.75134634.751_TTbar_14TeV+Run4D121PU_HLT75e33TimingAlpaka/step2_TTbar_14TeV+Run4D121PU_HLT75e33TimingAlpaka.log

Expand to see more relval errors ...

• 34634.704
• 34634.404

Comparison Summary

The workflows 2025.0010001, 2024.0060001, 2024.0050001 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

• You potentially removed 51 lines from the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 12234 differences found in the comparisons
• DQMHistoTests: Total files compared: 52
• DQMHistoTests: Total histograms compared: 3876189
• DQMHistoTests: Total failures: 6450
• DQMHistoTests: Total nulls: 24
• DQMHistoTests: Total successes: 3869695
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -2.018 KiB( 49 files compared)
• DQMHistoSizes: changed ( 2024.0060001 ): -1.891 KiB Hcal/DigiRunHarvesting
• DQMHistoSizes: changed ( 2024.0060001 ): -0.111 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 2024.0060001 ): -0.047 KiB RPC/DCSInfo
• DQMHistoSizes: changed ( 2024.0060001 ): 0.031 KiB JetMET/SUSYDQM
• Checked 222 log files, 193 edm output root files, 52 DQM output files
• TriggerResults: no differences found

Max Memory Comparisons exceeding threshold

@cms-sw/core-l2 , I found 2 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...

• Error: Workflow 2024.0050001_RunBTagMu2024G_10k step3 max memory diff -215.7 exceeds +/- 90.0 MiB
• Error: Workflow 2025.0010001_RunJetMET02025C_10k step3 max memory diff 129.4 exceeds +/- 90.0 MiB

[mmusich]

-1

all failures are due to #49795

DQMHistoSizes: Histogram memory added: -2.018 KiB( 49 files compared)
DQMHistoSizes: changed ( 2024.0060001 ): -1.891 KiB Hcal/DigiRunHarvesting
DQMHistoSizes: changed ( 2024.0060001 ): -0.111 KiB SiStrip/MechanicalView
DQMHistoSizes: changed ( 2024.0060001 ): -0.047 KiB RPC/DCSInfo
DQMHistoSizes: changed ( 2024.0060001 ): 0.031 KiB JetMET/SUSYDQM

changes are spurious, due to

The workflows 2025.0010001, 2024.0060001, 2024.0050001 have different files in step1_dasquery.log than the ones found in the baseline.

[mmusich]

ignore tests-rejected with ib-failure

[nothingface0]

+dqm

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (test failures were overridden). This pull request will now be reviewed by the release team before it's merged. @ftenchini, @mandrenguyen, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

[ftenchini]

GPU tests are failing but I think this is expected at the moment. Waiting for confirmation before merging.

[mmusich]

GPU tests are failing but I think this is expected at the moment. Waiting for confirmation before merging.

@ftenchini yes, see #49937 (comment)

[mandrenguyen]

+1