`DQMStreamerReader`: re-throw all exceptions when run in unit test mode #47430

mmusich · 2025-02-24T09:38:26Z

PR description:

This PR is a continuation of #45231. This will allow to cover all cases of "silently" broken streamer inputs as in #43108 (comment), when the input streamer files layouts of the DQM/Integration unit tests get broken.

PR validation:

cmssw compiles

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

N/A

cmsbuild · 2025-02-24T09:38:53Z

cms-bot internal usage

cmsbuild · 2025-02-24T09:40:39Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47430/43833

cmsbuild · 2025-02-24T09:41:05Z

A new Pull Request was created by @mmusich for master.

It involves the following packages:

DQMServices/StreamerIO (dqm)

@antoniovagnerini, @cmsbuild, @rseidita can you please review it and eventually sign? Thanks.
@barvic this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

mmusich · 2025-02-24T09:46:50Z

@cmsbuild, please test

expecting to have this fail unit tests, as TestDQMOnlineClient-sistrip_approx_dqm_sourceclient is currently failing silently (see log) due to the class layout change for GlobalObjectMapRecord due to add BX info to GlobalObjectMap (L1uGT emulation) #47030.

cmsbuild · 2025-02-24T11:46:38Z

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6f8ef5/44591/summary.html
COMMIT: 218938f
CMSSW: CMSSW_15_1_X_2025-02-23-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/47430/44591/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3920300
DQMHistoTests: Total failures: 43
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3920237
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 214 log files, 184 edm output root files, 49 DQM output files
TriggerResults: no differences found

mmusich · 2025-02-24T12:38:28Z

test parameters:

addpkg = DQM/Integration

mmusich · 2025-02-24T12:38:36Z

@cmsbuild, please test

cmsbuild · 2025-02-24T14:34:53Z

-1

Failed Tests: UnitTests
Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6f8ef5/44602/summary.html
COMMIT: 218938f
CMSSW: CMSSW_15_1_X_2025-02-24-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/47430/44602/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test TestDQMOnlineClient-sistrip_approx_dqm_sourceclient had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3920300
DQMHistoTests: Total failures: 107
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3920173
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 214 log files, 184 edm output root files, 49 DQM output files
TriggerResults: no differences found

mmusich · 2025-02-24T14:37:10Z

As expected now TestDQMOnlineClient-sistrip_approx_dqm_sourceclient fails:

----- Begin Fatal Exception 24-Feb-2025 13:57:42 CET-----------------------
An exception of category 'FileReadError' occurred while
   [0] Calling InputSource::getNextItemType
Exception Message:
DQMStreamerReader::checkNext Can't deserialize event or registry data: An exception of category 'FatalRootError' occurred.
   Additional Info:
      [a] Fatal Root Error: @SUB=TBufferFile::ReadClassBuffer
Could not find the StreamerInfo for version 2 of the class pair<short,int>, object skipped at offset 2369056


 error: data file corrupted, rethrowing!
----- End Fatal Exception -------------------------------------------------

mmusich · 2025-02-24T17:55:27Z

test parameters:

pull_request = Update streamHIDQM files due to class layout change in GlobalObjectMapRecord. cms-data/DQM-Integration#10

mmusich · 2025-02-24T17:55:33Z

@cmsbuild, please test

cmsbuild · 2025-02-25T00:37:36Z

-1

Failed Tests: UnitTests
Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6f8ef5/44615/summary.html
COMMIT: 218938f
CMSSW: CMSSW_15_1_X_2025-02-24-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/47430/44615/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6f8ef5/44615/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6f8ef5/44615/git-merge-result

Unit Tests

I found 1 errors in the following unit tests:

---> test testFWCoreConcurrency had ERRORS

Comparison Summary

Summary:

You potentially added 16 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3920300
DQMHistoTests: Total failures: 53
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3920227
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 214 log files, 184 edm output root files, 49 DQM output files
TriggerResults: no differences found

mmusich · 2025-02-25T05:55:48Z

@cmsbuild, please test

cmsbuild · 2025-02-25T08:22:13Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6f8ef5/44624/summary.html
COMMIT: 218938f
CMSSW: CMSSW_15_1_X_2025-02-24-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/47430/44624/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 15 lines to the logs
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3920300
DQMHistoTests: Total failures: 5
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3920275
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 214 log files, 184 edm output root files, 49 DQM output files
TriggerResults: no differences found

antoniovagnerini · 2025-02-25T10:21:42Z

+dqm

cmsbuild · 2025-02-25T10:22:06Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @rappoccio, @mandrenguyen, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)
Notice This PR was tested with additional Pull Request(s), please also merge them if necessary: cms-data/DQM-Integration#10

mandrenguyen · 2025-02-25T10:32:01Z

+1

…Monitor_client_15_0_X [15_0_X] Combined backport of #47430 and #47418

DQMStreamerReader: re-throw all exceptions in unit test mode

218938f

cmsbuild added this to the CMSSW_15_1_X milestone Feb 24, 2025

cmsbuild added dqm-pending pending-signatures tests-pending orp-pending code-checks-pending labels Feb 24, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 24, 2025

cmsbuild added tests-started and removed tests-pending labels Feb 24, 2025

cmsbuild added tests-approved and removed tests-started labels Feb 24, 2025

cmsbuild added tests-started and removed tests-approved labels Feb 24, 2025

cmsbuild added tests-rejected and removed tests-started labels Feb 24, 2025

mmusich mentioned this pull request Feb 24, 2025

Update streamHIDQM files due to class layout change in GlobalObjectMapRecord. cms-data/DQM-Integration#10

Merged

cmsbuild added the requires-external label Feb 24, 2025

cmsbuild added tests-started and removed tests-rejected labels Feb 24, 2025

cmsbuild added tests-rejected and removed tests-started labels Feb 25, 2025

cmsbuild added tests-started and removed tests-rejected labels Feb 25, 2025

cmsbuild added tests-approved and removed tests-started labels Feb 25, 2025

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Feb 25, 2025

cmsbuild added orp-approved and removed orp-pending labels Feb 25, 2025

cmsbuild merged commit 52c25a3 into cms-sw:master Feb 25, 2025
12 checks passed

mmusich deleted the fix_streamers_re-throwing branch February 25, 2025 10:56

cmsbuild mentioned this pull request Feb 25, 2025

DuplicateTrackMerger: fix relvals 29634.703,29834.703 failing in DBG_X IBs #47440

Merged

francescobrivio mentioned this pull request Feb 25, 2025

[15_0_X] Combined backport of #47430 and #47418 #47449

Merged

cmsbuild added a commit that referenced this pull request Feb 26, 2025

Merge pull request #47449 from francescobrivio/alca-update_onlineBeam…

2e53561

…Monitor_client_15_0_X [15_0_X] Combined backport of #47430 and #47418

DQMStreamerReader: re-throw all exceptions when run in unit test mode #47430

DQMStreamerReader: re-throw all exceptions when run in unit test mode #47430

Uh oh!

Conversation

mmusich commented Feb 24, 2025

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Uh oh!

cmsbuild commented Feb 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Feb 24, 2025

Uh oh!

cmsbuild commented Feb 24, 2025

Uh oh!

mmusich commented Feb 24, 2025

Uh oh!

cmsbuild commented Feb 24, 2025

Comparison Summary

Uh oh!

mmusich commented Feb 24, 2025

Uh oh!

mmusich commented Feb 24, 2025

Uh oh!

cmsbuild commented Feb 24, 2025

Unit Tests

Comparison Summary

Uh oh!

mmusich commented Feb 24, 2025

Uh oh!

mmusich commented Feb 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mmusich commented Feb 24, 2025

Uh oh!

cmsbuild commented Feb 25, 2025

Unit Tests

Comparison Summary

Uh oh!

mmusich commented Feb 25, 2025

Uh oh!

cmsbuild commented Feb 25, 2025

Comparison Summary

Uh oh!

antoniovagnerini commented Feb 25, 2025

Uh oh!

cmsbuild commented Feb 25, 2025

Uh oh!

mandrenguyen commented Feb 25, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

`DQMStreamerReader`: re-throw all exceptions when run in unit test mode #47430

`DQMStreamerReader`: re-throw all exceptions when run in unit test mode #47430

cmsbuild commented Feb 24, 2025 •

edited

Loading

mmusich commented Feb 24, 2025 •

edited

Loading