[12_5_X] Update SiStrip and SiPixel bad components for Run 3 MC GTs by francescobrivio · Pull Request #39646 · cms-sw/cmssw

francescobrivio · 2022-10-06T09:00:56Z

PR description:

Backport of #39645
This PR updates, in the Run 3 realistic MC GTs, the SiPixel (CMSTalk request) and SiStrip (CMSTalk request) bad components tags.

See master PR for list of updated tags and GT differences.

PR validation:

Tested with:
runTheMatrix.py -l 11634.0,7.23,159.0,12434.0,12834.0 --ibeos -j 16

Backport:

Backport of #39645

cmsbuild · 2022-10-06T09:01:18Z

A new Pull Request was created by @francescobrivio for CMSSW_12_5_X.

It involves the following packages:

Configuration/AlCa (alca)

@malbouis, @yuanchao, @cmsbuild, @saumyaphor4252, @francescobrivio, @ChrisMisan, @tvami can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from Update SiStrip and SiPixel bad components for Run 3 MC GTs #39645

francescobrivio · 2022-10-06T09:06:05Z

type trk

francescobrivio · 2022-10-06T09:06:27Z

backport of #39645

francescobrivio · 2022-10-06T09:06:45Z

test parameters:

workflows = 11634.0,7.23,159.0,12434.0,12834.0

francescobrivio · 2022-10-06T09:08:36Z

@cmsbuild please test

cmsbuild · 2022-10-06T17:06:01Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4acbc1/28054/summary.html
COMMIT: 598a1b8
CMSSW: CMSSW_12_5_X_2022-10-04-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39646/28054/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-hcalreco_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-dt4ml_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS
and more ...

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/pool/condor/dir_26258/jenkins/workspace/compare-root-files-short-matrix/data/PR-4acbc1/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21

Summary:

No significant changes to the logs found
Reco comparison results: 19902 differences found in the comparisons
DQMHistoTests: Total files compared: 54
DQMHistoTests: Total histograms compared: 3994404
DQMHistoTests: Total failures: 236477
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3757905
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -4.244 KiB( 53 files compared)
DQMHistoSizes: changed ( 11834.0 ): 0.997 KiB SiStrip/MechanicalView
DQMHistoSizes: changed ( 7.23 ): -5.241 KiB SiStrip/MechanicalView
Checked 227 log files, 49 edm output root files, 54 DQM output files
TriggerResults: found differences in 7 / 53 workflows

tvami · 2022-10-06T17:32:34Z

The error seems unrelated

----- Begin Fatal Exception 06-Oct-2022 11:34:25 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named ./src/DQM/Integration/python/clients/es_dqm_sourceclient-live_cfg.py
Exception Message:
 unknown python problem occurred.
IndexError: list index out of range

At:

francescobrivio · 2022-10-07T07:02:30Z

@cmsbuild please test

trying again

mmusich · 2022-10-07T07:06:14Z

@francescobrivio

trying again

as far as I can tell, there's no point in trying again, the tests are failing in the IB as well: https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc10/CMSSW_12_5_X_2022-10-06-2300/unitTestLogs/DQM/Integration#/

francescobrivio · 2022-10-07T07:23:39Z

@cmsbuild please abort

francescobrivio · 2022-10-07T07:24:12Z

@francescobrivio

trying again

as far as I can tell, there's no point in trying again, the tests are failing in the IB as well: https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc10/CMSSW_12_5_X_2022-10-06-2300/unitTestLogs/DQM/Integration#/

Thanks Marco I had missed the failure in the IBs! I'll wait for the tests to be working again then.

mmusich · 2022-10-07T07:42:57Z

I'll wait for the tests to be working again then.

the tests complain (presumably) about missing files:

Error in <TNetXNGFile::Open>: [ERROR] Server responded with an error: [3011] No servers are available to read the file.

but the very same tests (using the very same files) are running OK in 12.6.X:

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc10/CMSSW_12_6_X_2022-10-06-2300/unitTestLogs/DQM/Integration#/

perrotta · 2022-10-07T08:38:05Z

@francescobrivio @mmusich I've opened #39669 to report the issue

perrotta · 2022-10-19T08:12:25Z

@cmsbuild please test

cmsbuild · 2022-10-19T12:12:51Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4acbc1/28353/summary.html
COMMIT: 598a1b8
CMSSW: CMSSW_12_5_X_2022-10-18-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39646/28353/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-hcalreco_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-fed_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-ecal_dqm_sourceclient had ERRORS
and more ...

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/pool/condor/dir_8307/jenkins/workspace/compare-root-files-short-matrix/data/PR-4acbc1/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21

Summary:

No significant changes to the logs found
Reco comparison results: 19902 differences found in the comparisons
DQMHistoTests: Total files compared: 54
DQMHistoTests: Total histograms compared: 3991793
DQMHistoTests: Total failures: 26710
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3965061
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -4.244 KiB( 53 files compared)
DQMHistoSizes: changed ( 7.23 ): -5.241 KiB SiStrip/MechanicalView
DQMHistoSizes: changed ( 11834.0 ): 0.997 KiB SiStrip/MechanicalView
Checked 227 log files, 49 edm output root files, 54 DQM output files
TriggerResults: found differences in 7 / 53 workflows

francescobrivio · 2022-10-19T12:15:31Z

@perrotta seems like the issue already reported above is not yet solved since these failures look the same as before...

tvami · 2022-10-19T12:16:50Z

+alca

the unit test failure is known (there is a github issue about it)
otherwise tests pass

cmsbuild · 2022-10-19T12:17:11Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_5_X IBs (but tests are reportedly failing) and once validation in the development release cycle CMSSW_12_6_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

mmusich · 2022-10-19T12:18:24Z

seems like the issue already reported above is not yet solved since these failures look the same as before...

from yesterday's ORP meeting minutes

DQM/Integration unit tests are failing in all releases but 12_6_X (github issue #39669): do not rush to backport fixes in the production releases, as drawbacks are possible

not really sure what is the backport that needs to be done and what are its drawbacks though.

francescobrivio · 2022-10-19T12:23:20Z

seems like the issue already reported above is not yet solved since these failures look the same as before...

from yesterday's ORP meeting minutes

DQM/Integration unit tests are failing in all releases but 12_6_X (github issue #39669): do not rush to backport fixes in the production releases, as drawbacks are possible

not really sure what is the backport that needs to be done and what are its drawbacks though.

Thanks Marco for pointing out yesterday's minutes, I had missed that point.
Incidentally this 125X backport PR was opened 13 days ago and the tests were re-triggered this morning by release managers, so I was assuming some progress on the failing IBs/tests had been made. 😄

perrotta · 2022-10-19T14:05:24Z

@francescobrivio I restarted the tests this morning because the PR was in the "test pending" status after #39646 (comment)
As @mmusich pointed out these AddOn errors will stay in status "won't solve", at least for some while

tvami · 2022-10-19T14:46:04Z

so we are fully signed now @perrotta and the master of this PR is ok in the IBs, shall this be merged?

perrotta · 2022-10-19T14:50:17Z

+1

perrotta · 2022-10-19T14:51:08Z

merge

update SiStrip and SiPixel bad components

598a1b8

cmsbuild added this to the CMSSW_12_5_X milestone Oct 6, 2022

cmsbuild added alca-pending orp-pending pending-signatures tests-pending labels Oct 6, 2022

cmsbuild added the trk label Oct 6, 2022

cmsbuild added the backport label Oct 6, 2022

cmsbuild added tests-started and removed tests-pending labels Oct 6, 2022

cmsbuild added tests-rejected and removed tests-started labels Oct 6, 2022

cmsbuild added tests-started and removed tests-rejected labels Oct 7, 2022

cmsbuild added tests-pending and removed tests-started labels Oct 7, 2022

cmsbuild removed the backport label Oct 18, 2022

cmsbuild added the backport-ok label Oct 18, 2022

cmsbuild added tests-started and removed tests-pending labels Oct 19, 2022

cmsbuild added tests-rejected and removed tests-started labels Oct 19, 2022

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Oct 19, 2022

cmsbuild added orp-approved and removed orp-pending labels Oct 19, 2022

cmsbuild merged commit 0fb364f into cms-sw:CMSSW_12_5_X Oct 19, 2022

francescobrivio deleted the alca-BadComponentsUpdate_12_5_X branch December 5, 2022 13:28

Conversation

francescobrivio commented Oct 6, 2022

PR description:

PR validation:

Backport:

Uh oh!

cmsbuild commented Oct 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

francescobrivio commented Oct 6, 2022

Uh oh!

francescobrivio commented Oct 6, 2022

Uh oh!

francescobrivio commented Oct 6, 2022

Uh oh!

francescobrivio commented Oct 6, 2022

Uh oh!

cmsbuild commented Oct 6, 2022

Unit Tests

Comparison Summary

Uh oh!

tvami commented Oct 6, 2022

Uh oh!

francescobrivio commented Oct 7, 2022

Uh oh!

mmusich commented Oct 7, 2022

Uh oh!

francescobrivio commented Oct 7, 2022

Uh oh!

francescobrivio commented Oct 7, 2022

Uh oh!

mmusich commented Oct 7, 2022

Uh oh!

perrotta commented Oct 7, 2022

Uh oh!

perrotta commented Oct 19, 2022

Uh oh!

cmsbuild commented Oct 19, 2022

Unit Tests

Comparison Summary

Uh oh!

francescobrivio commented Oct 19, 2022

Uh oh!

tvami commented Oct 19, 2022

Uh oh!

cmsbuild commented Oct 19, 2022

Uh oh!

mmusich commented Oct 19, 2022

Uh oh!

francescobrivio commented Oct 19, 2022

Uh oh!

perrotta commented Oct 19, 2022

Uh oh!

tvami commented Oct 19, 2022

Uh oh!

perrotta commented Oct 19, 2022

Uh oh!

perrotta commented Oct 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

cmsbuild commented Oct 6, 2022 •

edited

Loading