DQM: Remove concurrent-lumi blocking DQM Modules. #30111

schneiml · 2020-06-04T17:24:24Z

PR description:

This PR removes (some?) of the remaining modules blocking concurrent lumisections in production jobs.

I ran a random 136 relval with concurrent lumis enabled and got these modules reported:

  RPCDcsInfo rpcDcsInfo
  DetStatus ALCARECOTkAlMinBiasDCSFilter
  EcalDQMonitorTask ecalMonitorTask
  ESIntegrityTask ecalPreshowerIntegrityTask
  L1TStage2CaloLayer1 l1tStage2CaloLayer1
  AlcaBeamMonitor AlcaBeamMonitor

(Update: DetStatus is gone (thanks @mmusich !), and after #30187, EcalDQMonitorTask is gone as well. ESIntegrityTask remains, however.)

DetStatus is not DQM related, I removed the others from the sequences. There might be more modules that are not used in this specific workflow.

Overall, some of these modules are quite important and we don't want to merge this PR. But it gives an idea of what these modules do and allows to try actually enabling concurrent lumis.

(@silviodonato update:
RPCDcsInfo rpcDcsInfo updated with #31056
ALCARECOTkAlMinBiasDCSFilter -> updated with #30124
)

PR validation:

None, so far; not supposed to be merged for now.

cmsbuild · 2020-06-04T17:24:49Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-06-04T17:30:38Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30111/15875

This PR adds an extra 20KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQMOffline/Configuration/python/DQMOffline_cff.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

schneiml · 2020-06-04T17:30:52Z

please test

cmsbuild · 2020-06-04T17:30:59Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6823/console Started: 2020/06/04 19:31

cmsbuild · 2020-06-04T17:31:00Z

A new Pull Request was created by @schneiml (Marcel Schneider) for master.

It involves the following packages:

DQM/EcalPreshowerMonitorModule
DQM/L1TMonitor
DQM/RPCMonitorClient
DQMOffline/Configuration
DQMOffline/Ecal

@andrius-k, @kmaeshima, @schneiml, @cmsbuild, @jfernan2, @fioriNTU can you please review it and eventually sign? Thanks.
@rchatter, @acimmino, @argiro, @threus, @rociovilar this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2020-06-04T19:40:39Z

+1
Tested at: 0dfe42c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-73693c/6823/summary.html
CMSSW: CMSSW_11_2_X_2020-06-04-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-06-04T19:40:42Z

Comparison job queued.

cmsbuild · 2020-06-04T21:44:08Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-73693c/6823/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 36
DQMHistoTests: Total histograms compared: 2753347
DQMHistoTests: Total failures: 8321
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2745004
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -294786.719 KiB( 35 files compared)
DQMHistoSizes: changed ( 1000.0,... ): -2837.981 KiB EcalBarrel/EBTimingTask
DQMHistoSizes: changed ( 1000.0,... ): -1683.506 KiB EcalEndcap/EETimingTask
DQMHistoSizes: changed ( 1000.0,... ): -1166.814 KiB EcalBarrel/EBOccupancyTask
DQMHistoSizes: changed ( 1000.0,... ): -1092.231 KiB EcalBarrel/EBIntegrityTask
DQMHistoSizes: changed ( 1000.0,... ): -1081.266 KiB EcalBarrel/EBPedestalOnlineTask
DQMHistoSizes: changed ( 1000.0,... ): -705.107 KiB EcalEndcap/EETriggerTowerTask
DQMHistoSizes: changed ( 1000.0,... ): -628.273 KiB EcalEndcap/EEOccupancyTask
DQMHistoSizes: changed ( 1000.0,... ): -459.070 KiB EcalEndcap/EEPedestalOnlineTask
DQMHistoSizes: changed ( 1000.0,... ): -441.843 KiB EcalEndcap/EEIntegrityTask
DQMHistoSizes: changed ( 1000.0,... ): -340.997 KiB EcalEndcap/EESummaryClient
DQMHistoSizes: changed ( 1000.0 ): ...
Checked 152 log files, 16 edm output root files, 36 DQM output files

silviodonato · 2020-06-09T12:27:07Z

@schneiml shall we include this in CMSSW_11_2_0_pre1?

schneiml · 2020-06-09T14:22:26Z

@silviodonato no, we need at least Ecal properly fixed before this can go in (@tanmaymudholkar ?)

Then, we also need a proper fix or removal for DetStatus ALCARECOTkAlMinBiasDCSFilter before this can even be used for experiments with concurrent lumis.

schneiml · 2020-06-09T14:22:35Z

hold

cmsbuild · 2020-06-09T14:22:46Z

Pull request has been put on hold by @schneiml
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

cmsbuild · 2020-09-03T14:31:04Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-73693c/9087/summary.html

Comparison Summary:

You potentially added 5922 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2602902
DQMHistoTests: Total failures: 505
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2602375
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -111013.462 KiB( 34 files compared)
DQMHistoSizes: changed ( 1000.0,... ): -540.738 KiB EcalBarrel/EBPedestalOnlineTask
DQMHistoSizes: changed ( 1000.0,... ): -540.000 KiB EcalBarrel/EBTimingTask
DQMHistoSizes: changed ( 1000.0,... ): -290.679 KiB EcalPreshower/ESIntegrityTask
DQMHistoSizes: changed ( 1000.0,... ): -285.020 KiB EcalBarrel/EBSummaryClient
DQMHistoSizes: changed ( 1000.0,... ): -251.787 KiB EcalEndcap/EESummaryClient
DQMHistoSizes: changed ( 1000.0,... ): -243.489 KiB EcalBarrel/EBIntegrityTask
DQMHistoSizes: changed ( 1000.0,... ): -243.270 KiB EcalBarrel/EBIntegrityClient
DQMHistoSizes: changed ( 1000.0,... ): -229.588 KiB EcalEndcap/EEPedestalOnlineTask
DQMHistoSizes: changed ( 1000.0,... ): -229.219 KiB EcalEndcap/EETimingTask
DQMHistoSizes: changed ( 1000.0,... ): -162.824 KiB EcalEndcap/EETriggerTowerTask
DQMHistoSizes: changed ( 1000.0 ): ...
Checked 149 log files, 22 edm output root files, 35 DQM output files

silviodonato · 2020-09-04T06:58:29Z

please test

cmsbuild · 2020-09-04T06:58:53Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-09-03-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9121/console Started: 2020/09/04 08:59

cmsbuild · 2020-09-04T09:06:14Z

+1
Tested at: 6893487
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-73693c/9121/summary.html
CMSSW: CMSSW_11_2_X_2020-09-03-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-09-04T09:06:17Z

Comparison job queued.

silviodonato · 2020-09-04T09:48:33Z

If I understand correctly,
RPCDcsInfo rpcDcsInfo has been updated with #31056
ALCARECOTkAlMinBiasDCSFilter has been updated with #30124
AlcaBeamMonitor will be updated with #31354.

This means that after having merged #31354 the list of the offending modules will be

  EcalDQMonitorTask ecalMonitorTask
  ESIntegrityTask ecalPreshowerIntegrityTask
  L1TStage2CaloLayer1 l1tStage2CaloLayer1

After having merged #31354, I would suggest @cms-sw/dqm-l2 to update this PR, re-adding the modules that have been migrated.

cmsbuild · 2020-09-04T10:35:29Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-73693c/9121/summary.html

Comparison Summary:

You potentially added 5942 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2602902
DQMHistoTests: Total failures: 505
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2602375
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -111013.462 KiB( 34 files compared)
DQMHistoSizes: changed ( 1000.0,... ): -540.738 KiB EcalBarrel/EBPedestalOnlineTask
DQMHistoSizes: changed ( 1000.0,... ): -540.000 KiB EcalBarrel/EBTimingTask
DQMHistoSizes: changed ( 1000.0,... ): -290.679 KiB EcalPreshower/ESIntegrityTask
DQMHistoSizes: changed ( 1000.0,... ): -285.020 KiB EcalBarrel/EBSummaryClient
DQMHistoSizes: changed ( 1000.0,... ): -251.787 KiB EcalEndcap/EESummaryClient
DQMHistoSizes: changed ( 1000.0,... ): -243.489 KiB EcalBarrel/EBIntegrityTask
DQMHistoSizes: changed ( 1000.0,... ): -243.270 KiB EcalBarrel/EBIntegrityClient
DQMHistoSizes: changed ( 1000.0,... ): -229.588 KiB EcalEndcap/EEPedestalOnlineTask
DQMHistoSizes: changed ( 1000.0,... ): -229.219 KiB EcalEndcap/EETimingTask
DQMHistoSizes: changed ( 1000.0,... ): -162.824 KiB EcalEndcap/EETriggerTowerTask
DQMHistoSizes: changed ( 1000.0 ): ...
Checked 149 log files, 22 edm output root files, 35 DQM output files

makortel · 2020-09-04T13:05:27Z

AlcaBeamMonitor will be updated with #31354.

It seems to me that wrt. concurrent lumis AlcaBeamMonitor was updated in #31267 (#31354 only fixes a crash in case no lumis were processed).

gennai · 2020-09-04T13:09:15Z

yes, at least I think I have fixed it (well I have followed the guidelines maybe I made something wrong?)

silviodonato · 2020-09-04T15:39:29Z

AlcaBeamMonitor will be updated with #31354.

It seems to me that wrt. concurrent lumis AlcaBeamMonitor was updated in #31267 (#31354 only fixes a crash in case no lumis were processed).

yes, thanks.
@cms-sw/dqm-l2 could you update this pull request removing only

  EcalDQMonitorTask ecalMonitorTask
  ESIntegrityTask ecalPreshowerIntegrityTask
  L1TStage2CaloLayer1 l1tStage2CaloLayer1

?

jfernan2 · 2020-09-05T10:50:25Z

@silviodonato the owner of this PR is no longer in CMS (@schneiml ), so I understand I cannot modify this PR. I may create a new equivalent on Monday once I come back from Holidays if it is still on time. Thanks

silviodonato · 2020-09-05T12:16:04Z

@silviodonato the owner of this PR is no longer in CMS (@schneiml ), so I understand I cannot modify this PR. I may create a new equivalent on Monday once I come back from Holidays if it is still on time. Thanks

Yes, sure. Thanks @jfernan2 !

jfernan2 · 2020-09-05T19:22:00Z

@silviodonato I managed to get a stable connection and made the PR, so you can close this one.
#31369

silviodonato · 2020-09-06T10:32:57Z

@jfernan2 thank you!

cmsbuild added this to the CMSSW_11_2_X milestone Jun 4, 2020

cmsbuild added code-checks-pending comparison-pending dqm-pending orp-pending pending-signatures tests-pending labels Jun 4, 2020

cmsbuild removed the code-checks-pending label Jun 4, 2020

cmsbuild added the code-checks-approved label Jun 4, 2020

cmsbuild added tests-started and removed tests-pending labels Jun 4, 2020

cmsbuild added tests-approved and removed tests-started labels Jun 4, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jun 4, 2020

This was referenced Jun 7, 2020

HCAL only wf #30136

Merged

HCAL Run3: removing redundant hbhereco=HBHEIsolatedNoiseReflagger from wf's #30138

Merged

cmsbuild added the hold label Jun 9, 2020

cmsbuild added tests-rejected and removed tests-started labels Sep 3, 2020

cmsbuild added comparison-available and removed comparison-pending labels Sep 3, 2020

cmsbuild added comparison-pending and removed comparison-available tests-rejected labels Sep 4, 2020

cmsbuild added tests-pending tests-started and removed tests-pending labels Sep 4, 2020

cmsbuild added tests-approved and removed tests-started labels Sep 4, 2020

cmsbuild added comparison-available and removed comparison-pending labels Sep 4, 2020

jfernan2 mentioned this pull request Sep 5, 2020

Remove legacy dqm #31369

Closed

silviodonato closed this Sep 6, 2020

DQM: Remove concurrent-lumi blocking DQM Modules. #30111

DQM: Remove concurrent-lumi blocking DQM Modules. #30111

Uh oh!

Conversation

schneiml commented Jun 4, 2020 • edited by silviodonato Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR description:

PR validation:

Uh oh!

cmsbuild commented Jun 4, 2020

Uh oh!

cmsbuild commented Jun 4, 2020

Uh oh!

schneiml commented Jun 4, 2020

Uh oh!

cmsbuild commented Jun 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Jun 4, 2020

Uh oh!

cmsbuild commented Jun 4, 2020

Uh oh!

cmsbuild commented Jun 4, 2020

Uh oh!

cmsbuild commented Jun 4, 2020

Uh oh!

silviodonato commented Jun 9, 2020

Uh oh!

schneiml commented Jun 9, 2020

Uh oh!

schneiml commented Jun 9, 2020

Uh oh!

cmsbuild commented Jun 9, 2020

Uh oh!

cmsbuild commented Sep 3, 2020

Uh oh!

silviodonato commented Sep 4, 2020

Uh oh!

cmsbuild commented Sep 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Sep 4, 2020

Uh oh!

cmsbuild commented Sep 4, 2020

Uh oh!

silviodonato commented Sep 4, 2020

Uh oh!

cmsbuild commented Sep 4, 2020

Uh oh!

makortel commented Sep 4, 2020

Uh oh!

gennai commented Sep 4, 2020

Uh oh!

silviodonato commented Sep 4, 2020

Uh oh!

jfernan2 commented Sep 5, 2020

Uh oh!

silviodonato commented Sep 5, 2020

Uh oh!

jfernan2 commented Sep 5, 2020

Uh oh!

silviodonato commented Sep 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

schneiml commented Jun 4, 2020 •

edited by silviodonato

Loading

cmsbuild commented Jun 4, 2020 •

edited

Loading

cmsbuild commented Sep 4, 2020 •

edited

Loading