add layer-1 monitoring for new slot-7 cards [13_1_0] by hftsoi · Pull Request #42076 · cms-sw/cmssw

hftsoi · 2023-06-24T09:18:01Z

PR description:

This PR modifies Calo-Layer1 unpacker to adapt the additions of a new CTP7 card in slot-7 in each of the three layer-1 crates (FEDs 1354, 1356, 1358), where each card sends the same payload header and trailer as all other existing calo cards, but with a fixed payload data size of 6 32-bit words, regardless of normal or FAT events being sent. New monitoring elements are added to layer-1 DQM for the 3x6x32 bits. The modification is done in such a way that it works before and after the card addition.

Note that the monitoring elements for HCAL FB4-5 are commented out, we will put them back once HCAL fixes them (FB4-5 are reserved bits and not used for LLP, but they are sending unphysical data there which layer-1 could not read out, causing discrepancies seen when comparing them).

PR validation:

Validated by running offline DQM on past commissioning runs, it works as expected for current production firmware. Tested with online DQM at P5 with new firmware.

This is a backport of #42048

cmsbuild · 2023-06-24T09:18:21Z

A new Pull Request was created by @hftsoi (Ho-Fung Tsoi) for CMSSW_13_1_X.

It involves the following packages:

DQM/L1TMonitor (dqm)
EventFilter/L1TRawToDigi (l1)

@aloeliger, @epalencia, @nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@dinyar, @missirol, @Martin-Grunewald, @thomreis, @eyigitba this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from add layer-1 monitoring for new slot-7 cards [13_2_0_pre1] #42048

aloeliger · 2023-06-24T09:43:47Z

backport of #42048

aloeliger · 2023-06-24T09:44:24Z

please test

cmsbuild · 2023-06-24T12:16:05Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f5c177/33370/summary.html
COMMIT: 8fa7a6c
CMSSW: CMSSW_13_1_X_2023-06-23-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42076/33370/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3222771
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3222743
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -925.8799999999998 KiB( 47 files compared)
DQMHistoSizes: changed ( 10024.0,... ): -46.294 KiB L1T/L1TStage2CaloLayer1
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

aloeliger · 2023-06-24T14:28:37Z

+l1

emanueleusai · 2023-06-27T06:25:48Z

+1

cmsbuild · 2023-06-27T06:26:06Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-06-29T08:39:03Z

@hftsoi please backport the very same code as in #42048 (i.e. by including the latest updates)

minor updates

emanueleusai · 2023-07-03T04:42:16Z

please test

cmsbuild · 2023-07-03T06:10:05Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f5c177/33506/summary.html
COMMIT: 7bb8b9b
CMSSW: CMSSW_13_1_X_2023-07-02-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42076/33506/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

10224.0

----- Begin Fatal Exception 03-Jul-2023 07:26:45 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root
      [c] Input file root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.�
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root?tried=+1213llrxrd-redir.in2p3.fr,&xrdcl.requuid=3f4bf302-ffb0-4bf1-a439-676d6a4cbf4a
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

23.023.0_JpsiMM/step2_JpsiMM.log
4.294.29_RunMinBias2011B/step2_RunMinBias2011B.log
134.806134.806_RunMuonEG2015C/step2_RunMuonEG2015C.log

Expand to see more relval errors ...

hftsoi · 2023-07-03T08:01:25Z

-1

Failed Tests: RelVals RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f5c177/33506/summary.html COMMIT: 7bb8b9b CMSSW: CMSSW_13_1_X_2023-07-02-2300/el8_amd64_gcc11 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42076/33506/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

10224.0

----- Begin Fatal Exception 03-Jul-2023 07:26:45 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root
      [c] Input file root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.�
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/relval/CMSSW_12_0_0_pre4/RelValMinBias_13/GEN-SIM/113X_mc2017_realistic_v5-v1/00000/a21693e9-4d25-496a-96e2-c28232a7a712.root?tried=+1213llrxrd-redir.in2p3.fr,&xrdcl.requuid=3f4bf302-ffb0-4bf1-a439-676d6a4cbf4a
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

23.023.0_JpsiMM/step2_JpsiMM.log
4.294.29_RunMinBias2011B/step2_RunMinBias2011B.log
134.806134.806_RunMuonEG2015C/step2_RunMuonEG2015C.log

Expand to see more relval errors ...

Hi, I don't think the failed test has anything to do with this PR (similar failing has been seen in #38361), could you please check? Thank you!

perrotta · 2023-07-04T05:13:06Z

please test

cmsbuild · 2023-07-04T07:45:36Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f5c177/33535/summary.html
COMMIT: 7bb8b9b
CMSSW: CMSSW_13_1_X_2023-07-03-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42076/33535/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 8 lines to the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3222771
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3222746
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -925.8799999999998 KiB( 47 files compared)
DQMHistoSizes: changed ( 10024.0,... ): -46.294 KiB L1T/L1TStage2CaloLayer1
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

aloeliger · 2023-07-04T08:57:33Z

+l1

perrotta · 2023-07-04T17:19:36Z

@emanueleusai you signed this already before the latest minor updates that made this backport PR identical to its already merged master version, also signed by @cms-sw/dqm-l2
I assume that you will also sign after those minor updates, and merge this PR in order to have it included in the going to be built 13_0_10 release and have it mirrored also in this 13_1_X.
Please complain if there is anything wrong with it.

perrotta · 2023-07-04T17:19:48Z

+1

perrotta · 2023-07-04T17:20:08Z

merge

emanueleusai · 2023-07-05T05:09:39Z

+1

cmsbuild · 2023-07-05T05:09:59Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will be automatically merged.

cmsbuild added this to the CMSSW_13_1_X milestone Jun 24, 2023

cmsbuild added dqm-pending l1-pending pending-signatures tests-pending orp-pending labels Jun 24, 2023

hftsoi mentioned this pull request Jun 24, 2023

add layer-1 monitoring for new slot-7 cards [13_0_7] #42021

Merged

cmsbuild added the backport label Jun 24, 2023

cmsbuild added tests-started and removed tests-pending labels Jun 24, 2023

cmsbuild added tests-approved and removed tests-started labels Jun 24, 2023

cmsbuild added l1-approved and removed l1-pending labels Jun 24, 2023

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Jun 27, 2023

cmsbuild added backport-ok and removed backport labels Jun 28, 2023

add layer-1 monitoring for new slot-7 cards 13_1_0

7bb8b9b

minor updates

hftsoi force-pushed the layer1slot7-13_1_0 branch from 8fa7a6c to 7bb8b9b Compare July 1, 2023 19:06

cmsbuild added tests-rejected and removed tests-started labels Jul 2, 2023

cmsbuild added tests-started and removed tests-rejected labels Jul 3, 2023

cmsbuild added tests-rejected and removed tests-started labels Jul 3, 2023

cmsbuild added tests-started and removed tests-rejected labels Jul 4, 2023

cmsbuild added tests-approved and removed tests-started labels Jul 4, 2023

cmsbuild added l1-approved and removed l1-pending labels Jul 4, 2023

cmsbuild added orp-approved and removed orp-pending labels Jul 4, 2023

cmsbuild merged commit f3335e8 into cms-sw:CMSSW_13_1_X Jul 4, 2023

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Jul 5, 2023

Conversation

hftsoi commented Jun 24, 2023

PR description:

PR validation:

Uh oh!

cmsbuild commented Jun 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aloeliger commented Jun 24, 2023

Uh oh!

aloeliger commented Jun 24, 2023

Uh oh!

cmsbuild commented Jun 24, 2023

Comparison Summary

Uh oh!

aloeliger commented Jun 24, 2023

Uh oh!

emanueleusai commented Jun 27, 2023

Uh oh!

cmsbuild commented Jun 27, 2023

Uh oh!

perrotta commented Jun 29, 2023

Uh oh!

emanueleusai commented Jul 3, 2023

Uh oh!

cmsbuild commented Jul 3, 2023

RelVals

RelVals-INPUT

Uh oh!

hftsoi commented Jul 3, 2023

RelVals

RelVals-INPUT

Uh oh!

perrotta commented Jul 4, 2023

Uh oh!

cmsbuild commented Jul 4, 2023

Comparison Summary

Uh oh!

aloeliger commented Jul 4, 2023

Uh oh!

perrotta commented Jul 4, 2023

Uh oh!

perrotta commented Jul 4, 2023

Uh oh!

perrotta commented Jul 4, 2023

Uh oh!

emanueleusai commented Jul 5, 2023

Uh oh!

cmsbuild commented Jul 5, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

cmsbuild commented Jun 24, 2023 •

edited

Loading