move JetMatching package from SimDataFormats into DataFormats #33189

davidlange6 · 2021-03-16T09:27:34Z

as with #33188 and #33187, this PR reduces the SimDataFormats dependencies in DataFormats. I believe this is the last one. SimDataFormats/JetMatching has no non-DataFormats dependencies that I see, so I just moved it to DataFormats.

cmsbuild · 2021-03-16T09:35:05Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33189/21601

This PR adds an extra 76KB to repository

cmsbuild · 2021-03-16T09:35:28Z

A new Pull Request was created by @davidlange6 (David Lange) for master.

It involves the following packages:

DataFormats/JetMatching
DataFormats/PatCandidates
HLTriggerOffline/Btag
PhysicsTools/JetCharge
PhysicsTools/JetExamples
PhysicsTools/JetMCAlgos
PhysicsTools/NanoAOD
PhysicsTools/PatAlgos
RecoJets/JetAnalyzers
SimDataFormats/JetMatching
Validation/RecoB

The following packages do not have a category, yet:

DataFormats/JetMatching
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@perrotta, @andrius-k, @gouskos, @kmaeshima, @civanch, @ErnestaP, @mdhildreth, @cmsbuild, @jfernan2, @fgolf, @ahmad3213, @slava77, @jpata, @mariadalfonso, @santocch, @rvenditti can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @schoef, @emilbols, @mbluj, @peruzzim, @seemasharmafnal, @mmarionncern, @apsallid, @ahinzmann, @smoortga, @fabiocos, @jdolen, @ferencek, @rovere, @jdamgov, @missirol, @nhanvtran, @gkasieczka, @hatakeyamak, @mtosi, @andrzejnovak, @clelange, @swertz, @swozniewski, @JyothsnaKomaragiri, @lecriste, @cbernet, @gpetruc, @mariadalfonso this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

davidlange6 · 2021-03-16T09:50:04Z

Please test

…

On Mar 16, 2021, at 10:35 AM, cmsbuild ***@***.***> wrote: A new Pull Request was created by @davidlange6 (David Lange) for master. It involves the following packages: DataFormats/JetMatching DataFormats/PatCandidates HLTriggerOffline/Btag PhysicsTools/JetCharge PhysicsTools/JetExamples PhysicsTools/JetMCAlgos PhysicsTools/NanoAOD PhysicsTools/PatAlgos RecoJets/JetAnalyzers SimDataFormats/JetMatching Validation/RecoB The following packages do not have a category, yet: DataFormats/JetMatching Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category @perrotta, @andrius-k, @gouskos, @kmaeshima, @civanch, @ErnestaP, @mdhildreth, @cmsbuild, @jfernan2, @fgolf, @ahmad3213, @slava77, @jpata, @mariadalfonso, @santocch, @rvenditti can you please review it and eventually sign? Thanks. @rappoccio, @gouskos, @schoef, @emilbols, @mbluj, @peruzzim, @seemasharmafnal, @mmarionncern, @apsallid, @ahinzmann, @smoortga, @fabiocos, @jdolen, @ferencek, @rovere, @jdamgov, @missirol, @nhanvtran, @gkasieczka, @hatakeyamak, @mtosi, @andrzejnovak, @clelange, @swertz, @swozniewski, @JyothsnaKomaragiri, @lecriste, @cbernet, @gpetruc, @mariadalfonso this is something you requested to watch as well. @silviodonato, @dpiparo, @qliphy you are the release manager for this. cms-bot commands are listed here — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

cmsbuild · 2021-03-16T13:07:57Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9f44bd/13540/summary.html
COMMIT: b4cf062
CMSSW: CMSSW_11_3_X_2021-03-15-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/33189/13540/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 3744 lines to the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2635087
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2635056
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
Checked 155 log files, 37 edm output root files, 37 DQM output files
TriggerResults: no differences found

slava77 · 2021-03-16T16:05:05Z

DataFormats/JetMatching/interface/JetFlavourInfoMatching.h

 #include "DataFormats/Common/interface/AssociationVector.h"
 #include "DataFormats/Common/interface/RefToBaseProd.h"
-#include "SimDataFormats/JetMatching/interface/JetFlavourInfo.h"
+#include "DataFormats/JetMatching/interface/JetFlavourInfo.h"


it would be nice to update the include guards as well, s/SimDataFormats_/DataFormats_/ (this applies to all files in this moved package)

Sorry I missed those. Fixed.

davidlange6 · 2021-03-17T07:49:35Z

however in the summary file there is 598 598 595 556 117 41 3 1 1 tests passed, 0 0 0 0 0 0 0 0 0 failed

yea - I've had this happen a few times recently. @smuzaffar ?

mariadalfonso · 2021-03-17T08:05:15Z

+1

smuzaffar · 2021-03-17T08:12:04Z

Failed Tests: RelVals-INPUT

however in the summary file there is 598 598 595 556 117 41 3 1 1 tests passed, 0 0 0 0 0 0 0 0 0 failed

this happens when dasgoclient itself failed e.g. in this case

2021/03/16 21:26:49 ERROR DAS QL ERROR, query=file block = /SingleMu/Run2011A-WMu-08Nov2011-v1/RAW-RECO#388c2990-0de6-11e1-bb7e-00221959e72f run = 165121 site = T2_CH_CERN, idx=0, msg=unable to parse DAS query

I will update bot to search for "ERROR DAS QL" and report it on the summary comment.

smuzaffar · 2021-03-17T09:11:40Z

the problem here with matrix INPUT is that when we run multiple das queries e.g. for workflow 4.24

 (dasgoclient --limit 0 --query 'file block=/SingleMu/Run2011A-WMu-08Nov2011-v1/RAW-RECO#388c2990-0de6-11e1-bb7e-00221959e72f run=165121 site=T2_CH_CERN' ;\
 dasgoclient --limit 0 --query 'file block=/SingleMu/Run2011A-WMu-08Nov2011-v1/RAW-RECO#388c2990-0de6-11e1-bb7e-00221959e72f run=172802 site=T2_CH_CERN')

then , due to ; , we only get the exit code of last query. In this case if first query fails then runTheMatrix does not see it as a failure. The proper way to run such commands is to use && instead of ;

davidlange6 · 2021-03-17T12:35:55Z

@smuzaffar - I believe the goal was just to get the return value of the last dasquery (eg, to try N times in case of failure)

slava77 · 2021-03-22T15:20:18Z

@cms-sw/simulation-l2
please check this PR, your signature is needed.
Thank you.

civanch · 2021-03-22T15:32:24Z

+1

qliphy · 2021-03-23T00:45:43Z

+1

qliphy · 2021-03-23T00:46:26Z

merge

mariadalfonso · 2021-03-23T13:00:13Z

@davidlange6

we started to see these kind of errors ? does it means we need to wait ?
@silviodonato @qliphy@smuzaffar

In file included from SimDataFormatsJetMatching_xr dictionary payload:63:
350/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw/CMSSW_11_3_X_2021-03-22-2300/src/SimDataFormats/JetMatching/interface/JetFlavourInfo.h:18:9: error: redefinition of 'JetFlavourInfo'
351 class JetFlavourInfo {
352 ^
353/builds/cms-nanoAOD/nanoAOD-integration/CMSSW_11_3_X_2021-03-22-2300/src/DataFormats/JetMatching/interface/JetFlavourInfo.h:18:9: note: previous definition is here
354 class JetFlavourInfo {
355 ^
356Rivet.Analysis.HiggsTemplateCrossSections: WARN Unkown Higgs production mechanism. Cannot classify event. Classification for all events will most likely fail.
357----- Begin Fatal Exception 23-Mar-2021 13:23:45 CET-----------------------
358An exception of category 'FatalRootError' occurred while
359 [0] Processing Event run: 1 lumi: 25061 event: 20890031 stream: 0
360 [1] Running path 'nanoAOD_step'
361 [2] Calling method for module SimpleCandidateFlatTableProducer/'fatJetMCTable'
362 Additional Info:
363 [a] Fatal Root Error: @sub=TInterpreter::AutoParse
364Error parsing payload code for class reco::JetFlavourInfo with content:
365#line 1 "SimDataFormatsJetMatching_xr dictionary payload"

smuzaffar · 2021-03-23T13:04:46Z

@mariadalfonso , do you get the same error in CMSSW_11_3_X_2021-03-23-1100 release?

mariadalfonso · 2021-03-23T19:11:45Z

@davidlange6 @smuzaffar
I'm sure you saw these failing for the same reason
#33251
#32966

smuzaffar · 2021-03-23T19:18:44Z

there is a new full IB available now ( CMSSW_11_3_X_2021-03-23-1400 ) and I hope that should fix these issues

santocch · 2021-03-23T21:24:00Z

+1

cmsbuild · 2021-03-23T21:24:22Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing).

mv JetMatching package into DataFormats

b4cf062

cmsbuild added this to the CMSSW_11_3_X milestone Mar 16, 2021

cmsbuild added analysis-pending code-checks-pending dqm-pending new-package-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending xpog-pending labels Mar 16, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 16, 2021

cmsbuild added tests-started and removed tests-pending labels Mar 16, 2021

cmsbuild added tests-approved and removed tests-started new-package-pending labels Mar 16, 2021

slava77 reviewed Mar 16, 2021

View reviewed changes

smuzaffar mentioned this pull request Mar 16, 2021

[LLVM] Added EventSetupRecord::get checker cms-sw/cmsdist#6734

Merged

fix include guards

ae04659

cmsbuild added code-checks-pending tests-pending and removed code-checks-approved tests-approved labels Mar 16, 2021

cmsbuild added dqm-approved and removed dqm-pending labels Mar 17, 2021

cmsbuild added xpog-approved and removed xpog-pending labels Mar 17, 2021

davidlange6 mentioned this pull request Mar 17, 2021

Update LLVM to 11.1 cms-sw/cmsdist#6691

Merged

cmsbuild added simulation-approved and removed simulation-pending labels Mar 22, 2021

cmsbuild added orp-approved and removed orp-pending labels Mar 23, 2021

cmsbuild merged commit e95eb16 into cms-sw:master Mar 23, 2021

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels Mar 23, 2021

perrotta mentioned this pull request Mar 29, 2021

Clean BuildFiles and add source_only flags to fix warnings #33282

Merged

move JetMatching package from SimDataFormats into DataFormats #33189

move JetMatching package from SimDataFormats into DataFormats #33189

Uh oh!

Conversation

davidlange6 commented Mar 16, 2021

Uh oh!

cmsbuild commented Mar 16, 2021

Uh oh!

cmsbuild commented Mar 16, 2021

Uh oh!

davidlange6 commented Mar 16, 2021 via email

Uh oh!

cmsbuild commented Mar 16, 2021

Comparison Summary

Uh oh!

slava77 Mar 16, 2021

Choose a reason for hiding this comment

Uh oh!

davidlange6 Mar 16, 2021

Choose a reason for hiding this comment

Uh oh!

davidlange6 commented Mar 17, 2021

Uh oh!

mariadalfonso commented Mar 17, 2021

Uh oh!

smuzaffar commented Mar 17, 2021

Uh oh!

smuzaffar commented Mar 17, 2021

Uh oh!

davidlange6 commented Mar 17, 2021

Uh oh!

slava77 commented Mar 22, 2021

Uh oh!

civanch commented Mar 22, 2021

Uh oh!

qliphy commented Mar 23, 2021

Uh oh!

qliphy commented Mar 23, 2021

Uh oh!

mariadalfonso commented Mar 23, 2021

Uh oh!

smuzaffar commented Mar 23, 2021

Uh oh!

mariadalfonso commented Mar 23, 2021

Uh oh!

smuzaffar commented Mar 23, 2021

Uh oh!

santocch commented Mar 23, 2021

Uh oh!

cmsbuild commented Mar 23, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants