MultiCluster-CaloParticle associator #32941

lecriste · 2021-02-18T02:27:31Z

PR description:

This PR introduces an associator that can be consumed by several analyzers, removing code duplication. The first replacement is made for the HGVHistoProducerAlgo::fill_multi_cluster_histos method, no changes are expected in the output.

PR validation:

runTheMatrix -l limited

cmsbuild · 2021-02-18T02:35:09Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32941/21171

This PR adds an extra 76KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32941/21171/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32941/21171/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

slava77 · 2021-02-18T14:20:51Z

Code check has found code style and quality issues which could be resolved by applying following patch(s)

please follow up; we can not do further tests without it

slava77 · 2021-02-25T15:46:19Z

Code check has found code style and quality issues which could be resolved by applying following patch(s)

please follow up; we can not do further tests without it

slava77 · 2021-02-25T15:47:31Z

@lecriste
(tagging explicitly, in case the earlier messages got lost)

lecriste · 2021-02-25T15:56:42Z

Hi Slava, after opening the PR I found a corner case in which results differ from original code.
I would push the code-checks patch together with the fix commit once I have it.
Please let me know if that's not OK with you.

slava77 · 2021-02-25T16:17:26Z

Please let me know if that's not OK with you.

OK.
I was just pinging to make sure you received the message(s).

slava77 · 2021-03-11T15:13:04Z

@lecriste
please clarify on the current status of the PR
Thank you.

lecriste · 2021-03-11T15:30:38Z

We decided to change a reco DataFormat (the Trackster) along with a SimDataFormat, in order to proceed with the implementation of the associator in this PR and, at the same time, keep the current feature of a dedicated validation per TICL iteration.
I am using this branch to test locally the DataFormat change, once happy I will open a separated PR for that and then come back to this one.

slava77 · 2021-04-05T14:08:41Z

@lecriste
please clarify on the status and plans for this PR.
Thank you.

lecriste · 2021-04-05T14:21:14Z

I need to rebase it to the other merged/open PRs and adapt the code to use the new Trackster data format.
Will do by next week.

cmsbuild · 2021-04-09T11:30:52Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32941/21980

This PR adds an extra 216KB to repository
Found files with invalid states:
- Geometry/HGCalCommonData/python/hgcalParametersV15Initialization_cfi.py:
  - Added: 2a151d0
  - Deleted: d1a585e
- HLTrigger/JetMET/plugins/MultiplicityValueProducerFromNestedCollection.h:
  - Added: 4be38a4
  - Modified: c24657f, 35d06bb, d613da7
  - Deleted: 6f15bd7
- HLTrigger/JetMET/plugins/MultiplicityValueProducers.cc:
  - Added: 43459a5
  - Deleted: 4be38a4
- HLTrigger/JetMET/plugins/MultiplicityValueProducerFromNestedCollection.cc:
  - Added: 4be38a4
  - Deleted: 6f15bd7
- HLTrigger/JetMET/plugins/MultiplicityValueProducer.h:
  - Added: 4be38a4
  - Modified: c24657f, 35d06bb, d613da7
  - Deleted: 6f15bd7
- HLTrigger/JetMET/plugins/MultiplicityValueProducers.h:
  - Added: 43459a5
  - Modified: 1f1038b, e62cd8b
  - Deleted: 4be38a4
- SimG4Core/Notification/go.1:
  - Added: 4d7cd73
  - Deleted: d5d5ee0
There are other open Pull requests which might conflict with changes you have proposed:
- File CondCore/SiPixelPlugins/interface/SiPixelPayloadInspectorHelper.h modified in PR(s): [RFC] Test clang-tidy --checks misc-definitions-in-headers #33366
- File Configuration/Geometry/python/dict2026Geometry.py modified in PR(s): update the tracking material for Phase2 geometry D49 #30126
- File Configuration/PyReleaseValidation/python/MatrixInjector.py modified in PR(s): Pilot PR for the GPU attributes in workflow injected by runTheMatrix #33057
- File Configuration/PyReleaseValidation/python/relval_2017.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel workflows (12/N) #31854
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Add customisation to make sure that upgrade WFs will cross LS in the short matrix #32957, [PPS] Prompt calibration loop component for timing detector #33215, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel workflows (12/N) #31854
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): New_samples (TTbarLepton_14 and QQH1352T_14) added #33324, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel workflows (12/N) #31854
- File Configuration/PyReleaseValidation/scripts/runTheMatrix.py modified in PR(s): Pilot PR for the GPU attributes in workflow injected by runTheMatrix #33057, Add the possibility to run multiple matrices in runTheMatrix.py #33338
- File Configuration/StandardSequences/python/Eras.py modified in PR(s): PPS: update of era modifiers #33250
- File Configuration/StandardSequences/python/Reconstruction_cff.py modified in PR(s): PPS: update of era modifiers #33250
- File DQM/Integration/python/clients/beam_dqm_sourceclient-live_cfg.py modified in PR(s): fix beam*_dqm_sourceclient-live_cfg.py #33378, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File DataFormats/RecoCandidate/BuildFile.xml modified in PR(s): moving DataFormats/RecoCandidate/interface/TrackAssociation.h into SimDataFormats #33186
- File DataFormats/RecoCandidate/src/classes.h modified in PR(s): moving DataFormats/RecoCandidate/interface/TrackAssociation.h into SimDataFormats #33186
- File EventFilter/HcalRawToDigi/plugins/PackerHelp.h modified in PR(s): [RFC] Test clang-tidy --checks misc-definitions-in-headers #33366
- File FWCore/Framework/src/EDAnalyzer.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/EDLooperBase.cc modified in PR(s): Allow consumes declarations in EDLooper #33337
- File FWCore/Framework/src/EDProducer.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/EventProcessor.cc modified in PR(s): Allow consumes declarations in EDLooper #33337
- File FWCore/Framework/src/global/EDAnalyzerBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/global/EDFilterBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/global/EDProducerBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/limited/EDAnalyzerBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/limited/EDFilterBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/limited/EDProducerBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/one/EDAnalyzerBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/one/EDFilterBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/one/EDProducerBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/stream/EDAnalyzerAdaptorBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/stream/EDFilterAdaptorBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/stream/EDProducerAdaptorBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/Framework/src/stream/ProducingModuleAdaptorBase.cc modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746
- File FWCore/MessageLogger/interface/MessageLogger.h modified in PR(s): Add support of printf()-like formatting to MessageLogger #30629
- File GeneratorInterface/LHEInterface/plugins/BuildFile.xml modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167
- File GeneratorInterface/LHEInterface/plugins/ExternalLHEProducer.cc modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167
- File Geometry/EcalCommonData/plugins/dd4hep/DDEcalBarrelNewAlgo.cc modified in PR(s): [DD4hep] Ecal Barrel algorithm - use assemblies. Fix all overlaps in EBAR. #33339
- File Geometry/HGCalCommonData/plugins/DDHGCalCell.cc modified in PR(s): Phase2-hgx281 Compare DDD against DD4Hep scenario for 2026D76 scenario #33379
- File Geometry/HGCalCommonData/plugins/DDHGCalEEFileAlgo.cc modified in PR(s): Phase2-hgx281 Compare DDD against DD4Hep scenario for 2026D76 scenario #33379
- File Geometry/HGCalCommonData/plugins/DDHGCalHEFileAlgo.cc modified in PR(s): Phase2-hgx281 Compare DDD against DD4Hep scenario for 2026D76 scenario #33379
- File Geometry/HGCalCommonData/plugins/DDHGCalWafer8.cc modified in PR(s): Phase2-hgx281 Compare DDD against DD4Hep scenario for 2026D76 scenario #33379
- File Geometry/HGCalCommonData/plugins/dd4hep/DDHGCalHEFileAlgo.cc modified in PR(s): Phase2-hgx281 Compare DDD against DD4Hep scenario for 2026D76 scenario #33379
- File Geometry/MTDGeometryBuilder/test/MTDDigiGeometryAnalyzer.cc modified in PR(s): MTD geometry, digitization, reconstruction and validation: technical cleaning of code #33374, Rectangular mtd cleaning #33340
- File L1Trigger/CSCTriggerPrimitives/src/CSCCathodeLCTProcessor.cc modified in PR(s): New binning of Run-3 CLCT slope (CCLUT-13) #33380
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): PPS: update of era modifiers #33250
- File RecoHGCal/Configuration/python/RecoHGCal_EventContent_cff.py modified in PR(s): [HGCAL] TICL SimTracksters #33318
- File RecoHGCal/TICL/python/HADStep_cff.py modified in PR(s): [HGCAL] Update TICL iterations naming #33336
- File RecoHGCal/TICL/python/iterativeTICL_cff.py modified in PR(s): [HGCAL] TICL SimTracksters #33318, [HGCAL] Update TICL iterations naming #33336
- File RecoLocalMuon/CSCRecHitD/python/cscRecHitD_cfi.py modified in PR(s): Migrate module configurations in RecoLocalMuon to use automatically generated cfi default configurations #33354
- File RecoLocalTracker/SiPixelRecHits/plugins/SiPixelRecHitSoAFromLegacy.cc modified in PR(s): Make the the size of the binner (HistoContainer) settable at run time. The total number of Pixel Clusters is not a limit anymore on GPU #33371
- File RecoMTD/DetLayers/test/MTDRecoGeometryAnalyzer.cc modified in PR(s): MTD geometry, digitization, reconstruction and validation: technical cleaning of code #33374
- File RecoMTD/TrackExtender/plugins/TrackExtenderWithMTD.cc modified in PR(s): MTD geometry, digitization, reconstruction and validation: technical cleaning of code #33374
- File RecoPPS/Configuration/python/recoCTPPS_cff.py modified in PR(s): Re-enable ctppsProtons from recoCTPPSTask in Run-3 #33266
- File RecoTracker/TkSeedGenerator/plugins/BuildFile.xml modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File RecoVertex/PrimaryVertexProducer/interface/DAClusterizerInZ_vect.h modified in PR(s): modified merging scheme and tighter outlier suppression in primary vertex reconstruction #33341
- File RecoVertex/PrimaryVertexProducer/src/DAClusterizerInZ_vect.cc modified in PR(s): modified merging scheme and tighter outlier suppression in primary vertex reconstruction #33341
- File RecoVertex/V0Producer/src/V0Fitter.cc modified in PR(s): EXO regional tracking iteration #32806
- File RecoVertex/V0Producer/src/V0Fitter.h modified in PR(s): EXO regional tracking iteration #32806
- File SimCalorimetry/EcalTrigPrimAlgos/interface/EcalFenixStrip.h modified in PR(s): ECAL Emulator Update: Full Configuration Flexibility including L1 Double Weights #33349
- File SimCalorimetry/EcalTrigPrimAlgos/interface/EcalFenixTcp.h modified in PR(s): ECAL Emulator Update: Full Configuration Flexibility including L1 Double Weights #33349
- File SimCalorimetry/EcalTrigPrimAlgos/interface/EcalTrigPrimFunctionalAlgo.h modified in PR(s): ECAL Emulator Update: Full Configuration Flexibility including L1 Double Weights #33349
- File SimCalorimetry/EcalTrigPrimAlgos/src/EcalFenixStrip.cc modified in PR(s): ECAL Emulator Update: Full Configuration Flexibility including L1 Double Weights #33349
- File SimCalorimetry/EcalTrigPrimAlgos/src/EcalFenixTcp.cc modified in PR(s): ECAL Emulator Update: Full Configuration Flexibility including L1 Double Weights #33349
- File SimCalorimetry/EcalTrigPrimAlgos/src/EcalTrigPrimFunctionalAlgo.cc modified in PR(s): ECAL Emulator Update: Full Configuration Flexibility including L1 Double Weights #33349
- File SimCalorimetry/EcalTrigPrimProducers/plugins/EcalTrigPrimProducer.cc modified in PR(s): ECAL Emulator Update: Full Configuration Flexibility including L1 Double Weights #33349
- File SimCalorimetry/HGCalAssociatorProducers/interface/AssociatorTools.h modified in PR(s): [RFC] Test clang-tidy --checks misc-definitions-in-headers #33366
- File SimDataFormats/Associations/plugins/BuildFile.xml modified in PR(s): [HGCAL] Tracksters to SimClusters associator #33333
- File SimDataFormats/Associations/src/classes.h modified in PR(s): moving DataFormats/RecoCandidate/interface/TrackAssociation.h into SimDataFormats #33186, [HGCAL] Tracksters to SimClusters associator #33333
- File SimDataFormats/Associations/src/classes_def.xml modified in PR(s): moving DataFormats/RecoCandidate/interface/TrackAssociation.h into SimDataFormats #33186, [HGCAL] Tracksters to SimClusters associator #33333
- File Validation/Configuration/python/globalValidation_cff.py modified in PR(s): [HGCAL] TICL SimTracksters #33318
- File Validation/Configuration/python/hgcalSimValid_cff.py modified in PR(s): [HGCAL] EDAnalyzer for TICL tracksters validation #33360, [HGCAL] TICL SimTracksters #33318
- File Validation/HGCalValidation/interface/HGVHistoProducerAlgo.h modified in PR(s): [HGCAL] EDAnalyzer for TICL tracksters validation #33360
- File Validation/HGCalValidation/plugins/HGCalValidator.cc modified in PR(s): [HGCAL] EDAnalyzer for TICL tracksters validation #33360
- File Validation/HGCalValidation/python/CaloParticleSelectionForEfficiency_cfi.py modified in PR(s): [HGCAL] EDAnalyzer for TICL tracksters validation #33360
- File Validation/HGCalValidation/python/HGCalValidator_cfi.py modified in PR(s): [HGCAL] EDAnalyzer for TICL tracksters validation #33360, [HGCAL] TICL SimTracksters #33318, [HGCAL] Update TICL iterations naming #33336
- File Validation/HGCalValidation/python/PostProcessorHGCAL_cfi.py modified in PR(s): [HGCAL] TICL SimTracksters #33318, [HGCAL] Update TICL iterations naming #33336
- File Validation/HGCalValidation/python/hgcalPlots.py modified in PR(s): [HGCAL] EDAnalyzer for TICL tracksters validation #33360
- File Validation/HGCalValidation/scripts/makeHGCalValidationPlots.py modified in PR(s): [HGCAL] EDAnalyzer for TICL tracksters validation #33360, [HGCAL] TICL SimTracksters #33318
- File Validation/HGCalValidation/src/HGVHistoProducerAlgo.cc modified in PR(s): [HGCAL] EDAnalyzer for TICL tracksters validation #33360
- File Validation/RecoTrack/python/plotting/plotting.py modified in PR(s): [HGCAL] EDAnalyzer for TICL tracksters validation #33360

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32941/21980/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32941/21980/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

lecriste · 2021-04-09T11:52:18Z

This PR is superseded by #33384, which contains only the associator implementation.
We do not use it in Validation/HGCalValidation/src/HGVHistoProducerAlgo.cc because we are going to replace MultiCluster with Trackster therein.

cmsbuild added this to the CMSSW_11_3_X milestone Feb 18, 2021

cmsbuild added code-checks-pending dqm-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending upgrade-pending labels Feb 18, 2021

cmsbuild added code-checks-rejected and removed code-checks-pending labels Feb 18, 2021

cmsbuild mentioned this pull request Feb 21, 2021

[RFC] Outline of HGCAL ML ntuples using NanoAOD #32187

Closed

cmsbuild mentioned this pull request Mar 4, 2021

[HGCAL] Reorganizing layer cluster plots into subfolders #33063

Merged

This was referenced Mar 14, 2021

HFNose: ticl updates #33001

Merged

moving DataFormats/RecoCandidate/interface/TrackAssociation.h into SimDataFormats #33186

Merged

move TTTrackTruthPair to SimDataFormats/Associations #33188

Merged

smorovic and others added 6 commits March 16, 2021 19:43

change fillDescriptions to generate EvFOutputModule cfi file for confDB.

c8aa48b

code checks

04a1e0b

Code check

5a245f5

improved file name generation in FRDOutputModule and test script

a47ce9b

code-format

cf6d18b

Update HCAL code in material budger studies for DD4HEP

246388c

This was referenced Apr 5, 2021

[HGCAL] Update TICL iterations naming #33336

Merged

[HGCAL] EDAnalyzer for TICL tracksters validation #33360

Merged

[RFC] Test clang-tidy --checks misc-definitions-in-headers #33366

Closed

Associator only

e437a0c

cmsbuild added code-checks-rejected and removed code-checks-pending labels Apr 9, 2021

cmsbuild mentioned this pull request Apr 9, 2021

[HGCAL] MultiCluster to CaloParticle associator #33384

Merged

lecriste closed this Apr 9, 2021

MultiCluster-CaloParticle associator #32941

MultiCluster-CaloParticle associator #32941

Uh oh!

Conversation

lecriste commented Feb 18, 2021

PR description:

PR validation:

Uh oh!

cmsbuild commented Feb 18, 2021

Uh oh!

slava77 commented Feb 18, 2021

Uh oh!

slava77 commented Feb 25, 2021

Uh oh!

slava77 commented Feb 25, 2021

Uh oh!

lecriste commented Feb 25, 2021

Uh oh!

slava77 commented Feb 25, 2021

Uh oh!

slava77 commented Mar 11, 2021

Uh oh!

lecriste commented Mar 11, 2021

Uh oh!

slava77 commented Apr 5, 2021

Uh oh!

lecriste commented Apr 5, 2021

Uh oh!

cmsbuild commented Apr 9, 2021

Uh oh!

lecriste commented Apr 9, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants