Infrastructure for using channel-dependent pulse shapes for HB/HE #45995

igv4321 · 2024-09-12T19:10:56Z

PR description:

Adding infrastructure for modeling Hcal front-end pulse shapes channel-by-channel. The pulse shapes were determined by phase scans, as described in https://indico.cern.ch/event/1336438/contributions/5626382/attachments/2733633/4753627/HBHE_Pulse_shape_study.pdf. The new pulse shapes have not yet been turned on, pending final timing adjustment.

PR validation:

The usual matrix text were run. Also, privately, the new pulse shapes were switched on (parameter "useChannelShapes" in HBHEPhase1Reconstructor_cfi.py was set True), and the code appears to produce reasonable results on the 2023 data workflow, as indicated by the subsequent Hcal rechit dump.

cmsbuild · 2024-09-12T19:11:23Z

cms-bot internal usage

cmsbuild · 2024-09-12T19:18:32Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45995/41766

There are other open Pull requests which might conflict with changes you have proposed:
- File RecoLocalCalo/HcalRecProducers/src/HBHEPhase1Reconstructor.cc modified in PR(s): [RFC] descriptions.addDefault -> descriptions.addWithDefaultLabel #44372

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45995/41766/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45995/41766/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2024-09-12T19:26:21Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45995/41767

There are other open Pull requests which might conflict with changes you have proposed:
- File RecoLocalCalo/HcalRecProducers/src/HBHEPhase1Reconstructor.cc modified in PR(s): [RFC] descriptions.addDefault -> descriptions.addWithDefaultLabel #44372

cmsbuild · 2024-09-12T19:26:47Z

A new Pull Request was created by @igv4321 for master.

It involves the following packages:

CalibCalorimetry/HcalAlgos (alca)
DataFormats/HcalRecHit (reconstruction)
RecoLocalCalo/Configuration (reconstruction)
RecoLocalCalo/HcalRecAlgos (reconstruction)
RecoLocalCalo/HcalRecProducers (reconstruction)

@cmsbuild, @consuegs, @jfernan2, @mandrenguyen, @perrotta, @saumyaphor4252 can you please review it and eventually sign? Thanks.
@ReyerBand, @abdoulline, @apsallid, @argiro, @bsunanda, @mariadalfonso, @missirol, @mmusich, @rchatter, @rovere, @rsreds, @sameasy, @thomreis, @tocheng, @wang0jin, @youyingli, @yuanchao this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

jfernan2 · 2024-09-13T14:34:32Z

please test

missirol · 2024-09-13T16:44:54Z

RecoLocalCalo/HcalRecProducers/python/HBHEPhase1Reconstructor_cfi.py

+    # indicating whether we are using it
+    channelShapesLabel = cms.string("HcalDataShapes"),
+    useChannelShapes = cms.bool(False),
+


Hi, I'm not a reviewer, I just have a general question on these developments.

What is their target (if there is one) for going in production ? 2025 data-taking ?

Are these changes only needed for the HCAL offline reconstruction, or are they also supposed to be used at HLT ?

Ever since the heterogeneous version of the HCAL local reconstruction was ported to Alpaka (#44910), the plugin HBHEPhase1Reconstructor is not used anymore in the Run-3 HLT menus (in other words, HLT is blind to this PR, as far as I understand).

This will be useful for 2023 and 2024 re-reco as well as for the future data taking. Unfortunately, at the moment we don't have an alpaka expert in the HCAL DPG, so there is no concrete plan how to get this to work in the HLT. Of course, help from Martin Kwok (or any other alpaka expert) would be appreciated.

Thanks for the clarification.

Notifying @kakwok, @fwyzard and @cms-sw/hlt-l2.

I have to say that this

for the future data taking

combined with

there is no concrete plan how to get this to work in the HLT.

sounds to me like the HCAL DPG is not taking full ownership of the HCAL code running in production today in the HLT. I don't know the quantitative impact of these channel-dependent pulse shapes, but I would assume that we (well, HCAL specifically) should make an effort to keep HLT and offline as close as possible to each other.

cmsbuild · 2024-09-13T17:50:37Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0a1618/41506/summary.html
COMMIT: 3c0b14f
CMSSW: CMSSW_14_2_X_2024-09-12-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45995/41506/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 3 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3331158
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3331135
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

RecoLocalCalo/HcalRecAlgos/interface/MahiFit.h

mmusich · 2024-09-14T09:30:21Z

RecoLocalCalo/HcalRecAlgos/python/hcalPulseShapesESProd_cfi.py

+            label = cms.string('default'),
+            t0 = cms.uint32(0),
+            pulse = cms.vdouble(
+                5.22174e-12, 7.04852e-10, 3.49584e-08, 7.78029e-07, 9.11847e-06,


wouldn't this large amount of parameters be better stored in the database, or at least in some external cms-data repository? This makes this file ~ unreadable.

Yes, this was discussed with the Hcal calibration group. However, this is the first time we are introducing this channel-by-channel dependence since the beginning of CMS operations, and the pulse shapes are not expected to change more often than once per era, so it was decided to keep the information in a config file. While this file is large (and, of course, computer-generated), its structure is rather simple, as can be seen from the "fillDescriptions" function of the HcalPulseShapesEP.cc code.

This approach is not acceptable.

Please move the pulse shapes either to a text file in an external cms-data package, or to a database payload.

Could you please substantiate your statement? Haven't seen anything against this approach in CMSSW guidelines and can't understand why it can be harmful.

Have you seen many other modules with a configuration that takes twelve thousand lines of python code ?

Co-authored-by: Andrea Bocci <fwyzard@gmail.com>

abdoulline · 2024-09-16T04:27:48Z

For a support, you can find the names of the AlCaDB contacts in HCAL from this list: https://twiki.cern.ch/twiki/bin/viewauth/CMS/AlCaDBContacts

I'm afraid HCAL (three) names are obsolete. Yildiray Komurcu to replace them.

cmsbuild · 2024-11-22T13:12:05Z

Milestone for this pull request has been moved to CMSSW_15_0_X. Please open a backport if it should also go in to CMSSW_14_2_X.

fwyzard · 2024-12-10T22:32:31Z

type hcal

cmsbuild · 2025-02-07T08:32:36Z

Milestone for this pull request has been moved to CMSSW_15_1_X. Please open a backport if it should also go in to CMSSW_15_0_X.

cmsbuild · 2025-09-10T06:55:08Z

Milestone for this pull request has been moved to CMSSW_16_0_X. Please open a backport if it should also go in to CMSSW_15_1_X.

igv4321 · 2025-12-05T11:45:00Z

Superseeded by #49556

Infrastructure for using channel-dependent pulse shapes for HB/HE

1f6e0cb

cmsbuild added this to the CMSSW_14_2_X milestone Sep 12, 2024

cmsbuild added reconstruction-pending alca-pending pending-signatures tests-pending orp-pending code-checks-pending labels Sep 12, 2024

cmsbuild added code-checks-rejected and removed code-checks-pending labels Sep 12, 2024

code-format fix

3c0b14f

cmsbuild added code-checks-pending and removed code-checks-rejected labels Sep 12, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 12, 2024

cmsbuild added tests-started and removed tests-pending labels Sep 13, 2024

missirol reviewed Sep 13, 2024

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Sep 13, 2024

cmsbuild mentioned this pull request Sep 13, 2024

Remove legacy CUDA modules for HCAL unpacking and local reconstruction #46004

Merged

fwyzard reviewed Sep 14, 2024

View reviewed changes

RecoLocalCalo/HcalRecAlgos/interface/MahiFit.h Outdated Show resolved Hide resolved

mmusich reviewed Sep 14, 2024

View reviewed changes

Update RecoLocalCalo/HcalRecAlgos/interface/MahiFit.h

6227f8f

Co-authored-by: Andrea Bocci <fwyzard@gmail.com>

cmsbuild mentioned this pull request Sep 20, 2024

Remove the configuration of the legacy CUDA workflows #46076

Draft

cmsbuild modified the milestones: CMSSW_14_2_X, CMSSW_15_0_X Nov 22, 2024

cmsbuild added the hcal label Dec 10, 2024

cmsbuild modified the milestones: CMSSW_15_0_X, CMSSW_15_1_X Feb 7, 2025

cmsbuild mentioned this pull request Sep 8, 2025

add fillDescriptions to several plugins used at HLT (6/N) #48858

Merged

cmsbuild modified the milestones: CMSSW_15_1_X, CMSSW_16_0_X Sep 10, 2025

cmsbuild mentioned this pull request Nov 17, 2025

Fix location of alpaka modification for particleFlowClusterHCAL #49295

Merged

igv4321 mentioned this pull request Dec 5, 2025

Infrastructure for HCAL channel-dependent pulse shapes using database configurations #49556

Open

igv4321 closed this Dec 5, 2025

Infrastructure for using channel-dependent pulse shapes for HB/HE #45995

Infrastructure for using channel-dependent pulse shapes for HB/HE #45995

Uh oh!

Conversation

igv4321 commented Sep 12, 2024

PR description:

PR validation:

Uh oh!

cmsbuild commented Sep 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Sep 12, 2024

Uh oh!

cmsbuild commented Sep 12, 2024

Uh oh!

cmsbuild commented Sep 12, 2024

Uh oh!

jfernan2 commented Sep 13, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmsbuild commented Sep 13, 2024

Comparison Summary

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abdoulline commented Sep 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Nov 22, 2024

Uh oh!

fwyzard commented Dec 10, 2024

Uh oh!

cmsbuild commented Feb 7, 2025

Uh oh!

cmsbuild commented Sep 10, 2025

Uh oh!

igv4321 commented Dec 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

cmsbuild commented Sep 12, 2024 •

edited

Loading

abdoulline commented Sep 16, 2024 •

edited

Loading