HcalRecHit SoA dataFormat #45199

kakwok · 2024-06-11T20:39:38Z

PR description:

A subset of changes from #44910, including only the data format changes for HcalRecHit SoA output.
Since the output data format is stable, this PR allows @jsamudio in parallel to develop for the changes on the PF cluster side to use HcalRecHit SoA as input.

PR validation:

See #44910.

cmsbuild · 2024-06-11T20:40:06Z

cms-bot internal usage

cmsbuild · 2024-06-11T20:44:17Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45199/40549

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/HcalRecHit/BuildFile.xml modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/interface/HcalRecHitHostCollection.h modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/interface/HcalRecHitSoA.h modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/interface/alpaka/HcalRecHitDeviceCollection.h modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/src/alpaka/classes_cuda.h modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/src/alpaka/classes_cuda_def.xml modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/src/alpaka/classes_rocm.h modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/src/alpaka/classes_rocm_def.xml modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/src/classes.cc modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/src/classes.h modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910
- File DataFormats/HcalRecHit/src/classes_def.xml modified in PR(s): Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910

cmsbuild · 2024-06-11T20:44:40Z

A new Pull Request was created by @kakwok for master.

It involves the following packages:

DataFormats/HcalRecHit (reconstruction)

@jfernan2, @cmsbuild, @mandrenguyen can you please review it and eventually sign? Thanks.
@missirol, @bsunanda, @rovere, @mariadalfonso, @abdoulline this is something you requested to watch as well.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.

cms-bot commands are listed here

mandrenguyen · 2024-06-17T01:52:38Z

@kakwok is there a relval workflows that tests this PR, or not yet?

kakwok · 2024-06-17T15:48:05Z

@mandrenguyen Not yet, the workflow for testing will need #44910 .

fwyzard · 2024-06-17T16:21:29Z

assign heterogeneous

cmsbuild · 2024-06-17T16:21:51Z

New categories assigned: heterogeneous

@fwyzard,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

fwyzard · 2024-06-17T16:22:25Z

DataFormats/HcalRecHit/interface/HcalRecHitSoA.h

+
+  GENERATE_SOA_LAYOUT(HcalRecHitSoALayout,
+                      SOA_SCALAR(uint32_t, size),
+                      SOA_COLUMN(uint32_t, did),


could you change did to detId ?

fwyzard · 2024-06-17T16:27:11Z

DataFormats/HcalRecHit/BuildFile.xml

+<use name="DataFormats/DetId"/>
+<use name="DataFormats/Portable"/>
+<use name="DataFormats/SoATemplate"/>
+<use name="FWCore/MessageLogger"/>


Suggested change

<use name="FWCore/MessageLogger"/>

This doesn't seem to be needed.

fwyzard · 2024-06-17T16:27:36Z

DataFormats/HcalRecHit/BuildFile.xml

 <use name="DataFormats/Common"/>
 <use name="DataFormats/HcalDetId"/>
 <use name="DataFormats/HcalDigi"/>
+<use name="DataFormats/DetId"/>


Suggested change

<use name="DataFormats/DetId"/>

This one doesn't seem to be needed, either.

fwyzard · 2024-06-17T16:29:34Z

In addition, @jsamudio is working on updating the alpaka PF clustering code to replace the ad hoc SoA they are currently using with this one.

mandrenguyen · 2024-06-17T16:47:40Z

@mandrenguyen Not yet, the workflow for testing will need #44910 .

Thanks, I was just trying to understand the expectation. So I guess the proposal is just to visually inspect this and integrate for upcoming developments. Looks straightforward anyway.

fwyzard · 2024-06-17T16:50:24Z

@mandrenguyen of course. To clarify: the idea is to merge the data formats from #44910 early, while the validation of the algorithms is still ongoing, to let the PF code adopt them.

fwyzard · 2024-06-17T16:51:19Z

allow @kakwok test rights

fwyzard · 2024-06-17T16:52:10Z

@kakwok could you address the small comments I left, and then ask "please test" so the bot can verify the changes ?

kakwok · 2024-06-17T17:06:51Z

@cmsbuild please test

cmsbuild · 2024-06-18T04:53:47Z

Pull request #45199 was updated. @jfernan2, @mandrenguyen can you please check and sign again.

fwyzard · 2024-06-18T06:56:06Z

please test

cmsbuild · 2024-06-18T11:45:38Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-55664a/39932/summary.html
COMMIT: c73ad34
CMSSW: CMSSW_14_1_X_2024-06-17-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45199/39932/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
Reco comparison results: 12 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3345018
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3344992
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 202 log files, 165 edm output root files, 48 DQM output files
TriggerResults: no differences found

mandrenguyen · 2024-06-18T13:48:07Z

The bot is noting a coding rule violation here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-55664a/39932/codeRules/cmsCodeRule1.txt
and a duplicate dictionary warning here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-55664a/39932/dupDict/

Please comment.

fwyzard · 2024-06-18T14:26:13Z

@mandrenguyen thanks for looking into this.

The bot is noting a coding rule violation here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-55664a/39932/codeRules/cmsCodeRule1.txt

Let's call this a false positive.

The bot should warn about using namespace in the global namespace.

What the code is doing here is using namespace hcal inside the ALPAKA_ACCELERATOR_NAMESPACE namespace, which is not an issue.

and a duplicate dictionary warning here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-55664a/39932/dupDict/

This is a real issue, I will look into it.

fwyzard · 2024-06-19T13:29:20Z

and a duplicate dictionary warning here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-55664a/39932/dupDict/

This is a real issue, I will look into it.

Actually after going through CMSSW I cannot find anything called hcal::HcalRecHitSoA or even HcalRecHitSoA.

The CUDA code uses hcal::RecHitCollection<...>, so there are no conflicts there.

fwyzard · 2024-06-19T13:29:46Z

@mandrenguyen I think we can go ahead and merge this.

fwyzard · 2024-06-19T13:31:05Z

@kakwok could you make a PR with a backport to 14.0.9 ?

fwyzard · 2024-06-20T14:31:08Z

@kakwok could you make a PR with a backport to 14.0.9 ?

Done in #45277 .

mandrenguyen · 2024-06-20T14:44:54Z

+1

cmsbuild · 2024-06-20T14:45:14Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @sextonkennedy, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2024-06-20T16:35:08Z

type hcal

antoniovilela · 2024-06-20T22:20:29Z

+1

cmsbuild added this to the CMSSW_14_1_X milestone Jun 11, 2024

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending code-checks-pending labels Jun 11, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 11, 2024

cmsbuild mentioned this pull request Jun 14, 2024

Alpaka implementation of Hcal Local Reconstruction (Mahi) #44910

Merged

cmsbuild added the heterogeneous-pending label Jun 17, 2024

fwyzard reviewed Jun 17, 2024

View reviewed changes

cmsbuild added the allow-kakwok label Jun 17, 2024

cmsbuild added code-checks-pending and removed code-checks-approved labels Jun 17, 2024

cmsbuild removed the tests-pending label Jun 17, 2024

cmsbuild added the code-checks-approved label Jun 18, 2024

cmsbuild added tests-approved and removed tests-started labels Jun 18, 2024

jsamudio mentioned this pull request Jun 18, 2024

Change CaloRecHitProducer to produce new HCAL RecHit SoA #45263

Merged

fwyzard mentioned this pull request Jun 20, 2024

HcalRecHit SoA DataFormats [14.0.x] #45277

Merged

fwyzard mentioned this pull request Jun 20, 2024

Change CaloRecHitProducer to produce new HCAL RecHit SoA [14.0.x] #45278

Merged

cmsbuild added reconstruction-approved fully-signed and removed reconstruction-pending pending-signatures labels Jun 20, 2024

cmsbuild added the hcal label Jun 20, 2024

cmsbuild added orp-approved and removed orp-pending labels Jun 20, 2024

cmsbuild merged commit f2b3a36 into cms-sw:master Jun 20, 2024

cmsbuild mentioned this pull request Jun 21, 2024

Heterogeneous Workflow (CLUE) in Phase2 HLT Simplified Menu #45178

Merged

smuzaffar mentioned this pull request Jun 25, 2024

Protect DQMEventInfo when there is no PoolDBESSource@GlobalTag PSet in the process history #45286

Merged

mmusich mentioned this pull request Jun 25, 2024

Fix lost Dict Def: Add HcalRecHitSoA for HcalRecHit package #45298

Merged

HcalRecHit SoA dataFormat #45199

HcalRecHit SoA dataFormat #45199

Uh oh!

Conversation

kakwok commented Jun 11, 2024

PR description:

PR validation:

Uh oh!

cmsbuild commented Jun 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Jun 11, 2024

Uh oh!

cmsbuild commented Jun 11, 2024

Uh oh!

mandrenguyen commented Jun 17, 2024

Uh oh!

kakwok commented Jun 17, 2024

Uh oh!

fwyzard commented Jun 17, 2024

Uh oh!

cmsbuild commented Jun 17, 2024

Uh oh!

fwyzard Jun 17, 2024

Choose a reason for hiding this comment

Uh oh!

fwyzard Jun 17, 2024

Choose a reason for hiding this comment

Uh oh!

fwyzard Jun 17, 2024

Choose a reason for hiding this comment

Uh oh!

fwyzard commented Jun 17, 2024

Uh oh!

mandrenguyen commented Jun 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fwyzard commented Jun 17, 2024

Uh oh!

fwyzard commented Jun 17, 2024

Uh oh!

fwyzard commented Jun 17, 2024

Uh oh!

kakwok commented Jun 17, 2024

Uh oh!

cmsbuild commented Jun 18, 2024

Uh oh!

fwyzard commented Jun 18, 2024

Uh oh!

cmsbuild commented Jun 18, 2024

Comparison Summary

Uh oh!

mandrenguyen commented Jun 18, 2024

Uh oh!

fwyzard commented Jun 18, 2024

Uh oh!

fwyzard commented Jun 19, 2024

Uh oh!

fwyzard commented Jun 19, 2024

Uh oh!

fwyzard commented Jun 19, 2024

Uh oh!

fwyzard commented Jun 20, 2024

Uh oh!

mandrenguyen commented Jun 20, 2024

Uh oh!

cmsbuild commented Jun 20, 2024

Uh oh!

mandrenguyen commented Jun 20, 2024

Uh oh!

antoniovilela commented Jun 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

cmsbuild commented Jun 11, 2024 •

edited

Loading

mandrenguyen commented Jun 17, 2024 •

edited

Loading