Improve the memory usage in the alpaka pixel reconstruction [14.0.x] by fwyzard · Pull Request #44459 · cms-sw/cmssw

fwyzard · 2024-03-18T22:10:21Z

PR description:

Fix the TrackingRecHitsDevice constructor:
Fix the hitsModuleStart copy to use the correct source (device-to-device instead of host-to-device).
Improve the offsetBPIX2 copy and use the data member as source, to guarantee the lifetime of the source.
Remove the queue synchronisation.
Remove an unused variable from SiPixelRawToCluster::acquire().
Improve memory usage in SiPixelRawToCluster::acquire():
Allocate device memory only when actually used.
Use cached memory buffers in WordFedAppender.

PR validation:

Run the HLT 2024 v1.0 menu with the changes.

Backport status

Backported of #44458 to CMSSW 14.0.x for data taking.

fwyzard · 2024-03-18T22:10:26Z

type bugfix

fwyzard · 2024-03-18T22:10:29Z

enable gpu

fwyzard · 2024-03-18T22:10:32Z

please test

cmsbuild · 2024-03-18T22:10:45Z

A new Pull Request was created by @fwyzard for CMSSW_14_0_X.

It involves the following packages:

DataFormats/TrackingRecHitSoA (heterogeneous, reconstruction)
RecoLocalTracker/SiPixelClusterizer (reconstruction)

@fwyzard, @jfernan2, @mandrenguyen, @makortel can you please review it and eventually sign? Thanks.
@tsusa, @threus, @gpetruc, @dkotlins, @mtosi, @rovere, @JanFSchulte, @mroguljic, @mmusich, @ferencek, @felicepantaleo, @VinInn, @missirol, @GiacomoSguazzoni, @VourMa, @tvami this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

Backported from Improve the memory usage in the alpaka pixel reconstruction #44458

cmsbuild · 2024-03-18T22:10:45Z

cms-bot internal usage

Fix the hitsModuleStart copy to use the correct source (device-to-device instead of host-to-device). Improve the offsetBPIX2 copy and use the data member as source, to guarantee the lifetime of the source. Remove the queue synchronisation. Update the unit test accordingly.

Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>

Allocate device memory only when actually used. Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>

Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>

cmsbuild · 2024-03-18T23:27:28Z

Pull request #44459 was updated. @cmsbuild, @fwyzard, @makortel, @jfernan2, @mandrenguyen can you please check and sign again.

fwyzard · 2024-03-18T23:27:29Z

please test

fwyzard · 2024-03-18T23:30:53Z

These changes improve the throughput of the HLT 2024 v1.0 menu with the Pixel alpaka customisation from 587 ev/s (-3%) to 594 ev/s (-2%), compared with the CUDA version that achieves 606 ev/s.

cmsbuild · 2024-03-19T02:39:47Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-531d99/38243/summary.html
COMMIT: 30162d7
CMSSW: CMSSW_14_0_X_2024-03-18-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44459/38243/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 3 lines to the logs
Reco comparison results: 40 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3346212
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3346190
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 205 log files, 166 edm output root files, 49 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 35 differences found in the comparisons
DQMHistoTests: Total files compared: 3
DQMHistoTests: Total histograms compared: 39740
DQMHistoTests: Total failures: 1058
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 38682
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
Checked 8 log files, 10 edm output root files, 3 DQM output files
TriggerResults: no differences found

mandrenguyen · 2024-03-19T10:45:58Z

backport of #44458

mandrenguyen · 2024-03-19T10:46:38Z

+1

cmsbuild · 2024-03-20T14:03:07Z

Pull request #44459 was updated. @makortel, @fwyzard, @cmsbuild can you please check and sign again.

fwyzard · 2024-03-20T14:04:44Z

please test

cmsbuild · 2024-03-20T17:47:00Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-531d99/38303/summary.html
COMMIT: d857392
CMSSW: CMSSW_14_0_X_2024-03-20-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44459/38303/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 7 lines from the logs
Reco comparison results: 66 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3346212
DQMHistoTests: Total failures: 1358
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3344832
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 205 log files, 166 edm output root files, 49 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 35 differences found in the comparisons
DQMHistoTests: Total files compared: 3
DQMHistoTests: Total histograms compared: 39740
DQMHistoTests: Total failures: 1057
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 38683
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
Checked 8 log files, 10 edm output root files, 3 DQM output files
TriggerResults: no differences found

fwyzard · 2024-03-21T16:27:13Z

+heterogeneous

cmsbuild · 2024-03-21T16:27:36Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_14_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_14_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

antoniovilela · 2024-03-22T09:36:52Z

+1

cmsbuild added this to the CMSSW_14_0_X milestone Mar 18, 2024

cmsbuild added reconstruction-pending pending-signatures orp-pending tests-started bug-fix heterogeneous-pending tracking trk labels Mar 18, 2024

fwyzard mentioned this pull request Mar 18, 2024

Improve the memory usage in the alpaka pixel reconstruction #44458

Merged

fwyzard and others added 4 commits March 19, 2024 00:26

Remove unused variable from SiPixelRawToCluster::acquire()

8e6e68e

Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>

Improve memory usage in SiPixelRawToCluster::acquire()

a40c65e

Allocate device memory only when actually used. Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>

Use cached memory buffers in WordFedAppender

30162d7

Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>

fwyzard force-pushed the improve_alpaka_Pixel_memory_usage_140x branch from ba5d4ab to 30162d7 Compare March 18, 2024 23:27

cmsbuild added tests-pending and removed tests-started labels Mar 18, 2024

cmsbuild added tests-started and removed tests-pending labels Mar 18, 2024

cmsbuild added tests-approved and removed tests-started labels Mar 19, 2024

cmsbuild added the backport label Mar 19, 2024

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 19, 2024

Update OneToManyAssocRandomAccess to use the preferred warp size

d857392

cmsbuild added tests-pending and removed tests-approved labels Mar 20, 2024

cmsbuild added tests-started and removed tests-pending labels Mar 20, 2024

cmsbuild added tests-approved and removed tests-started labels Mar 20, 2024

cmsbuild added fully-signed heterogeneous-approved and removed pending-signatures heterogeneous-pending labels Mar 21, 2024

cmsbuild added backport-ok and removed backport labels Mar 21, 2024

cmsbuild added orp-approved and removed orp-pending labels Mar 22, 2024

cmsbuild merged commit d6c7df2 into cms-sw:CMSSW_14_0_X Mar 22, 2024

fwyzard deleted the improve_alpaka_Pixel_memory_usage_140x branch April 7, 2024 10:02

Conversation

fwyzard commented Mar 18, 2024

PR description:

PR validation:

Backport status

Uh oh!

fwyzard commented Mar 18, 2024

Uh oh!

fwyzard commented Mar 18, 2024

Uh oh!

fwyzard commented Mar 18, 2024

Uh oh!

cmsbuild commented Mar 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Mar 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Mar 18, 2024

Uh oh!

fwyzard commented Mar 18, 2024

Uh oh!

fwyzard commented Mar 18, 2024

Uh oh!

cmsbuild commented Mar 19, 2024

Comparison Summary

GPU Comparison Summary

Uh oh!

mandrenguyen commented Mar 19, 2024

Uh oh!

mandrenguyen commented Mar 19, 2024

Uh oh!

cmsbuild commented Mar 20, 2024

Uh oh!

fwyzard commented Mar 20, 2024

Uh oh!

cmsbuild commented Mar 20, 2024

Comparison Summary

GPU Comparison Summary

Uh oh!

fwyzard commented Mar 21, 2024

Uh oh!

cmsbuild commented Mar 21, 2024

Uh oh!

antoniovilela commented Mar 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cmsbuild commented Mar 18, 2024 •

edited

Loading

cmsbuild commented Mar 18, 2024 •

edited

Loading