Skip to content

[15_0_X] Digi Morphing for HLT #48832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
cmsbuild merged 4 commits into from
Oct 7, 2025
Merged

[15_0_X] Digi Morphing for HLT #48832

cmsbuild merged 4 commits into from
Oct 7, 2025

Conversation

@Chirayu18
Copy link
Contributor

@Chirayu18 Chirayu18 commented Sep 2, 2025 •
edited
Loading

PR description:

Morphing Algorithm for Fake Digi Addition

Backport of #48734 to CMSSW_15_0_X

This PR introduces an alternative alpaka-based algorithm for pixel digi morphing. This implementation is equivalent in principle to the legacy implementation and to the other alpaka-based implementation in PR #48343 but supercedes the latter as it has issues with throughput due to memory usage. Like the other implementations, digi morphing is applied here only to specific detector regions which can be configured as shown below.

Configuration Options

The morphing behavior can be controlled in the EDProducer configuration with the following options:

  • Enable Morphing: 
    DoDigiMorphing = cms.bool(True)  # Default: False
  • Maximum Fakes per Module: 
    MaxFakesInModule = cms.uint32(10000)  # Default: maxPixInModule * 2 / 5

Regional Morphing

Regions can be specified in the configuration as follows:

  • Barrel Regions:
    Use LAYER,LADDER,MODULE coordinates, with support for individual values or ranges (e.g., 1-12 for ladders 1 through 12).

    barrelRegions = cms.vstring(
    "1,1-12,1-2",
    "1,1-12,7-8",
    "2,1-28,1",
    "2,1-28,8"
)  # Default: as shown above

  • Endcap Regions:
    Use DISK,BLADE,SIDE,PANEL coordinates, with support for ranges.

    endcapRegions = cms.vstring(
    # Default: empty (no endcap regions selected)
)

Also includes fixes for digi morphing #49021 backported to CMSSW_15_0_X. These fixes are needed for digi morphing to work properly for differen conditions (when acitve or not):

  • defining a maxPixInModuleForMorphing constant depending on the TrackerTraits to be used to define the number of threads for the FindClus kernel. This also sizes the histogram holding the pixels in a module:
using Hist = cms::alpakatools::HistoContainer<uint16_t,
                                                      TrackerTraits::clusterBinning,
                                                      TrackerTraits::maxPixInModuleForMorphing,
                                                      TrackerTraits::clusterBits,
                                                      uint16_t>;
  • having a different maxIterGPU per topology (given the different number of pixels affects the number of iterations we can use to cover the full module);
  • limiting the maxFakesInModule configuration parameter to take into account the maxPixInModuleForMorphing max to avoid the histogram overflowing.

PR validation:

Validation results can be found here: https://indico.cern.ch/event/1567986/contributions/6647897/attachments/3116797/5527819/HLT_digi_morphing_validation.pdf

@Chirayu18 Chirayu18 marked this pull request as ready for review September 2, 2025 14:14
@cmsbuild cmsbuild added this to the CMSSW_15_0_X milestone Sep 2, 2025
@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2025 •
edited
Loading

A new Pull Request was created by @Chirayu18 for CMSSW_15_0_X.

It involves the following packages:

  • RecoLocalTracker/SiPixelClusterizer (reconstruction)

@cmsbuild, @jfernan2, @mandrenguyen can you please review it and eventually sign? Thanks.
@GiacomoSguazzoni, @VinInn, @VourMa, @dkotlins, @felicepantaleo, @ferencek, @gpetruc, @missirol, @mmusich, @mroguljic, @mtosi, @rovere, @threus, @tsusa, @tvami this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2025 •
edited
Loading

cms-bot internal usage

@mmusich
Copy link
Contributor

mmusich commented Sep 2, 2025

backport of #48734

@Chirayu18 Chirayu18 force-pushed the digimorphing_backport_v2 branch from acc09de to da8306c Compare September 2, 2025 14:48
@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2025

Pull request #48832 was updated. @cmsbuild, @jfernan2, @mandrenguyen can you please check and sign again.

@mmusich
Copy link
Contributor

mmusich commented Sep 3, 2025

enable gpu

@mmusich
Copy link
Contributor

mmusich commented Sep 3, 2025

@cmsbuild, please test

@mmusich
Copy link
Contributor

mmusich commented Sep 3, 2025

@iarspider do we need to nudge the unit tests here too?

@iarspider
Copy link
Contributor

iarspider commented Sep 3, 2025

@mmusich we only have one node with l40s, so need to wait until it's available.

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 4, 2025

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9ad4a1/47954/summary.html
COMMIT: da8306c
CMSSW: CMSSW_15_0_X_2025-09-02-2300/el8_amd64_gcc12
Additional Tests: GPU,AMD_MI300X,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48832/47954/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 6 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4020932
  • DQMHistoTests: Total failures: 10
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4020902
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 218 log files, 189 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

AMD_MI300X Comparison Summary

Summary:

NVIDIA_H100 Comparison Summary

Summary:

NVIDIA_L40S Comparison Summary

Summary:

NVIDIA_T4 Comparison Summary

Summary:

@jfernan2
Copy link
Contributor

jfernan2 commented Sep 4, 2025

assign heterogeneous

@jfernan2
Copy link
Contributor

jfernan2 commented Sep 4, 2025

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 4, 2025

New categories assigned: heterogeneous

@fwyzard,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 5, 2025

Pull request #48832 was updated. @Dr15Jones, @bsunanda, @civanch, @cmsbuild, @fwyzard, @jfernan2, @kpedro88, @makortel, @mandrenguyen, @mdhildreth can you please check and sign again.

@ferencek
Copy link
Contributor

ferencek commented Oct 5, 2025

IIUC this PR needs to be updated with a backport of the commits at #49021.

Done.

@mmusich
Copy link
Contributor

mmusich commented Oct 5, 2025

test parameters:

  • relvals_gpu = 160.03502
  • relvals_opts_gpu = -w gpu

@mmusich
Copy link
Contributor

mmusich commented Oct 5, 2025

@cmsbuild, please test

@ferencek
Copy link
Contributor

ferencek commented Oct 5, 2025

@Chirayu18, can you please update the PR description and mention that it also includes a backport of #49021. Thanks

@mmusich
Copy link
Contributor

mmusich commented Oct 5, 2025

backport of #49021

@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 5, 2025

+1

Size: This PR adds an extra 40KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9ad4a1/48465/summary.html
COMMIT: 0e4b133
CMSSW: CMSSW_15_0_X_2025-10-05-0000/el8_amd64_gcc12
Additional Tests: GPU,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48832/48465/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 2 lines from the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4031982
  • DQMHistoTests: Total failures: 2
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4031960
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 218 log files, 189 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

AMD_MI300X Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 39 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 53486
  • DQMHistoTests: Total failures: 3079
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 50407
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 29 log files, 36 edm output root files, 7 DQM output files
  • TriggerResults: no differences found

AMD_W7900 Comparison Summary

Summary:

NVIDIA_H100 Comparison Summary

Summary:

NVIDIA_L40S Comparison Summary

Summary:

NVIDIA_T4 Comparison Summary

Summary:

@fwyzard
Copy link
Contributor

fwyzard commented Oct 6, 2025

+heterogeneous

These are mostly the same commits as in #48734 and in #49021

@fwyzard
Copy link
Contributor

fwyzard commented Oct 6, 2025

(although the description of the PR should be updated to reflect the changes from #49021)

@jfernan2
Copy link
Contributor

jfernan2 commented Oct 6, 2025

+1

@mandrenguyen
Copy link
Contributor

mandrenguyen commented Oct 6, 2025

urgent
@cms-sw/geometry-l2 please have a look at this backport

@cmsbuild cmsbuild added the urgent label Oct 6, 2025
@mandrenguyen
Copy link
Contributor

mandrenguyen commented Oct 7, 2025

+1
@cms-sw/geometry-l2 please have a look

@mandrenguyen
Copy link
Contributor

mandrenguyen commented Oct 7, 2025

merge
Given the urgency I'm assuming this is ok for @cms-sw/geometry-l2
It's just the declaration of a few variables

@cmsbuild cmsbuild merged commit bf53d59 into cms-sw:CMSSW_15_0_X Oct 7, 2025
24 checks passed
@ferencek ferencek deleted the digimorphing_backport_v2 branch October 7, 2025 08:47
@mandrenguyen
Copy link
Contributor

mandrenguyen commented Oct 7, 2025

@Chirayu18 Is everything included in this PR also present in 15_1_X?

@aneliyaKF aneliyaKF mentioned this pull request Oct 7, 2025
Closed
@mmusich mmusich mentioned this pull request Oct 10, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

backport-ok geometry-pending heterogeneous-approved orp-approved pending-signatures reconstruction-approved tests-approved trk urgent

Projects

None yet

Milestone

CMSSW_15_0_X

Development

Successfully merging this pull request may close these issues.

9 participants

@Chirayu18 @cmsbuild @mmusich @iarspider @jfernan2 @ferencek @fwyzard @mandrenguyen @AdrianoDee