Update high purity ID and simplify HLT LST configurations

[VourMa]

This PR updates the high purity ID with two passthrough options:
a) Passthrough for tracks without pixel tracker hits. The only requirement for these tracks is that they have at least 4 hits (that covers T5s with a potential loss of a layer during fitting). The latter requirement is configurable.
b) Passthrough for all tracks. No requirement is applied on the tracks. This is exploited in the single iteration configurations running LST seeding, as these configurations have a very low fake+duplicate. Running the high purity module unifies the configuration and enables the correct usage in downstream objects, in case they require tracks with high purity ID.

The above updates enable a large simplification of the HLT tracking configurations, which is done also in this PR. Below, the relevant procModifier combinations have been validated. The current configuration is in blue and the configuration proposed in this PR is in red:

1. trackingLST, falling under option a) above

2. trackingLST,seedingLST, falling under option a) above

3. singleIterPatatrack,phase2CAExtension,trackingLST, falling under option a) above

4. singleIterPatatrack,phase2CAExtension,trackingLST,seedingLST, falling under option b) above

5. phase2CAExtension,singleIterPatatrack,trackingLST,seedingLST,trackingMkFitCommon,hltTrackingMkFitInitialStep, falling under option b) above

Comments on the physics perfomance:

• Configurations 1, 2, 3 utilize the TrackListMerger module in a different way (cleaning tracks within the same collection instead of separate collections). This leads to a small increase of the efficiency and either some small decrease of fake rate (for the two iteration setup) or some increase of fake+duplicate rate (in the single iteration setup, where it still remains below 2.5% in any η range).
• Configurations 4, 5 remain unchanged: before they were using built tracks in the general track collection, while they now use the high purity ID tracks, with the passthrough option enabled, in the general track collection, and these two configurations are fully equivalent.

The corresponding presentation can be found in:
https://indico.cern.ch/event/1615436/#2-on-cmssw-pr-49382-update-hig

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49382/46807

• There are other open Pull requests which might conflict with changes you have proposed:
  • File DQM/Integration/python/clients/beam_dqm_sourceclient-live_cfg.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
  • File HLTrigger/Configuration/python/HLT_75e33/modules/hltInitialStepTracks_cfi.py modified in PR(s): [mkFit] final fit initial PR and testing workflow #49339
  • File HLTrigger/Configuration/python/HLT_75e33/sequences/HLTInitialStepSequence_cfi.py modified in PR(s): [mkFit] final fit initial PR and testing workflow #49339

[cmsbuild]

A new Pull Request was created by @VourMa for master.

It involves the following packages:

• Configuration/EventContent (operations)
• DQM/Integration (dqm)
• DQMOffline/Trigger (dqm)
• HLTrigger/Configuration (hlt)
• RecoTracker/FinalTrackSelectors (reconstruction)
• Validation/RecoTrack (dqm)

@Martin-Grunewald, @cmsbuild, @ctarricone, @davidlange6, @fabiocos, @ftenchini, @gabrielmscampos, @jfernan2, @mandrenguyen, @mmusich, @nothingface0, @rseidita, @srimanob can you please review it and eventually sign? Thanks.
@Fedespring, @GiacomoSguazzoni, @HuguesBrun, @Martin-Grunewald, @SohamBhattacharya, @VinInn, @VourMa, @batinkov, @cericeci, @dgulhan, @elusian, @fabiocos, @felicepantaleo, @francescobrivio, @gpetruc, @jhgoh, @missirol, @mmasciov, @mmusich, @mtosi, @rociovilar, @rovere, @threus, @trocino, @wmtford this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[mmusich]

test parameters:

• enable = hlt_p2_integration, hlt_p2_timing
• workflows = ph2_hlt

[mmusich]

@cmsbuild, please test

[cmsbuild]

+1

Size: This PR adds an extra 80KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8dbabd/49433/summary.html
COMMIT: 4f2c71e
CMSSW: CMSSW_16_0_X_2025-11-12-2300/el8_amd64_gcc13
Additional Tests: HLT_P2_INTEGRATION,HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/49382/49433/install.sh to create a dev area with all the needed externals and cmssw changes.

HLT P2 Timing: chart

Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 2 differences found in the comparisons
• Reco comparison had 2 failed jobs
• DQMHistoTests: Total files compared: 70
• DQMHistoTests: Total histograms compared: 4581810
• DQMHistoTests: Total failures: 3704
• DQMHistoTests: Total nulls: 32
• DQMHistoTests: Total successes: 4578054
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -10569.804 KiB( 69 files compared)
• DQMHistoSizes: changed ( 29634.754,... ): -3525.225 KiB HLT/Tracking
• DQMHistoSizes: changed ( 29634.7571,... ): -1759.677 KiB HLT/Tracking
• Checked 283 log files, 241 edm output root files, 70 DQM output files
• TriggerResults: found differences in 4 / 68 workflows

[mmusich]

@VourMa thanks for the simplification of the configuration.
I am not sure to appreciate all the ramifications of the changes proposed, in particular the removal of the monitoring of the intermediate LST collections.
May I ask you for a dedicated discussion either at a tracking POG or a phase-2 HLT upgrade meeting?

[VourMa]

@VourMa thanks for the simplification of the configuration. I am not sure to appreciate all the ramifications of the changes proposed, in particular the removal of the monitoring of the intermediate LST collections. May I ask you for a dedicated discussion either at a tracking POG or a phase-2 HLT upgrade meeting?

Sure, I will arrange a presentation in the TRK POG meeting next week, and we can discuss there.

[VourMa]

@VourMa thanks for the simplification of the configuration. I am not sure to appreciate all the ramifications of the changes proposed, in particular the removal of the monitoring of the intermediate LST collections. May I ask you for a dedicated discussion either at a tracking POG or a phase-2 HLT upgrade meeting?

@mmusich I have requested the slot for the presentation and I am waiting the confirmation by the TRK conveners. I will post the link here and in the description when this is done.

In the meantime, this is becoming a blocking piece for other developments that we have. In the interest of time, would you like to elaborate on your concerns?
The way I see this, this is reinstating the regular validation for track collections (which is not breaking it up in collections of different track lengths). I see the benefit of monitoring more but this should perhaps be done in a separate DQM workflow, without complicating the actual reconstruction sequences. Did you have something else in mind?
That said, we can open a broader discussion with the TRK POG about how much we need to break up the different tracking reconstruction parts to monitor them, but this goes beyond the scope of this PR.

[slava77]

@cmsbuild please test

[cmsbuild]

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8dbabd/49563/summary.html
COMMIT: 178b635
CMSSW: CMSSW_16_0_X_2025-11-19-1100/el8_amd64_gcc13
Additional Tests: HLT_P2_INTEGRATION,HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/49382/49563/install.sh to create a dev area with all the needed externals and cmssw changes.

HLT P2 Timing: chart

Comparison Summary

Summary:

• You potentially added 5 lines to the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 7 differences found in the comparisons
• DQMHistoTests: Total files compared: 71
• DQMHistoTests: Total histograms compared: 4627740
• DQMHistoTests: Total failures: 5887
• DQMHistoTests: Total nulls: 35
• DQMHistoTests: Total successes: 4621798
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -12329.481 KiB( 70 files compared)
• DQMHistoSizes: changed ( 29634.754,... ): -3525.225 KiB HLT/Tracking
• DQMHistoSizes: changed ( 29634.7571,... ): -1759.677 KiB HLT/Tracking
• Checked 286 log files, 243 edm output root files, 71 DQM output files
• TriggerResults: found differences in 5 / 69 workflows

[VourMa]

A presentation about this PR will be given in:
https://indico.cern.ch/event/1615436/#2-on-cmssw-pr-49382-update-hig

[mmusich]

+hlt

• the goal of the PR was clarified in the Tracking POG working meeting on 24th Nov
• a possible improved way to check the tracks populating specific phase-spaces (along the lines of what has been suggested at Update high purity ID and simplify HLT LST configurations #49382 (comment)) might follow along in the coming weeks
• @VourMa will check explicitly that wf XXXXX.775: (HLT phase-2 NGT Scouting menu Alpaka variant, with Pixeltracks CA Extension + LST T5s as GeneralTracks) still has the same performance (for some reason the DQM bin-by-bin comparisons above are empty now).

[mmasciov]

@cms-sw/reconstruction-l2, @cms-sw/dqm-l2 and @cms-sw/orp-l2: it would be great if this could be included in pre3 already, to allow for testing by other POGs.

[jfernan2]

+1

[VourMa]

• @VourMa will check explicitly that wf XXXXX.775: (HLT phase-2 NGT Scouting menu Alpaka variant, with Pixeltracks CA Extension + LST T5s as GeneralTracks) still has the same performance (for some reason the DQM bin-by-bin comparisons above are empty now).

For completeness, here is the efficiency and fake+duplicate rate in CMSSW_16_0_X_2025-11-23-2300 (blue) and in CMSSW_16_0_X_2025-11-23-2300 with this PR merged on top (red) for the procModifier combination ngtScouting,phase2CAExtension,trackingLST (i.e. equivalent to wf .775). As expected, no changes.

[gabrielmscampos]

+dqm

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @sextonkennedy, @ftenchini (and backports should be raised in the release meeting by the corresponding L2)

[ftenchini]

+1

[mmusich]

My point was that monitoring by selecting the tracks with 0 pixel hits should work.

Specific to the per-iteration: there is no "T5" iteration in LST, it's just candidates without pixel hits.

I have a proposal at #49517.