Replace pT3 R-Phi Cuts with DNN #178

GNiendorf · 2025-06-08T22:58:01Z

Replaces the pT3 R-Phi chi-squared cuts with a single cut on the DNN for lower FR, improved efficiency (especially at low-pT), and fewer unmatched pLS's (pLS->pT3 conversion).

GNiendorf · 2025-06-08T22:58:10Z

/run standalone

github-actions · 2025-06-08T23:22:30Z

The PR was built and ran successfully in standalone mode. Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     34.5    438.3    224.8    142.6     70.2    722.9    149.0    186.4    183.8      2.3    2154.7    1397.4+/- 334.1     627.8   explicit[s=4] (target branch)
   avg     32.3    440.6    223.3    133.6     69.7    783.9    139.9    191.4    198.0      2.0    2214.7    1398.5+/- 333.0     647.5   explicit[s=4] (this PR)

slava77 · 2025-06-09T14:33:47Z

please post the breakdown plots (before/after; at least for eta efficiency).

slava77 · 2025-06-09T14:41:20Z

reminding myself that the r-phi chi2 is a part of pt3 DNN. So, it makes sense to not cut on chi2 directly and use the DNN.

Compared to the category-based cut, the layer pattern is not in the DNN. So, perhaps we are missing some information here.

GNiendorf · 2025-06-09T15:05:13Z

Compared to the category-based cut, the layer pattern is not in the DNN. So, perhaps we are missing some information here.

Sure, I can try adding the layer1, layer2, layer3 as inputs to the DNN and see if it improves.

GNiendorf · 2025-06-09T21:12:21Z

/run all

github-actions · 2025-06-09T21:33:49Z

The PR was built and ran successfully in standalone mode. Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     30.9    398.3    197.2    121.5     60.0    669.9    132.9    166.6    170.6      1.8    1949.7    1248.8+/- 304.1     567.0   explicit[s=4] (target branch)
   avg     28.8    395.7    196.6    113.8     59.9    722.2    133.4    170.4    180.4      1.9    2003.1    1252.2+/- 304.8     582.5   explicit[s=4] (this PR)

github-actions · 2025-06-09T23:13:45Z

The PR was built and ran successfully with CMSSW. Here are some plots.

OOTB All Tracks

The full set of validation and comparison plots can be found here.

GNiendorf · 2025-06-10T00:42:17Z

Two slides if you want to flip, 1000 events: eff_comp.pdf

Fewer unmatched pLS's as you can see below, good conversion from pLS's->pT3's.

GNiendorf · 2025-06-10T00:50:22Z

Change in FR, breakdown plots normal PU200. Note that the y-axis changes a bit from the reduced FR.

GNiendorf · 2025-06-10T03:21:33Z

/run all-gpu

github-actions · 2025-06-10T03:43:24Z

The PR was built and ran successfully in standalone mode on GPU. Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
[target branch]
   avg     15.1      0.4      0.3      0.6      0.8      0.2      0.7      0.4      0.8      0.0      19.2       3.9+/-  0.7      19.2   explicit[s=1]
   avg     29.8      0.6      0.4      0.8      1.0      0.3      1.0      0.4      1.1      0.0      35.5       5.4+/-  1.0      17.7   explicit[s=2]
   avg     59.3      0.9      0.8      1.2      1.5      0.4      1.5      0.7      1.9      0.0      68.1       8.4+/-  1.7      17.1   explicit[s=4]
   avg     88.7      1.2      1.0      1.7      2.2      0.6      2.2      0.9      2.5      0.0     101.2      11.9+/-  2.5      16.9   explicit[s=6]
   avg    118.0      1.7      1.4      2.2      2.7      0.7      2.8      1.3      3.1      0.0     134.0      15.3+/-  3.5      16.8   explicit[s=8]
[this PR]
   avg     15.1      0.4      0.3      0.6      0.8      0.2      0.7      0.4      0.8      0.0      19.3       3.9+/-  0.7      19.3   explicit[s=1]
   avg     29.6      0.6      0.4      0.8      1.0      0.3      1.0      0.5      1.2      0.0      35.2       5.4+/-  1.0      17.6   explicit[s=2]
   avg     58.6      0.9      0.7      1.2      1.5      0.4      1.6      0.7      1.9      0.0      67.4       8.4+/-  1.7      16.9   explicit[s=4]
   avg     87.7      1.3      1.1      1.7      2.1      0.6      2.1      0.9      2.7      0.0     100.1      11.9+/-  2.8      16.8   explicit[s=6]
   avg    116.3      1.7      1.4      2.3      2.8      0.7      2.9      1.2      3.3      0.0     132.6      15.6+/-  3.9      16.6   explicit[s=8]

github-actions · 2025-06-10T04:36:46Z

The PR was built and ran successfully with CMSSW on GPU. Here are some plots.

OOTB All Tracks

The full set of validation and comparison plots can be found here.

slava77 · 2025-06-10T12:18:47Z

Change in FR, breakdown plots normal PU200. Note that the y-axis changes a bit from the reduced FR.

what's happening at η~±1.6 in pT3? It's suspiciously localized, the only place where FR goes up.
I'm not sure if there is some degeneracy there (could it be that some fakes that were scattered in neighboring bins are passed in this bin).

GNiendorf · 2025-06-10T13:56:47Z

@slava77 I think it goes back to this issue here, where the DNN performance drops right around abs(eta) 1.6 and afterwards, which is why I need to place a tighter cut (84% signal efficiency) than in the barrel (95% signal efficiency).

slava77

in addition to already mentioned layer pattern information loss, I noticed now that the inward chi2 is lost as well.

It would be good to check if this is a useful variable.

From a separate chat earlier today it sounded like this one is urgent, although from a meeting yesterday it sounded like some developments are still to be included.
If updates are coming in a few days later, perhaps it's better to combine.
Please clarify.

RecoTracker/LSTCore/src/alpaka/NeuralNetwork.h

RecoTracker/LSTCore/src/alpaka/PixelTriplet.h

slava77 · 2025-06-11T13:51:57Z

RecoTracker/LSTCore/src/alpaka/PixelTriplet.h

      rPhiChiSquaredInwards = computePT3RPhiChiSquaredInwards(g, f, tripletRadius, xPix, yPix);
-      if (runChiSquaredCuts && pixelSegmentPt < 5.0f) {
-        if (!passPT3RPhiChiSquaredInwardsCuts(
-                modules, lowerModuleIndex, middleModuleIndex, upperModuleIndex, rPhiChiSquaredInwards))


ah, rPhiChiSquaredInwards would be another variable not used in the dnn

Yes, I tried to add it a while ago but it didn't add any performance. It is still included in the "score" variable however, which is why I had to leave it for now.

RecoTracker/LSTCore/src/alpaka/NeuralNetwork.h

RecoTracker/LSTCore/interface/alpaka/Common.h

GNiendorf · 2025-06-11T14:33:41Z

in addition to already mentioned layer pattern information loss, I noticed now that the inward chi2 is lost as well.

It would be good to check if this is a useful variable.

From a separate chat earlier today it sounded like this one is urgent, although from a meeting yesterday it sounded like some developments are still to be included. If updates are coming in a few days later, perhaps it's better to combine. Please clarify.

Thank you for the review comments! I'm not in a rush, I would like to understand at least a little better what's going on for eta > 1.6 and other small things (including layer numbers as suggested).

GNiendorf · 2025-06-11T19:37:30Z

@slava77 Here are the layer distributions btw. Working on adding these as inputs to the DNN. Low score means a real track with DNN score < 0.5.

GNiendorf · 2025-09-07T19:18:39Z

/run standalone

GNiendorf · 2025-09-07T19:19:04Z

/run cmssw

github-actions · 2025-09-07T19:41:23Z

The PR was built and ran successfully in standalone mode. Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     29.2    377.8    274.5    128.6     61.9    717.8    131.8    143.1    180.8      1.5    2047.0    1299.9+/- 312.4     595.3   explicit[s=4] (target branch)
   avg     29.4    384.0    273.1    122.2     61.3    735.6    130.6    143.1    184.8      1.4    2065.4    1300.4+/- 311.0     602.6   explicit[s=4] (this PR)

github-actions · 2025-09-07T21:32:30Z

The PR was built and ran successfully with CMSSW. Here are some plots.

OOTB All Tracks

The full set of validation and comparison plots can be found here.

GNiendorf · 2025-09-17T21:24:14Z

/run all

github-actions · 2025-09-17T21:31:38Z

There was a problem while building and running in standalone mode. The logs can be found here.

GNiendorf · 2025-10-03T20:44:54Z

/run gpu-all

github-actions · 2025-10-03T20:46:07Z

There was a problem while building and running in standalone mode on GPU. The logs can be found here.

github-actions · 2025-10-03T20:50:00Z

There was a problem while building and running with CMSSW on GPU. The logs can be found here.

GNiendorf · 2025-10-04T19:46:01Z

/run all

github-actions · 2025-10-04T19:55:13Z

There was a problem while building and running in standalone mode. The logs can be found here.

github-actions · 2025-10-04T20:04:52Z

There was a problem while building and running with CMSSW. The logs can be found here.

GNiendorf · 2025-10-05T19:56:39Z

/run all

github-actions · 2025-10-05T20:02:53Z

There was a problem while building and running in standalone mode. The logs can be found here.

github-actions · 2025-10-05T20:22:51Z

There was a problem while building and running with CMSSW. The logs can be found here.

GNiendorf · 2025-10-06T12:58:56Z

/run all

github-actions · 2025-10-06T13:09:04Z

There was a problem while building and running in standalone mode. The logs can be found here.

github-actions · 2025-10-06T13:14:15Z

There was a problem while building and running with CMSSW. The logs can be found here.

ariostas · 2025-10-06T14:58:15Z

Hmm it's weird that the latest IB is still CMSSW_16_0_X_2025-09-28. Hopefully they update that soon.

slava77 · 2025-10-06T15:26:00Z

Hmm it's weird that the latest IB is still CMSSW_16_0_X_2025-09-28. Hopefully they update that soon.

I think that this is related to the gcc13 switch.

slava77 · 2025-10-06T15:28:07Z

need to use el8_amd64_gcc13

GNiendorf · 2025-10-07T14:51:08Z

/run all

github-actions · 2025-10-07T14:54:50Z

There was a problem while building and running in standalone mode. The logs can be found here.

ariostas · 2025-10-07T15:00:30Z

I restarted the CI. Hopefully it works now. Thank you, @slava77!

github-actions · 2025-10-07T15:25:29Z

The PR was built and ran successfully in standalone mode. Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     28.3    373.4    275.5    120.1     44.5    685.9    121.6    126.4    167.9      1.4    1944.9    1230.7+/- 290.2     595.9   explicit[s=4] (target branch)
   avg     30.9    365.1    269.5    119.7     44.4    850.1    121.5    127.6    190.3      1.5    2120.6    1239.5+/- 293.0     638.2   explicit[s=4] (this PR)

GNiendorf · 2025-10-07T15:53:34Z

/run gpu-all

github-actions · 2025-10-07T17:10:53Z

The PR was built and ran successfully with CMSSW. Here are some plots.

OOTB All Tracks

The full set of validation and comparison plots can be found here.

github-actions · 2025-10-07T18:19:16Z

The PR was built and ran successfully with CMSSW on GPU. Here are some plots.

OOTB All Tracks

The full set of validation and comparison plots can be found here.

github-actions · 2025-10-07T18:36:15Z

The PR was built and ran successfully in standalone mode on GPU. Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
[target branch]
   avg     15.2      0.4      0.4      0.5      0.6      0.3      0.6      0.3      0.9      0.0      19.4       3.8+/-  0.8      19.4   explicit[s=1]
   avg      0.9      0.6      0.6      0.7      0.8      0.3      1.0      0.5      1.2      0.0       6.6       5.4+/-  1.1       3.3   explicit[s=2]
   avg      1.5      0.9      1.0      1.2      1.2      0.4      1.5      0.7      1.9      0.0      10.4       8.5+/-  1.7       2.7   explicit[s=4]
   avg      2.2      1.3      1.5      1.7      1.8      0.6      2.1      0.9      2.6      0.0      14.7      11.9+/-  2.5       2.5   explicit[s=6]
   avg      2.9      1.7      1.9      2.3      2.5      0.7      2.8      1.2      3.2      0.0      19.4      15.8+/-  3.6       2.5   explicit[s=8]
[this PR]
   avg     15.1      0.4      0.4      0.5      0.6      0.3      0.6      0.4      0.9      0.0      19.2       3.9+/-  0.8      19.2   explicit[s=1]
   avg      0.9      0.6      0.6      0.7      0.8      0.3      1.0      0.5      1.3      0.0       6.6       5.4+/-  1.1       3.3   explicit[s=2]
   avg      1.5      0.8      1.0      1.2      1.2      0.4      1.6      0.7      2.0      0.0      10.4       8.5+/-  1.9       2.7   explicit[s=4]
   avg      2.1      1.2      1.5      1.7      1.8      0.6      2.2      1.0      2.6      0.0      14.8      12.1+/-  3.0       2.5   explicit[s=6]
   avg      2.9      1.7      2.1      2.4      2.5      0.7      2.6      1.1      3.2      0.0      19.1      15.6+/-  2.8       2.4   explicit[s=8]

GNiendorf · 2025-10-07T20:39:01Z

/run standalone lowpt

github-actions · 2025-10-07T21:19:39Z

The PR was built and ran successfully in standalone mode (low pT setup). Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     31.5    371.2    814.9    450.2    173.4   1526.9    279.1    543.6    379.0      3.2    4573.1    3014.7+/- 842.7    1264.4   explicit[s=4] (target branch)
   avg     34.2    372.8    812.2    448.6    173.7   1883.4    280.7    550.2    438.8      3.2    4997.9    3080.2+/- 861.9    1371.6   explicit[s=4] (this PR)

GNiendorf · 2025-10-08T01:49:55Z

/run CMSSW lowpt

github-actions · 2025-10-08T02:38:08Z

There was a problem while building and running with CMSSW. The logs can be found here.

GNiendorf · 2025-10-08T14:18:45Z

@slava77 low-pT CMSSW CI fail is independent of this PR btw.

slava77 · 2025-10-08T23:21:46Z

there is now some repetition in RecoTracker/LSTCore/standalone/code/core/write_lst_ntuple.cc, but it's probably OK, especially if the goal is to maintain a fast variant working for the pT3 DNN case specifically.

@GNiendorf
For the CMSSW PR please add an update of your slides from Sep 19 covering just this PR.
The plots by category are quite informative.

GNiendorf marked this pull request as ready for review June 9, 2025 23:19

slava77 reviewed Jun 11, 2025

View reviewed changes

GNiendorf force-pushed the no_rphi branch from 51981fd to ccdbe78 Compare September 7, 2025 18:20

GNiendorf force-pushed the no_rphi branch 4 times, most recently from fe4f368 to 0eb0efb Compare September 17, 2025 21:23

github-actions bot merged commit e8e155a into master Oct 17, 2025
6 of 7 checks passed

Replace pT3 R-Phi Cuts with DNN #178

Replace pT3 R-Phi Cuts with DNN #178

Uh oh!

Conversation

GNiendorf commented Jun 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GNiendorf commented Jun 8, 2025

Uh oh!

github-actions bot commented Jun 8, 2025

Uh oh!

slava77 commented Jun 9, 2025

Uh oh!

slava77 commented Jun 9, 2025

Uh oh!

GNiendorf commented Jun 9, 2025

Uh oh!

GNiendorf commented Jun 9, 2025

Uh oh!

github-actions bot commented Jun 9, 2025

Uh oh!

github-actions bot commented Jun 9, 2025

Uh oh!

GNiendorf commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GNiendorf commented Jun 10, 2025

Uh oh!

GNiendorf commented Jun 10, 2025

Uh oh!

github-actions bot commented Jun 10, 2025

Uh oh!

github-actions bot commented Jun 10, 2025

Uh oh!

slava77 commented Jun 10, 2025

Uh oh!

GNiendorf commented Jun 10, 2025

Uh oh!

slava77 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

slava77 Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

GNiendorf Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

GNiendorf commented Jun 11, 2025

Uh oh!

GNiendorf commented Jun 11, 2025

Uh oh!

GNiendorf commented Sep 7, 2025

Uh oh!

GNiendorf commented Sep 7, 2025

Uh oh!

github-actions bot commented Sep 7, 2025

Uh oh!

github-actions bot commented Sep 7, 2025

Uh oh!

GNiendorf commented Sep 17, 2025

Uh oh!

github-actions bot commented Sep 17, 2025

Uh oh!

GNiendorf commented Oct 3, 2025

Uh oh!

github-actions bot commented Oct 3, 2025

Uh oh!

github-actions bot commented Oct 3, 2025

Uh oh!

GNiendorf commented Oct 4, 2025

Uh oh!

github-actions bot commented Oct 4, 2025

Uh oh!

github-actions bot commented Oct 4, 2025

Uh oh!

GNiendorf commented Oct 5, 2025

GNiendorf commented Jun 8, 2025 •

edited

Loading

GNiendorf commented Jun 10, 2025 •

edited

Loading