Follow-up tasks for the LST algorithm

[VourMa]

This issue is a simplified version of SegmentLinking#75, meant to list and keep track of the various upcoming developments for the LST algorithm that were either brought up during the PR #45117 review or planned by the development team. The order of the tasks below roughly corresponds to their priority/timescale, as estimated by the developers.

• Integrate accumulated algorithm developments held back during integration PR review (Devel changes that need to be rebased SegmentLinking/cmssw#117).

  More details

  • Update LST README, printouts and minor fixes:

    • Covered partially by Improvements to the README SegmentLinking/cmssw#55. A follow up to accommodate the developments made in Total removal of sdl/SDL SegmentLinking/cmssw#72 will be needed.
    Links to specific comments

    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)

  • Deal with the configurability of the pT threshold:

    • Covered by Add pT Runtime Toggle + pT Toggle for Geometry Inputs SegmentLinking/cmssw#39.
    • CMSSW Integration of LST #45117 (comment)

• LST in HLT

  More details

  • Rebase and merge Single Iteration Patatrack and Single Iteration Patatrack-only-seeded LST in HLT SegmentLinking/cmssw#107.

• Changes in LST conditions data

  More details

  • CMSSW Integration of LST #45117 (comment):
    I would prefer if the possibility of overriding the conditions data setting LST_BASE or TRACKLOOPERDIR would be disabled when building this code inside CMSSW.
  • Remove lst_INF
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
  • CMSSW Integration of LST #45117 (comment):
    LSTESData::geometryDataDir needs a check that getenv("CMSSW_SEARCH_PATH") is not null.

• LST workflow

  More details

  • CMSSW Integration of LST #45117 (comment):
    Respect new relval_Run4.py structure.
  • CMSSW Integration of LST #45117 (comment):
    Decide if we want different workflows for different backends.
  • CMSSW Integration of LST #45117 (comment):
    Implement a "GPU vs CPU" workflow.
  • CMSSW Integration of LST #45117 (comment):
    Fill seed stop info when LST runs (expected by DQM/validation).

• Make general-purpose functions widely available in CMSSW

  More details

  • CMSSW Integration of LST #45117 (comment)

  • CMSSW Integration of LST #45117 (comment)

  • The plan would be to move all general-purpose functions (work started in More work on Constants and removal of vector copies SegmentLinking/cmssw#71):

    • Follow up PR with general functions.
    • Follow up PR moving LST to use those and delete custom implementations.

• Rewrite the loops in the kernels using uniform_elements, independent_groups, independent_group_elements, etc.

  More details

  • CMSSW Integration of LST #45117 (comment):
    Experiment with make_workdiv.

  • CMSSW Integration of LST #45117 (comment):
    No need to set the grid size to 1, the alpaka execution will run the blocks one after the other automatically. You can set the grid size to 1 - the difference will be in how the kernel are executed over the elements. Either way, I think whatever approach you choose should be documented.

  • CMSSW Integration of LST #45117 (review)

    Links to specific comments

    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)

• Rewrite kernels with proper, concrete dimensions instead of templated types

  More details

  • CMSSW Integration of LST #45117 (review):

    Links to specific comments

    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)

• Improvements to the data format interface between LST and CMSSW

  More details

  • CMSSW Integration of LST #45117 (review):
    Define the LST inputs as PortableCollections, and pass them (or their Views) to the LST algorithm.

    Links to specific comments

    • CMSSW Integration of LST #45117 (comment):
      Fill a host SoA directly in LSTPhase2OTHitsInputProducer, copy it to device, and avoid the 6 intermediate copies of the std::vectors.

  • CMSSW Integration of LST #45117 (review):
    Define the LST output as PortableCollections, and pass them (or their Views) to the LST algorithm.

    Links to specific comments

    • CMSSW Integration of LST #45117 (comment):
      Performance-wise best would be to move them in LSTProducer::produce(), that would require the LST::hits() etc to either return a mutable reference, or a value that was moved-from in the hits() method itself.
    • CMSSW Integration of LST #45117 (comment):
      Eventually (or ideally) we'd want to issue the alpaka::memcpy() calls to copy the data from device to host in acquire(), and have the destination host memory accessed only in produce().

• Construct LST condition data from the existing EventSetup info

  More details

  • CMSSW Integration of LST #45117 (review):
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)

• Improvements in storing and using "magic numbers

  More details

  • CMSSW Integration of LST #45117 (review):

    Links to specific comments

    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)
    • CMSSW Integration of LST #45117 (comment)

• Miscellaneous improvements

  More details

  • CMSSW Integration of LST #45117 (comment):
    Try std::binary_search(data, data + ndata, search_val) from C++20.
  • CMSSW Integration of LST #45117 (comment):
    Review uint4 usage.

• Improvements to ESProducer

  More details

  • CMSSW Integration of LST #45117 (comment):
    Make it more clear which members are on the device memory and which are on the host memory. Suggest to consider moving all data that are used only in host code into a separate ES data product.

FYI @slava77 @ariostas

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @VourMa.

@Dr15Jones, @antoniovilela, @makortel, @mandrenguyen, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[makortel]

assign reconstruction, heterogeneous, alca, hlt, upgrade

[makortel]

@cms-sw/tracking-pog-l2

[cmsbuild]

New categories assigned: reconstruction,heterogeneous,alca,hlt,upgrade

@atpathak,@consuegs,@fwyzard,@jfernan2,@makortel,@mandrenguyen,@Martin-Grunewald,@mmusich,@Moanwar,@perrotta,@srimanob,@subirsarkar you have been requested to review this Pull request/Issue and eventually sign? Thanks