Skip to content

Conversation

@sbaldu
Copy link
Contributor

@sbaldu sbaldu commented Jul 22, 2025

PR description:

This PR implements helper functions and type traits to handle the behaviours required by the different column types without the need to use boost conditions, thus reducing the use of macros and improving readability.
There are some helper functions that are marked as TODOs, as they are waiting for further developments in the backend of the SoA Layout class.

PR validation:

All the tests still pass, as the behaviour is unchanged.

FYI @leobeltra @fwyzard @felicepantaleo

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 22, 2025

cms-bot internal usage

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48607/45574

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @sbaldu for master.

It involves the following packages:

  • DataFormats/SoATemplate (heterogeneous)

@cmsbuild, @fwyzard, @makortel can you please review it and eventually sign? Thanks.
@missirol, @mmusich, @rovere this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48607/45580

@cmsbuild
Copy link
Contributor

Pull request #48607 was updated. @cmsbuild, @fwyzard, @makortel can you please check and sign again.

@makortel
Copy link
Contributor

enable gpu

@makortel
Copy link
Contributor

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0c482d/47361/summary.html
COMMIT: 66e461b
CMSSW: CMSSW_15_1_X_2025-07-23-1100/el8_amd64_gcc12
Additional Tests: CUDA,ROCM
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48607/47361/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4076481
  • DQMHistoTests: Total failures: 65
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4076396
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

CUDA Comparison Summary

Summary:

ROCM Comparison Summary

Summary:

@fwyzard
Copy link
Contributor

fwyzard commented Jul 24, 2025

The changes look OK to me, from what I can gleam of the old macros.
@makortel do you want to have a look as well ?

@sbaldu @leobeltra could you post the fully expanded code of a sample SoA before and after these changes, to have a better look at what is the impact ?

@makortel
Copy link
Contributor

@makortel do you want to have a look as well ?

I don't have further comments

@fwyzard
Copy link
Contributor

fwyzard commented Jul 24, 2025

OK, thanks.

@cmsbuild
Copy link
Contributor

Pull request #48607 was updated. @cmsbuild, @fwyzard, @makortel can you please check and sign again.

Comment on lines 859 to 861
// Helper functions for accumulating column elements
template <typename ColumnType>
struct AccumulateSoAElements;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly the implementation, these do not accumulate "column elements" but "column size in bytes" ?
Then I would suggest a more accurate name, like AccumulateByteSize ?

Copy link
Contributor

@fwyzard fwyzard Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, could you move these and any other names that users should not be aware of to cms::soa::impl or something similar ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a curiosity - why computePitch are functions and AccumulateSoAElements are structs ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a curiosity - why computePitch are functions and AccumulateSoAElements are structs ?

That is because the computation is called inside computeDataSize, which is a static method, so we couldn't call a non-static function inside of it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I though that AccumulateSoAElements are structs and computePitch are free functions - none of them are methods ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes they are free functions, what I meant is that when I call functions like computePitch I pass the column member to resolve the overload, but inside computeDataSize I couldn't do this because I couldn't access the data members of the Layout through this.
So I had to diversify the behaviour for the different types of columns based on their type, and since I can't partially specialize a free function I used a struct.

Copy link
Contributor

@fwyzard fwyzard Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I admit to being confused...

Still, can you

  • rename AccumulateSoAElements to ComputeSoAElementByteSize
  • make the operator() method static (maybe rename it for convenience)
  • add a free function interface like computeSoAElementByteSize(...) that calls ComputeSoAElementByteSize::operator()(...)

?

Would that be doable ?

Use template functions instead of macro `SWITCH_ON_TYPE`s

Put template utilities inside `cms::soa::detail` namespace
@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48607/45906

@cmsbuild
Copy link
Contributor

Pull request #48607 was updated. @cmsbuild, @fwyzard, @makortel can you please check and sign again.

@fwyzard
Copy link
Contributor

fwyzard commented Aug 26, 2025

please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 40KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0c482d/47868/summary.html
COMMIT: 3e41c72
CMSSW: CMSSW_15_1_X_2025-08-26-1100/el8_amd64_gcc12
Additional Tests: CUDA,ROCM
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48607/47868/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 7 lines from the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4113272
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4113246
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

CUDA Comparison Summary

Summary:

ROCM Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 53486
  • DQMHistoTests: Total failures: 9441
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 44045
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 24 log files, 30 edm output root files, 7 DQM output files
  • TriggerResults: no differences found

@fwyzard
Copy link
Contributor

fwyzard commented Aug 26, 2025

+heterogeneous

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @mandrenguyen, @ftenchini (and backports should be raised in the release meeting by the corresponding L2)

@mandrenguyen
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit f2a060b into cms-sw:master Aug 27, 2025
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants