Add a 2023 data workflow for all patatrack reconstruction using the `gpuValidation` modifier #42674

mmusich · 2023-08-28T16:44:40Z

PR description:

As pointed out in #42631 (comment) it seems there is no data relval workflow that runs the patatrack reconstruction in CPU and GPU and does the comparison between the two (in short using the gpuValidation modifier).
I add a workflow here that does that.

PR validation:

Run runTheMatrix.py --what gpu -l 141.008583 successfully.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Perhaps to be backported to 13.2.X

cmsbuild · 2023-08-28T16:50:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42674/36749

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_gpu.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Custom_Muon_POG_NanoAOD #42039, extend PU Flat55To75 to 2024 workflows #42653

cmsbuild · 2023-08-28T16:51:15Z

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)

@bbilin, @srimanob, @cmsbuild, @AdrianoDee, @miquork, @sunilUIET can you please review it and eventually sign? Thanks.
@makortel, @missirol, @slomeo, @fabiocos, @Martin-Grunewald this is something you requested to watch as well.
@perrotta, @dpiparo, @antoniovilela, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Configuration/PyReleaseValidation/python/relval_steps.py

cmsbuild · 2023-08-28T16:55:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42674/36751

This PR adds an extra 56KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_gpu.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Custom_Muon_POG_NanoAOD #42039, extend PU Flat55To75 to 2024 workflows #42653

cmsbuild · 2023-08-28T16:56:15Z

Pull request #42674 was updated. @bbilin, @srimanob, @cmsbuild, @AdrianoDee, @miquork, @sunilUIET can you please check and sign again.

mmusich · 2023-08-29T06:32:17Z

test parameters:

enable = gpu
workflows_gpu = 141.008583

mmusich · 2023-08-29T06:32:23Z

@cmsbuild, please test

mmusich · 2023-08-29T07:23:56Z

test parameters:

enable = gpu
workflows_gpu = 141.008583

cmsbuild · 2023-08-29T07:26:40Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42674/36762

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_gpu.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Custom_Muon_POG_NanoAOD #42039, extend PU Flat55To75 to 2024 workflows #42653

mmusich · 2023-09-14T09:33:00Z

test parameters:

enable = gpu
workflows_gpu = 141.008583, 160.03502

cmsbuild · 2023-09-14T09:39:29Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42674/36898

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_gpu.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Custom_Muon_POG_NanoAOD #42039, add run3_miniAOD_12X era modifier #42740

cmsbuild · 2023-09-14T09:39:57Z

Pull request #42674 was updated. @bbilin, @srimanob, @cmsbuild, @AdrianoDee, @miquork, @sunilUIET can you please check and sign again.

antoniovilela · 2023-09-14T10:46:26Z

@mmusich
Once the tests pass, we can merge it.

antoniovilela · 2023-09-14T10:46:42Z

please test

slava77 · 2023-09-14T12:27:30Z

type tracking

cmsbuild · 2023-09-14T16:03:42Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3320af/34754/summary.html
COMMIT: 2cb3fed
CMSSW: CMSSW_13_3_X_2023-09-14-1100/el8_amd64_gcc11
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42674/34754/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3348648
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3348620
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 20 differences found in the comparisons
DQMHistoTests: Total files compared: 3
DQMHistoTests: Total histograms compared: 40118
DQMHistoTests: Total failures: 760
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 39358
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
Checked 8 log files, 10 edm output root files, 3 DQM output files
TriggerResults: no differences found

AdrianoDee · 2023-09-14T16:17:59Z

+pdmv

antoniovilela · 2023-09-14T16:43:24Z

+1

antoniovilela · 2023-09-14T16:44:19Z

merge

cmsbuild added this to the CMSSW_13_3_X milestone Aug 28, 2023

cmsbuild added pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending code-checks-pending labels Aug 28, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 28, 2023

mmusich commented Aug 28, 2023

View reviewed changes

Configuration/PyReleaseValidation/python/relval_steps.py Outdated Show resolved Hide resolved

Configuration/PyReleaseValidation/python/relval_steps.py Outdated Show resolved Hide resolved

cmsbuild added code-checks-pending and removed code-checks-approved labels Aug 28, 2023

mmusich changed the title ~~[RFC] add a 2023 data workflow for all patatrack reconstruction using the GPU modifier~~ [RFC] add a 2023 data workflow for all patatrack reconstruction using the gpuValidation modifier Aug 28, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 28, 2023

cmsbuild added tests-started and removed tests-pending labels Aug 29, 2023

mmusich force-pushed the add_141.583 branch from 9099ad2 to 2c60a70 Compare August 29, 2023 07:20

cmsbuild added tests-pending code-checks-pending and removed tests-started code-checks-approved labels Aug 29, 2023

mmusich force-pushed the add_141.583 branch from fe0ae2c to 2cb3fed Compare September 14, 2023 09:32

cmsbuild added tests-pending code-checks-pending and removed tests-rejected code-checks-approved labels Sep 14, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 14, 2023

cmsbuild added tests-started and removed tests-pending labels Sep 14, 2023

cmsbuild added the tracking label Sep 14, 2023

cmsbuild added tests-approved and removed tests-started labels Sep 14, 2023

cmsbuild added pdmv-approved and removed pdmv-pending labels Sep 14, 2023

cmsbuild added orp-approved and removed orp-pending labels Sep 14, 2023

cmsbuild merged commit c2003bf into cms-sw:master Sep 14, 2023

mmusich deleted the add_141.583 branch September 14, 2023 16:44

cmsbuild mentioned this pull request Sep 15, 2023

Phase2 MuonHLTSeed MVA Classifier CMSSW_13_3_X #42745

Merged

ferencek mentioned this pull request Sep 26, 2023

Fix for pixel FED error imbalance in DQM plots #42872

Merged

Add a 2023 data workflow for all patatrack reconstruction using the gpuValidation modifier #42674

Add a 2023 data workflow for all patatrack reconstruction using the gpuValidation modifier #42674

Uh oh!

Conversation

mmusich commented Aug 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Uh oh!

cmsbuild commented Aug 28, 2023

Uh oh!

cmsbuild commented Aug 28, 2023

Uh oh!

Uh oh!

Uh oh!

cmsbuild commented Aug 28, 2023

Uh oh!

cmsbuild commented Aug 28, 2023

Uh oh!

mmusich commented Aug 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mmusich commented Aug 29, 2023

Uh oh!

mmusich commented Aug 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Aug 29, 2023

Uh oh!

mmusich commented Sep 14, 2023

Uh oh!

cmsbuild commented Sep 14, 2023

Uh oh!

cmsbuild commented Sep 14, 2023

Uh oh!

antoniovilela commented Sep 14, 2023

Uh oh!

antoniovilela commented Sep 14, 2023

Uh oh!

slava77 commented Sep 14, 2023

Uh oh!

cmsbuild commented Sep 14, 2023

Comparison Summary

GPU Comparison Summary

Uh oh!

AdrianoDee commented Sep 14, 2023

Uh oh!

antoniovilela commented Sep 14, 2023

Uh oh!

antoniovilela commented Sep 14, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Add a 2023 data workflow for all patatrack reconstruction using the `gpuValidation` modifier #42674

Add a 2023 data workflow for all patatrack reconstruction using the `gpuValidation` modifier #42674

mmusich commented Aug 28, 2023 •

edited

Loading

mmusich commented Aug 29, 2023 •

edited

Loading

mmusich commented Aug 29, 2023 •

edited

Loading