Reset Phase-2 workflow ID #39571

srimanob · 2022-10-02T11:52:55Z

PR description:

This PR is to reset Phase-2 workflow ID as #38594
After this PR, we need to update the reco comparison matrix.

List of workflows can be found in /afs/cern.ch/user/s/srimanob/public/ForPhase2SW/ResetWorkflowID/list.log

In addition, I use this chance to enable phase-2 aging for all phase-2 workflows. Currently, it opens for ttbar only which cause more complication to mention the workflow ID with aging.

PR validation:

List workflows using runTheMatrix.py --what upgrade -n, then check.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

No need of backport

FYI @cms-sw/simulation-l2 @cms-sw/reconstruction-l2

srimanob · 2022-10-02T12:01:12Z

FYI @mmusich @cms-sw/trk-dpg-l2 @cms-sw/alca-l2
please let me know if what I clean up in Configuration/AlCa/python/autoCondPhase2.py is OK, or more than expected.

Thx.

cmsbuild · 2022-10-02T12:03:03Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39571/32355

This PR adds an extra 36KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Add run3 tracking low pu era #33532, [Draft] Update HGCAL pset for Phase-2 HLT menu #39450
- File Configuration/PyReleaseValidation/scripts/runTheMatrix.py modified in PR(s): Nano test in runthematrix #39472

cmsbuild · 2022-10-02T12:03:28Z

A new Pull Request was created by @srimanob (Phat Srimanobhas) for master.

It involves the following packages:

Configuration/AlCa (alca)
Configuration/Geometry (geometry, upgrade)
Configuration/PyReleaseValidation (pdmv, upgrade)

@malbouis, @civanch, @yuanchao, @bsunanda, @makortel, @bbilin, @saumyaphor4252, @ianna, @mdhildreth, @cmsbuild, @AdrianoDee, @srimanob, @Dr15Jones, @kskovpen, @sunilUIET, @tvami, @ChrisMisan, @francescobrivio can you please review it and eventually sign? Thanks.
@vargasa, @makortel, @kpedro88, @fabiocos, @Martin-Grunewald, @missirol, @tocheng, @trtomei, @mmusich, @beaucero, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

srimanob · 2022-10-02T12:03:31Z

FYI @fabiocos
For new BTL geometry dev.

srimanob · 2022-10-02T12:03:42Z

@cmsbuild please test

cmsbuild · 2022-10-02T17:20:02Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e5e5c6/27907/summary.html
COMMIT: 0c3b9e7
CMSSW: CMSSW_12_6_X_2022-10-02-0000/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39571/27907/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/20834.0_TTbar_14TeV+2026D88+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/20834.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/20896.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/20900.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/21034.999_TTbar_14TeV+2026D88PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/23234.0_TTbar_14TeV+2026D94+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 43
DQMHistoTests: Total histograms compared: 2866760
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2866735
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 42 files compared)
Checked 178 log files, 44 edm output root files, 43 DQM output files
TriggerResults: no differences found

srimanob · 2022-10-02T17:31:28Z

New workflows run as expected.

20834.0_TTbar_14TeV+2026D88+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGloba
20834.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal
20896.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
20900.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
21034.999_TTbar_14TeV+2026D88PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU
23234.0_TTbar_14TeV+2026D94+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

mmusich · 2022-10-03T07:20:29Z

test parameters:

full_cmssw=true

mmusich · 2022-10-03T07:20:53Z

please test

to see if there is any unit test still relying on the old symbolic Global Tags

mmusich · 2022-10-03T07:21:29Z

Configuration/AlCa/python/autoCondPhase2.py

 ## T21: Phase2 tilted tracker. Outer Tracker (v8.0.0): TBPS update in Layer 1 (facilitate IT insertion) + In all TEDD, update sensors Z inter-spacing. Inner Tracker: (v6.1.5) from previous T17
 ## (TFPX: Changed sensors spacing within all double-disks + Increased distance between Disks 6 and 7 + TBPX portcards between Disks 6 and 7.)
-## T25: Phase2 tilted tracker. Outer Tracker (v8.0.0): same as T24/T21. Inner Tracker (v7.0.2): Based on (v6.1.5) (T24/T21), but with 3D sensors in TBPX L1.
-## T26: Phase2 tilted tracker. Outer Tracker (v8.0.0): same as T24/T21. Inner Tracker (v7.0.3): Based on (v6.1.5) (T24/T21), but with 3D sensors in TBPX L1 and 50x50 pixel aspect ratio in TFPX and TEPX.


@emiglior can you please confirm that there is no need to maintain T25 / T26 conditions anymore at this point?

T25 and T26 are still active geometries.
We can probably dismiss T26 (square pixels in the Inner Tracker discs) but we must keep T25 as this is the only geometry with 3D pixels in TBPX L1 (actually we can start to produce regularly RelVals based on this geometry).

Hi @emiglior

Thanks for confirmation. Then we should have geometry with T25 and/or T26. Currently, there is none after cleaning PR is merged, #38702

I can put T25 and T26 back to this PR.

IIUC T25 was contained in D80 = T25+C14+M9+I13+O7+F6 and that geometry was removed in PR #38702 as a follow-up of issue #38594, so it won't be possible to submit any relvals with it, unless it's brought back. @srimanob

Does it fit with D88, D92, i.e. with current MTD ?
If fit, maybe you can have new geometry DXX = D88 + T25, DYY = D88+ T26.

@srimanob, I don't think Tracker particularly cares about what other subsytem versions are used to build the geometry, provided that there is at least one geometry containing T25 (or T26, though from the comment above #39571 (comment) it's not clear to me if it's still useful or not...)

Any other combinations of the other detectors is ok for the Tracker provided we can produce RelVals with T25.
What about having a geometry derived from D86 but featuring T25 instead of T24, e.g.
Dxx = T25+C17+M10+I14+O8+F6

Hi @emiglior
If there is no constraint on version of MTD, please go with DXX = D88+T25 because D88 is the current Phase-2 baseline now.

Fine with me! Who is going to prepare the PR?

I will do it.

fabiocos · 2022-10-03T11:22:14Z

FYI @fabiocos For new BTL geometry dev.

@srimanob thanks, I will rebase my dev branch on this, as soon as the new Tracker version we need is provided.

kskovpen · 2022-10-10T10:29:29Z

+pdmv

tvami · 2022-10-10T14:41:59Z

+alca

tests pass
code changes according to the PR desc

bsunanda · 2022-10-10T14:44:57Z

+geometry

srimanob · 2022-10-10T14:51:11Z

+Upgrade

This PR resets the workflow. D86 will start from 20000.0. New workflow ids are as expected as listed in the file mentioned in the PR description.

cmsbuild · 2022-10-10T14:51:36Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

srimanob · 2022-10-10T14:52:27Z

test parameters:

srimanob · 2022-10-10T14:53:55Z

@cmsbuild please test

Re-trigger the test after a week. In addition, all needed PRs were merged. We should not require externals.

cmsbuild · 2022-10-10T18:22:06Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e5e5c6/28160/summary.html
COMMIT: 63b29e4
CMSSW: CMSSW_12_6_X_2022-10-10-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39571/28160/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/20834.0_TTbar_14TeV+2026D88+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/20834.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/20896.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/20900.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/21034.999_TTbar_14TeV+2026D88PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e5e5c6/23234.0_TTbar_14TeV+2026D94+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

No significant changes to the logs found
Reco comparison results: 5 differences found in the comparisons
DQMHistoTests: Total files compared: 43
DQMHistoTests: Total histograms compared: 2833523
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2833495
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 42 files compared)
Checked 178 log files, 44 edm output root files, 43 DQM output files
TriggerResults: no differences found

rappoccio · 2022-10-11T15:54:16Z

+1

reset workflow ID

0c3b9e7

cmsbuild added this to the CMSSW_12_6_X milestone Oct 2, 2022

cmsbuild added alca-pending code-checks-pending geometry-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Oct 2, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 2, 2022

cmsbuild added tests-started and removed tests-pending labels Oct 2, 2022

cmsbuild added tests-approved and removed tests-started labels Oct 2, 2022

mmusich reviewed Oct 3, 2022

View reviewed changes

cmsbuild added tests-started and removed tests-approved labels Oct 3, 2022

Put T25 back

63b29e4

cmsbuild added pdmv-approved and removed pdmv-pending labels Oct 10, 2022

cmsbuild added alca-approved and removed alca-pending labels Oct 10, 2022

cmsbuild added geometry-approved and removed geometry-pending labels Oct 10, 2022

cmsbuild added fully-signed upgrade-approved and removed pending-signatures upgrade-pending labels Oct 10, 2022

cmsbuild removed the requires-external label Oct 10, 2022

cmsbuild added tests-started and removed tests-approved labels Oct 10, 2022

cmsbuild added tests-approved and removed tests-started labels Oct 10, 2022

cmsbuild added orp-approved and removed orp-pending labels Oct 11, 2022

cmsbuild merged commit 15c91fe into cms-sw:master Oct 11, 2022

slava77 mentioned this pull request Oct 13, 2022

reco comparisons new phase-2 wf numbering after reset around 2022-10-11 cms-sw/cms-bot#1860

Merged

srimanob mentioned this pull request Oct 25, 2022

Add D97 Phase-2 geometry #39841

Merged

clacaputo mentioned this pull request Dec 19, 2022

Update to include the timing reco through weights #40291

Merged

Reset Phase-2 workflow ID #39571

Reset Phase-2 workflow ID #39571

Uh oh!

Conversation

srimanob commented Oct 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Uh oh!

srimanob commented Oct 2, 2022

Uh oh!

cmsbuild commented Oct 2, 2022

Uh oh!

cmsbuild commented Oct 2, 2022

Uh oh!

srimanob commented Oct 2, 2022

Uh oh!

srimanob commented Oct 2, 2022

Uh oh!

cmsbuild commented Oct 2, 2022

Comparison Summary

Uh oh!

srimanob commented Oct 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mmusich commented Oct 3, 2022

Uh oh!

mmusich commented Oct 3, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

srimanob Oct 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

srimanob Oct 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fabiocos commented Oct 3, 2022

Uh oh!

kskovpen commented Oct 10, 2022

Uh oh!

tvami commented Oct 10, 2022

Uh oh!

bsunanda commented Oct 10, 2022

Uh oh!

srimanob commented Oct 10, 2022

Uh oh!

cmsbuild commented Oct 10, 2022

Uh oh!

srimanob commented Oct 10, 2022

Uh oh!

srimanob commented Oct 10, 2022

Uh oh!

cmsbuild commented Oct 10, 2022

Comparison Summary

Uh oh!

rappoccio commented Oct 11, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

srimanob commented Oct 2, 2022 •

edited

Loading

srimanob commented Oct 2, 2022 •

edited

Loading

srimanob Oct 3, 2022 •

edited

Loading

srimanob Oct 3, 2022 •

edited

Loading