Include new geometry tags in Run-3 MC GTs #35277

tvami · 2021-09-14T23:07:39Z

PR description:

Inclusion of new DDD geometry tags in Run-3 MC GTs. Requested in
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4460.html

This is meant for CMSSW_12_1_0_pre4 so should not merge before CMSSW_12_1_0_pre3 is cut (expected to be today, Sept 14)

Differences in the GTs:

2021 design: 121X_mcRun3_2021_design_v2
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/121X_mcRun3_2021_design_v1/121X_mcRun3_2021_design_v2
2021 realistic: 121X_mcRun3_2021_realistic_v3
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/121X_mcRun3_2021_realistic_v2/121X_mcRun3_2021_realistic_v3
2021 cosmics: 121X_mcRun3_2021cosmics_realistic_deco_v3
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/121X_mcRun3_2021cosmics_realistic_deco_v2/121X_mcRun3_2021cosmics_realistic_deco_v3
2021 heavy ion: 121X_mcRun3_2021_realistic_HI_v3
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/121X_mcRun3_2021_realistic_HI_v2/121X_mcRun3_2021_realistic_HI_v3
2023 realistic: 121X_mcRun3_2023_realistic_v2
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/121X_mcRun3_2023_realistic_v1/121X_mcRun3_2023_realistic_v2
2024 realistic: 121X_mcRun3_2024_realistic_v2
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/121X_mcRun3_2024_realistic_v1/121X_mcRun3_2024_realistic_v2

Differences are in the requested 2 geometry tags:
XMLFILE_Geometry_120YV2_Extended2021_mc and TKRECO_Geometry_120YV2

These revisions are included:

PPS Run 3 sim and reco geometry was implemented.
PPS material names were made unique.
Duplicate material names were eliminated.
In the extended geometry description, numerical drift of rotation and translation values was reduced by rounding values <=|1e-11| to zero to match their original definitions.
In the Tracker Reco geometry record, calculated values of translations <=|1.e-10| were rounded to zero. Also, the third position of the so-called "navType" (_nt2) was updated from 4 to 6. This change occurred in XML during 11_2 development, but it never made it to the DB until now.

PR validation:

Geometry validation talk in the Simulation meeting: https://indico.cern.ch/event/1071938/#56-progress-report-on-dd4hep-m

if this PR is a backport please specify the original PR and why you need to backport that PR:

This is not a backport but a backport to 12_0_X is expected

cc @cvuosalo @srimanob

cmsbuild · 2021-09-14T23:13:36Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35277/25274

This PR adds an extra 12KB to repository

cmsbuild · 2021-09-14T23:14:02Z

A new Pull Request was created by @tvami (Tamas Vami) for master.

It involves the following packages:

Configuration/AlCa (alca)

@yuanchao, @malbouis, @cmsbuild, @tvami, @francescobrivio can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

tvami · 2021-09-14T23:16:19Z

@cmsbuild , please test

tvami · 2021-09-14T23:27:44Z

As written in the description: "This is meant for CMSSW_12_1_0_pre4 so should not merge before CMSSW_12_1_0_pre3 is cut (expected to be today, Sept 14)"
@perrotta @qliphy can you confirm that this schedule is still valid?

qliphy · 2021-09-15T00:38:02Z

As written in the description: "This is meant for CMSSW_12_1_0_pre4 so should not merge before CMSSW_12_1_0_pre3 is cut (expected to be today, Sept 14)"
@perrotta @qliphy can you confirm that this schedule is still valid?

Yes, pre3 is to be built quite soon.

cmsbuild · 2021-09-15T04:06:44Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d96316/18612/summary.html
COMMIT: 5bb86ed
CMSSW: CMSSW_12_1_X_2021-09-14-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35277/18612/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d96316/18612/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d96316/18612/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 7480 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 3000833
DQMHistoTests: Total failures: 83370
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2917441
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: found differences in 3 / 38 workflows

francescobrivio · 2021-09-15T10:05:17Z

Tests passed, but, as always when changing the geometry, there are several differences in the comparisons.
@cms-sw/geometry-l2 please could you confirm these differences are expected?

tvami · 2021-09-15T15:10:51Z

@cmsbuild , please test

many other commits were included in the other test, maybe this time it will be cleaner?

cvuosalo · 2021-09-15T16:30:57Z

This PR changes the DDD geometry. There is no way to preserve regression in this case. The 2021 DDD workflows that re-generate MC (312.0, 11634.0, and 12434.0) will show comparison differences, but these differences look like statistical fluctuations.

cvuosalo · 2021-09-15T16:37:47Z

At Tuesday's Simulation meeting, it was decided that the DDD geometry will require one more technical revision. That means there will be one more DB upload, and a new GT version will be needed.
I think it is OK to go ahead with this PR. The GT in this PR contains the major PPS changes. Then the next GT will contain only very minor technical geometry changes.
@tvami Do you agree with this plan? The alternative is to stop this PR, wait for the new DDD revision, and create a new GT that contains all the changes together.

tvami · 2021-09-15T16:57:57Z

At Tuesday's Simulation meeting, it was decided that the DDD geometry will require one more technical revision. That means there will be one more DB upload, and a new GT version will be needed.
I think it is OK to go ahead with this PR. The GT in this PR contains the major PPS changes. Then the next GT will contain only very minor technical geometry changes.
@tvami Do you agree with this plan? The alternative is to stop this PR, wait for the new DDD revision, and create a new GT that contains all the changes together.

Hi @cvuosalo I agree with this plan and then we can consider that other payload as a bugfix maybe

francescobrivio · 2021-09-15T17:01:49Z

+alca

cmsbuild · 2021-09-15T17:02:11Z

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2021-09-15T20:01:10Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d96316/18631/summary.html
COMMIT: 5bb86ed
CMSSW: CMSSW_12_1_X_2021-09-14-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35277/18631/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d96316/18631/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d96316/18631/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 7478 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 3000833
DQMHistoTests: Total failures: 83364
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2917447
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: found differences in 3 / 38 workflows

tvami · 2021-09-15T20:06:22Z

This PR changes the DDD geometry. There is no way to preserve regression in this case. The 2021 DDD workflows that re-generate MC (312.0, 11634.0, and 12434.0) will show comparison differences, but these differences look like statistical fluctuations.

Ok so the new Jenkins results are similar, based on the above, we are ready for it to get merged

tvami · 2021-09-16T00:24:29Z

Hi @perrotta @qliphy sorry for urging you so often, we get a lot of requests for GTs now... Please see
#35295

qliphy · 2021-09-16T00:32:59Z

@tvami No problem. We are now building 12_1_0_pre3 and will start adding new PRs after the build is finished.

qliphy · 2021-09-16T14:21:33Z

+1

Include new geometry tags in Run-3 MC GTs

5bb86ed

cmsbuild added this to the CMSSW_12_1_X milestone Sep 14, 2021

cmsbuild added alca-pending code-checks-pending orp-pending pending-signatures tests-pending labels Sep 14, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 14, 2021

cmsbuild added tests-started and removed tests-pending labels Sep 14, 2021

cmsbuild added tests-approved and removed tests-started labels Sep 15, 2021

cmsbuild added tests-started and removed tests-approved labels Sep 15, 2021

cmsbuild added alca-approved and removed alca-pending pending-signatures labels Sep 15, 2021

cmsbuild added the fully-signed label Sep 15, 2021

malbouis mentioned this pull request Sep 15, 2021

[120X] Include new geometry tags in Run-3 MC GTs #35291

Merged

cmsbuild added tests-approved and removed tests-started labels Sep 15, 2021

tvami mentioned this pull request Sep 16, 2021

Remove unused pixel template records from ESProducers and update global tags #35295

Merged

cmsbuild added orp-approved and removed orp-pending labels Sep 16, 2021

cmsbuild merged commit ba8af8e into cms-sw:master Sep 16, 2021

cmsbuild mentioned this pull request Sep 20, 2021

Update Geant4 to 10.7.2 cms-sw/cmsdist#7316

Merged

Include new geometry tags in Run-3 MC GTs #35277

Include new geometry tags in Run-3 MC GTs #35277

Uh oh!

Conversation

tvami commented Sep 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

Uh oh!

cmsbuild commented Sep 14, 2021

Uh oh!

cmsbuild commented Sep 14, 2021

Uh oh!

tvami commented Sep 14, 2021

Uh oh!

tvami commented Sep 14, 2021

Uh oh!

qliphy commented Sep 15, 2021

Uh oh!

cmsbuild commented Sep 15, 2021

Comparison Summary

Uh oh!

francescobrivio commented Sep 15, 2021

Uh oh!

tvami commented Sep 15, 2021

Uh oh!

cvuosalo commented Sep 15, 2021

Uh oh!

cvuosalo commented Sep 15, 2021

Uh oh!

tvami commented Sep 15, 2021

Uh oh!

francescobrivio commented Sep 15, 2021

Uh oh!

cmsbuild commented Sep 15, 2021

Uh oh!

cmsbuild commented Sep 15, 2021

Comparison Summary

Uh oh!

tvami commented Sep 15, 2021

Uh oh!

tvami commented Sep 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qliphy commented Sep 16, 2021

Uh oh!

qliphy commented Sep 16, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tvami commented Sep 14, 2021 •

edited

Loading

tvami commented Sep 16, 2021 •

edited

Loading