Fix for multiprocessing in RecoTrack Validation script [12.1.X]

[mmusich]

backport of #35884

PR description:

This PR fixes a race condition introduced with the multiprocessing package.
No change is expected in the output as this concerns a plotting script.
Would be nice to have for validation purposes in this release cycle.

PR validation:

The errors reported in #35827 are gone.

cmsrel CMSSW_12_1_X_2021-10-27-2300
cd CMSSW_12_1_X_2021-10-27-2300/src/
git cms-merge-topic 35884
scramv1 b -j 20
cmsenv
voms-proxy-init -voms cms
cmsDriver.py step3 -s RAW2DIGI,RECO:reconstruction_trackingOnly,VALIDATION:@trackingOnlyValidation,DQM:@trackingOnlyDQM --conditions auto:phase1_2021_realistic --datatier GEN-SIM-RECO,DQMIO -n 10 --eventcontent RECOSIM,DQM --geometry DB:Extended --era Run3 --filein /store/relval/CMSSW_12_1_0_pre4/RelValTTbar_14TeV/GEN-SIM-DIGI-RAW/PU_121X_mcRun3_2021_realistic_v10_HighStat-v2/2580000/0e28f61f-234c-4358-874b-1fa4744b8d72.root --fileout file:step3.root --nThreads 8 --no_exec
cmsDriver.py step4 -s HARVESTING:@trackingOnlyValidation+@trackingOnlyDQM --conditions auto:phase1_2021_realistic --mc --geometry DB:Extended --scenario pp --filetype DQM --era Run3 -n 100 --filein file:step3_inDQM.root --fileout file:step4.root --no_exec
cmsRun step3_RAW2DIGI_RECO_VALIDATION_DQM.py
cmsRun step4_HARVESTING.py
makeTrackValidationPlots.py DQM_V0001_R000000001__Global__CMSSW_X_Y_Z__RECO.root

if this PR is a backport please specify the original PR and why you need to backport that PR:

Backport of #35884

[cmsbuild]

A new Pull Request was created by @mmusich (Marco Musich) for CMSSW_12_1_X.

It involves the following packages:

• Validation/RecoTrack (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please review it and eventually sign? Thanks.
@felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @ebrondol, @wmtford, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

• Backported from Fix for multiprocessing in RecoTrack Validation script #35884

[mmusich]

please test

[cmsbuild]

-1

Failed Tests: UnitTests RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-72bf43/20042/summary.html
COMMIT: 1b89b83
CMSSW: CMSSW_12_1_X_2021-10-28-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35894/20042/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMServicesDemo had ERRORS

RelVals-INPUT

• 138.3138.3_RunMinimumBias2021Splash+RunMinimumBias2021Splash+RECODR3Splash+HARVESTDR3/step2_RunMinimumBias2021Splash+RunMinimumBias2021Splash+RECODR3Splash+HARVESTDR3.log

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 2 differences found in the comparisons
• DQMHistoTests: Total files compared: 42
• DQMHistoTests: Total histograms compared: 2901440
• DQMHistoTests: Total failures: 6
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2901412
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 41 files compared)
• Checked 177 log files, 37 edm output root files, 42 DQM output files
• TriggerResults: no differences found

[mmusich]

type bug-fix

[mmusich]

failures are unrelated

[mmusich]

please test with #35930

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-72bf43/20145/summary.html
COMMIT: 1b89b83
CMSSW: CMSSW_12_1_X_2021-10-31-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35894/20145/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 42
• DQMHistoTests: Total histograms compared: 2901440
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2901418
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 41 files compared)
• Checked 177 log files, 37 edm output root files, 42 DQM output files
• TriggerResults: no differences found

[jfernan2]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[qliphy]

+1