Skip to content

Introducing SimOnGen and GenOnly wfs for 2024 and 2026D110 #45005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cmsbuild merged 3 commits into from
Jun 12, 2024
Merged

Introducing SimOnGen and GenOnly wfs for 2024 and 2026D110 #45005

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7ac2590
New GenOnly and SimOnGen wfs:
May 21, 2024
86a8712
Restricting GenOnly wfs to baseline
May 31, 2024
1250f1d
Fix ALCA input
Jun 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion Configuration/PyReleaseValidation/python/WorkFlowRunner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ def __init__(self, wf, noRun=False,dryRun=False,cafVeto=True,dasOptions="",jobRe
self.nStreams=nStreams
self.maxSteps=maxSteps
self.nEvents=nEvents
self.recoOutput=''

self.wfDir=str(self.wf.numId)+'_'+self.wf.nameId
return
Expand Down Expand Up @@ -158,9 +159,17 @@ def closeCmd(i,ID):
# Disable input for premix stage2 in FastSim to allow combined stage1+stage2 workflow (in FS, stage2 does also GEN)
# Ugly hack but works
if istep!=1 and not '--filein' in cmd and not 'premix_stage1' in cmd and not ("--fast" in cmd and "premix_stage2" in cmd):
cmd+=' --filein file:step%s.root '%(istep-1,)
steps = cmd.split("-s ")[1].split(" ")[0] ## relying on the syntax: cmsDriver -s STEPS --otherFlags
if "ALCA" not in steps:
cmd+=' --filein file:step%s.root '%(istep-1,)
elif "ALCA" in steps and "RECO" in steps:
cmd+=' --filein file:step%s.root '%(istep-1,)
else:
cmd+=' --filein %s'%(self.recoOutput)
if not '--fileout' in com:
cmd+=' --fileout file:step%s.root '%(istep,)
if "RECO" in cmd:
self.recoOutput = "file:step%d.root"%(istep)
if self.jobReport:
cmd += ' --suffix "-j JobReport%s.xml " ' % istep
if (self.nThreads > 1) and ('HARVESTING' not in cmd) and ('ALCAHARVEST' not in cmd):
Expand Down
17 changes: 15 additions & 2 deletions Configuration/PyReleaseValidation/python/relval_steps.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4222,6 +4222,7 @@ def gen2023HiMix(fragment,howMuch):
defaultDataSets['2023FS']='CMSSW_13_0_11-130X_mcRun3_2023_realistic_withEarly2023BS_v1_FastSim-v'
defaultDataSets['2024']='CMSSW_14_0_0_pre3-140X_mcRun3_2024_realistic_v1_STD_2024_noPU-v'
defaultDataSets['2024HLTOnDigi']='CMSSW_14_0_0_pre3-140X_mcRun3_2024_realistic_v1_STD_2024_noPU-v'
defaultDataSets["2024SimOnGen"] = 'CMSSW_14_0_0_pre3-140X_mcRun3_2024_realistic_v1_STD_2024_noPU-v'
defaultDataSets['2026D49']='CMSSW_12_0_0_pre4-113X_mcRun4_realistic_v7_2026D49noPU-v'
defaultDataSets['2026D76']='CMSSW_12_0_0_pre4-113X_mcRun4_realistic_v7_2026D76noPU-v'
defaultDataSets['2026D77']='CMSSW_12_1_0_pre2-113X_mcRun4_realistic_v7_2026D77noPU-v'
Expand Down Expand Up @@ -4302,6 +4303,7 @@ def gen2023HiMix(fragment,howMuch):
'--eventcontent': 'FEVTDEBUG',
'--geometry' : geom
}

if beamspot is not None: upgradeStepDict['GenSim'][k]['--beamspot']=beamspot

upgradeStepDict['GenSimHLBeamSpot'][k]= {'-s' : 'GEN,SIM',
Expand Down Expand Up @@ -4331,6 +4333,18 @@ def gen2023HiMix(fragment,howMuch):
'--geometry' : geom
}

upgradeStepDict['Sim'][k]= {'-s' : 'SIM',
'-n' : 10,
'--conditions' : gt,
'--beamspot' : 'Realistic25ns13TeVEarly2017Collision',
'--datatier' : 'SIM',
'--eventcontent': 'FEVTDEBUG',
'--geometry' : geom
}

if beamspot is not None: upgradeStepDict['Sim'][k]['--beamspot']=beamspot


upgradeStepDict['Digi'][k] = {'-s':'DIGI:pdigi_valid,L1,DIGI2RAW,HLT:%s'%(hltversion),
'--conditions':gt,
'--datatier':'GEN-SIM-DIGI-RAW',
Expand Down Expand Up @@ -4461,7 +4475,6 @@ def gen2023HiMix(fragment,howMuch):
'-n':'10',
'--eventcontent':'ALCARECO',
'--geometry' : geom,
'--filein':'file:step3.root'
}

upgradeStepDict['ALCAPhase2'][k] = merge([{'-s':'ALCA:SiPixelCalSingleMuonLoose+SiPixelCalSingleMuonTight+TkAlMuonIsolated+TkAlMinBias+MuAlOverlaps+EcalESAlign+TkAlZMuMu+TkAlDiMuonAndVertex+HcalCalHBHEMuonProducerFilter+TkAlUpsilonMuMu+TkAlJpsiMuMu'},upgradeStepDict['ALCA'][k]])
Expand Down Expand Up @@ -4545,7 +4558,7 @@ def gen2023HiMix(fragment,howMuch):

for step in upgradeStepDict.keys():
# we need to do this for each fragment
if ('Sim' in step and 'Fast' not in step) or ('Premix' in step) or ('Sim' not in step and 'Gen' in step):
if ('Sim' in step and ('Fast' not in step and step != 'Sim')) or ('Premix' in step) or ('Sim' not in step and 'Gen' in step):
for frag,info in upgradeFragments.items():
howMuch=info.howMuch
for key in [key for year in upgradeKeys for key in upgradeKeys[year]]:
Expand Down
14 changes: 13 additions & 1 deletion Configuration/PyReleaseValidation/python/relval_upgrade.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ def makeStepName(key,frag,step,suffix):

#just define all of them

## ... but we don't need all the flavors for the GenOnly
def notForGenOnly(key,specialType):
return "GenOnly" in key and specialType != 'baseline'

for year in upgradeKeys:
for i,key in enumerate(upgradeKeys[year]):
numWF=numWFAll[year][i]
Expand All @@ -26,11 +30,13 @@ def makeStepName(key,frag,step,suffix):
continue
stepList={}
for specialType in upgradeWFs.keys():
if notForGenOnly(key,specialType):
continue
stepList[specialType] = []
hasHarvest = False
for step in upgradeProperties[year][key]['ScenToRun']:
stepMaker = makeStepName
if 'Sim' in step and 'Fast' not in step:
if 'Sim' in step and 'Fast' not in step and step != "Sim":
if 'HLBeamSpot' in step:
if '14TeV' in frag:
step = 'GenSimHLBeamSpot14'
Expand All @@ -43,6 +49,10 @@ def makeStepName(key,frag,step,suffix):
if 'HARVEST' in step: hasHarvest = True

for specialType,specialWF in upgradeWFs.items():

if notForGenOnly(key,specialType): ## we don't need all the flavors for the GEN
continue

if (specialType != 'baseline') and ( ('PU' in step and step.replace('PU','') in specialWF.PU) or (step in specialWF.steps) ):
stepList[specialType].append(stepMaker(key,frag[:-4],step,specialWF.suffix))
# hack to add an extra step
Expand Down Expand Up @@ -71,6 +81,8 @@ def makeStepName(key,frag,step,suffix):

for specialType,specialWF in upgradeWFs.items():
# remove other steps for premixS1
if notForGenOnly(key,specialType):
continue
if specialType=="PMXS1":
stepList[specialType] = stepList[specialType][:1]
specialWF.workflow(workflows, numWF, info.dataset, stepList[specialType], key, hasHarvest)
Expand Down
46 changes: 40 additions & 6 deletions Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,9 @@
'2023HI',
'2023HIRP', #RawPrime
'2024HLTOnDigi',
'2024HLTOnDigiPU'

'2024HLTOnDigiPU',
'2024GenOnly',
'2024SimOnGen',
]

upgradeKeys[2026] = [
Expand Down Expand Up @@ -93,7 +94,9 @@
'2026D113',
'2026D113PU',
'2026D114',
'2026D114PU'
'2026D114PU',
'2026D110GenOnly',
'2026D110SimOnGen',
]

# pre-generation of WF numbers
Expand Down Expand Up @@ -178,7 +181,7 @@ def workflow_(self, workflows, num, fragment, stepList, key):
def condition(self, fragment, stepList, key, hasHarvest):
return False
def preventReuse(self, stepName, stepDict, k):
if "Sim" in stepName:
if "Sim" in stepName and stepName != "Sim":
stepDict[stepName][k] = None
if "Gen" in stepName:
stepDict[stepName][k] = None
Expand All @@ -198,6 +201,7 @@ def condition(self, fragment, stepList, key, hasHarvest):
upgradeWFs['baseline'] = UpgradeWorkflow_baseline(
steps = [
'Gen',
'Sim',
'GenSim',
'GenSimHLBeamSpot',
'GenSimHLBeamSpot14',
Expand Down Expand Up @@ -2620,7 +2624,7 @@ def condition(self, fragment, stepList, key, hasHarvest):

class UpgradeWorkflow_Run3FSMBMixing(UpgradeWorkflow):
def setup_(self, step, stepName, stepDict, k, properties):
if 'Gen' in step:
if 'Gen' in step and 'GenOnly' not in step:
stepDict[stepName][k] = merge([{'-s':'GEN,SIM,RECOBEFMIX',
'--fast':'',
'--era':'Run3_FastSim',
Expand Down Expand Up @@ -2933,7 +2937,22 @@ def condition(self, fragment, stepList, key, hasHarvest):
'Era':'Run3_pp_on_PbPb_approxSiStripClusters',
'BeamSpot': 'DBrealistic',
'ScenToRun' : ['GenSim','Digi','RecoNano','HARVESTNano','ALCA'],
}
},
'2024GenOnly' : {
'Geom' : 'DB:Extended',
'GT' : 'auto:phase1_2024_realistic',
'Era' : 'Run3',
'BeamSpot': 'DBrealistic',
'ScenToRun' : ['Gen'],
},
'2024SimOnGen' : {
'Geom' : 'DB:Extended',
'GT' : 'auto:phase1_2024_realistic',
'HLTmenu': '@relval2024',
'Era' : 'Run3',
'BeamSpot': 'DBrealistic',
'ScenToRun' : ['Gen','Sim','Digi','RecoNano','HARVESTNano','ALCA'],
},
}

# standard PU sequences
Expand Down Expand Up @@ -3133,6 +3152,21 @@ def condition(self, fragment, stepList, key, hasHarvest):
'Era' : 'Phase2C17I13M9',
'ScenToRun' : ['GenSimHLBeamSpot','DigiTrigger','RecoGlobal', 'HARVESTGlobal', 'ALCAPhase2'],
},
'2026D110GenOnly' : {
'Geom' : 'Extended2026D110',
'Beamspot' : 'HLLHC',
Copy link
Contributor

@srimanob srimanob Jul 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @AdrianoDee
Just wonder that we will have the right output with this setting of beamspot?

Copy link
Contributor Author

@AdrianoDee AdrianoDee Aug 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Phat, I see this message only now. Isn't it just dummy it here? I mean: the BS is defined at SIM step no? Or we have something already at GEN step?

Copy link
Contributor

@srimanob srimanob Aug 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @AdrianoDee
Thanks for coming back, I completely forgot about my comment.

process.VtxSmeared is part of process.pgen. So you will get wrong parameters for vertex smearing. Using DBrealisticHLLHC should give you the proper parameter.

Copy link
Contributor

@srimanob srimanob Aug 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

process.pgen = cms.Sequence(cms.SequencePlaceholder("randomEngineStateProducer")+process.VertexSmearing+process.GenSmeared+process.GeneInfo+process.genJetMET, process.PPSTransportTask)
and
process.VertexSmearing = cms.Sequence(cms.SequencePlaceholder("VtxSmeared"))

Copy link
Contributor

@srimanob srimanob Aug 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue comes from typo. It should be BeamSpot, not Beamspot. With typo, it will pick default BS of 'Gen' which is 2017.

Copy link
Contributor

@srimanob srimanob Aug 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix in #45685

Copy link
Contributor Author

@AdrianoDee AdrianoDee Aug 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great thanks!

'GT' : 'auto:phase2_realistic_T33',
'Era' : 'Phase2C17I13M9',
'ScenToRun' : ['Gen'],
},
'2026D110SimOnGen' : {
'Geom' : 'Extended2026D110',
'HLTmenu': '@relval2026',
'Beamspot' : 'HLLHC',
'GT' : 'auto:phase2_realistic_T33',
'Era' : 'Phase2C17I13M9',
'ScenToRun' : ['Gen','Sim','DigiTrigger','RecoGlobal', 'HARVESTGlobal', 'ALCAPhase2'],
},
}

# standard PU sequences
Expand Down