Automatic switching between CPU and GPU for DQMEDAnalyzer

[sroychow]

From Tracker DQM side, we are developing DQM modules to monitor hlt products(e.g. pixel Tracks, Vertices in SoA) which can either be produced on a GPU or a CPU. Right now in out tests, the same module is modified with the gpu modifier to use the correct product in a GPU wf. Example is :-

monitorpixelTrackSoA = DQMEDAnalyzer('SiPixelPhase1MonitorTrackSoA',
                                        pixelTrackSrc = cms.InputTag("pixelTracksSoA@cpu"),
                                        TopFolderName = cms.string("SiPixelHeterogeneous/PixelTrackSoA"),
)

gpu.toModify(monitorpixelTrackSoA,
    pixelTrackSrc = cms.InputTag("pixelTracksSoA@cuda")
)

Given that we want to run this at HLT, I wanted to understand if we can have a SwitchProducer mechanism for DQMEDAnalyzer so that we can do something like this:-

monitorpixelTrackSoA =  SwitchProducerCUDA(
    cpu =DQMEDAnalyzer(.....
                            pixelTrackSrc = cms.InputTag("pixelTracksSoA@cpu"),
            )
    cuda = DQMEDAnalyzer(.....
                             pixelTrackSrc = cms.InputTag("pixelTracksSoA@cuda"),
                ),
)

Can FrameWork experts give some guidance on this?
@arossi83 @mmusich @tsusa @connorpa

[cmsbuild]

A new Issue was created by @sroychow Suvankar Roy Chowdhury.

@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[mmusich]

assign dqm, heterogeneous

[cmsbuild]

New categories assigned: heterogeneous,dqm

@jfernan2,@ahmad3213,@rvenditti,@fwyzard,@emanueleusai,@makortel,@pbo0,@pmandrik you have been requested to review this Pull request/Issue and eventually sign? Thanks

[fwyzard]

@mmusich currently I don't think that is possible.

The reason is that the SwitchProducer does not run anything by itself, it simply "aliases" one of its branches to its name; the "alias" is then run when its products are requested by some other module.

Since an EDAnalyzer does not produce anything, there would be nothing triggering its execution.

You should be able to achieve the same effect simply with

monitorpixelTrackSoA = DQMEDAnalyzer('SiPixelPhase1MonitorTrackSoA',
                                        pixelTrackSrc = cms.InputTag("pixelTracksSoA"),
                                        TopFolderName = cms.string("SiPixelHeterogeneous/PixelTrackSoA"),
)

The pixelTracksSoA SwitchProducer will pick pixelTracksSoA@cpu or pixelTracksSoA@cuda automatically.

However, with this approach how do you disentangle what has been running on the CPU and on the GPU ?

[mmusich]

@fwyzard aren't all DQM modules EDProducers: https://github.com/cms-sw/cmssw/blob/master/DQMServices/Core/README.md ?

[fwyzard]

Ah, good point, I'm stuck to the pre-transition approach based on EDAnalyzers.

[mmusich]

However, with this approach how do you disentangle what has been running on the CPU and on the GPU ?

It shouldn't but perhaps we still haven't got the gist of what is requested.
I thought we would be submitting relvals with and without the gpu modifier and compare the products to validate that CPU SoA-based reco gives same results as GPU one.

[fwyzard]

Then this

monitorpixelTrackSoA = SwitchProducerCUDA(
    cpu = DQMEDAnalyzer('SiPixelPhase1MonitorTrackSoA',
                                        pixelTrackSrc = cms.InputTag("pixelTracksSoA@cpu"),
                                        TopFolderName = cms.string("SiPixelHeterogeneous/PixelTrackSoA")
    ),
    cuda = DQMEDAnalyzer('SiPixelPhase1MonitorTrackSoA',
                                        pixelTrackSrc = cms.InputTag("pixelTracksSoA@cuda"),
                                        TopFolderName = cms.string("SiPixelHeterogeneous/PixelTrackSoA")
    )
)

should work in principle.

Still

• why would you prefer this to just reading the pixelTracksSoA collection ?
• when looking at the plots, how do you disentagle what has been produced on GPU vs CPU ?

[fwyzard]

It shouldn't but perhaps we still haven't got the gist of what is requested.

Sorry, you wrote

Given that we want to run this at HLT

so I assumed you meant inside the HLT running online.

[mmusich]

Sorry, you wrote

first, I didn't wrote it, I am not the author of the issue :)

so I assumed you meant inside the HLT running online.

No, we're trying to address requests from PPD/ TSG about CPU/GPU validation.