Run 3 FastSim refinement for PUPPI jets #48587
Open
+113
−8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
cms-bot internal usage |
Contributor
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48587/45554 |
Contributor
|
A new Pull Request was created by @sbein for master. It involves the following packages:
@cmsbuild, @ftorrresd, @hqucms can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
sbein
force-pushed
the
from-CMSSW_15_1_X_2025-07-16-2300
branch
from
41ad6d7 to
bbff503
Compare
Contributor
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48587/45647
|
Contributor
|
Pull request #48587 was updated. @cmsbuild, @ftorrresd, @hqucms can you please check and sign again. |
Contributor
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48587/45648
|
Contributor
|
Pull request #48587 was updated. @cmsbuild, @ftorrresd, @hqucms can you please check and sign again. |
Contributor
|
enable nano |
Contributor
|
please test |
Contributor
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48587/46731 |
Contributor
|
Pull request #48587 was updated. @battibass, @cmsbuild, @ftorrresd can you please check and sign again. |
Contributor
Author
|
It's better now but the mask saga continues. I have implemented a mask that tosses the refined values and puts back in the originals for cases where an unrefined UParT tagger value was negative (cut made in the training). But the "originals" are not quite the originals, because nested ternaries caused the parser to either crash or assign completely wrong refined values. I think it's pretty good now, and the only down side seems to be that the neat -1 values for the UParT taggers can sometimes be -10. @kpedro88 or anyone, you are welcome to try to implement the -1 safety assignment as in In the masking expression, which is currently: (missing safety), Or suggest the fix if this is a known problem. The parser is just super fragile though, and the current commit might be the best we can do. |
|
please test |
Contributor
|
+1 Size: This PR adds an extra 28KB to repository The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: Comparison SummarySummary:
Nano size comparison Summary:
NANO Comparison SummarySummary:
|
…e other negative DeepJet and UParT tagger values (all less than -1) without breaking the parser
cmsbuild
added
tests-pending
code-checks-pending
and removed
tests-approved
code-checks-approved
labels
Contributor
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48587/46752 |
Contributor
|
Pull request #48587 was updated. @battibass, @cmsbuild, @ftorrresd can you please check and sign again. |
Contributor
|
please test |
Contributor
Author
|
I introduced max(x,-1) functions in lieu of ?x>0?x:-1, which restores the "uncalculated" DeepJet and UParT values from their unprotected default values of -10 or larger to -1, without breaking the seemingly fragile parser. The before (left) and after (right) scatter plot of [official branch] VS [original unrefined counterpart] for one example is: so it looks like it's getting the job done. |
Contributor
|
+1 Size: This PR adds an extra 28KB to repository The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: Comparison SummarySummary:
Nano size comparison Summary:
NANO Comparison SummarySummary:
|
Contributor
Author
|
The plots look as expected and individual values were also checked and confirmed. The higher stats private production confirms the max function worked for the mask: 
This one's ready for CMS consumption. This is a fairly sizeable change. It could be worth requesting a high-stats NanoDQM comparison with FullSim with (and without) PU. Please note, in a future PR, one could add the Type-1 MET correction for these jets, which is not implemented currently. Also, the ordering of jets matches the original pT-based ordering in NANO, so one could consider re-ordering those. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR accompanies a cms-data PR cms-data/PhysicsTools-NanoAOD#20 containing the .onnx file.
This PR is the Run 3 version of an earlier Run 2 PR: #40553, applying a correction to PUPPI jets in NANOAOD (previously was CHS jets for Run 2 UL). The PR effects only NANOAOD and NANOEDM files.
We update the nano-config file needed for regression-based refinement are included for FastSim Run 3 ak4 PUPPI jet variables, effecting only the NANO-level collections:
Jet_pt
Jet_btagDeepFlavB
Jet_btagDeepFlavCvB
Jet_btagDeepFlavCvL
Jet_btagDeepFlavQG
Jet_btagUParTAK4B
Jet_btagUParTAK4CvB
Jet_btagUParTAK4CvL
Jet_btagUParTAK4QvG
The results of the refinement compare the refined FastSim, original FastSim, and FullSim distributions for these variables:
https://www.desy.de/~beinsam/FastSim/Refinement/Jets/figs31July_long/index.html
The PR also adds 9 backup branches
Jet_ptunrefined
Jet_btagDeepFlavBunrefined
Jet_btagDeepFlavCvBunrefined
Jet_btagDeepFlavCvLunrefined
Jet_btagDeepFlavQGunrefined
Jet_btagUParTAK4Bunrefined
Jet_btagUParTAK4CvBunrefined
Jet_btagUParTAK4CvLunrefined
Jet_btagUParTAK4QvGunrefined
which store the original values, as a temporary measure for risk mitigation. The preliminary version of these results were approved for CHEP24 and discussed in numerous FastSim meetings, e.g.,
https://indico.cern.ch/event/1545175/
and at the PAGs Jamboree: https://indico.cern.ch/event/1519600/contributions/6444135/attachments/3056422/5403783/Run3FastSimPagJamboree.pdf