Just to say that this is great, and that in ICARUS we should do something that mirrors the result of this PR

Absolutely smashed it there @marcodeltutto !
Is this ready for review? (220 files!)

Ready now @absolution1! Yes, I apologize for the many files. I tried to split it in multiple PRs, but if I modify a file that is included in other 100 fcls...well, you get 100 modifies fcls... :(

@absolution1 I consolidated the SCE config for sim and reco, and added _sbnd to more files (hopefully all?). There may be some refinements to do later on, but I think out standard workflow should be solid

I am taking a look at the unit test failures

Hi @marcodeltutto , there are a few warnings / errors

CI

The CI needs updated fcl names provided, see these errors for example

61: Testing fcl file prodoverlay_corsika_cosmics_proton_genie_nu_spill_tpc_sbnd.fcl
62: Error parsing prodoverlay_corsika_cosmics_proton_genie_nu_spill_tpc_sbnd.fcl --- Exited with status 90
63: ===================================================================================================================================
64: Failed to parse the configuration file 'prodoverlay_corsika_cosmics_proton_genie_nu_spill_tpc_sbnd.fcl' with exception
65: ---- search_path BEGIN
66:   Can't find file "prodoverlay_corsika_cosmics_proton_genie_nu_spill_tpc_sbnd.fcl"
67: ---- search_path END
68: 
69: ===================================================================================================================================
70: 
71: Testing fcl file prodoverlay_corsika_cosmics_proton_genie_rockbox_sce.fcl
72: Error parsing prodoverlay_corsika_cosmics_proton_genie_rockbox_sce.fcl --- Exited with status 90
73: ===================================================================================================================================
74: Failed to parse the configuration file 'prodoverlay_corsika_cosmics_proton_genie_rockbox_sce.fcl' with exception
75: ---- search_path BEGIN
76:   Can't find file "prodoverlay_corsika_cosmics_proton_genie_rockbox_sce.fcl"
77: ---- search_path END

You should find the relevant files to edit here - https://github.com/SBNSoftware/sbndcode/tree/develop/test/ci

CAF

There are many CAF differences, which could be related to needing to update the workflow. You can see all differences here - https://dbweb8.fnal.gov:8443/LarCI/app/ns:sbnd/storage/docs/2025/03/05/stdout%23wdnX6XB.log

I suppose we need to update the CI before identifying the issues with other tests (including the CAF stage).

Thanks @bear-is-asleep. I updated the fcl list. That one only checks the fcl integrity though. So the CAF error will persist. I think I know why, I will check and comment back again later

In the latest CI tests, there are a few warnings:

FHiCL Checks

This is expected. They are related to:

• EnableSimEfield: this has been eliminated from the FHiCL as the SCE service doesn't have this variable
• is2DdriftSimHack: this is now set to true at sim, reco, and caf level (was only at sim level before)

G4 and DetSim Stages

This is expected. OpDetBacktrackerRecord and SimPhotonsLite data products are now missing. This is because the _lite version of the fcl files is now incorporated in the "standard" fcls. This is what we have been running in production.

CAF Stage

Differences appear to be on thruth matching variables. This is caused by having is2DdriftSimHack = True. The differences disappear if I run the CI with is2DdriftSimHack = False.

CAFMakes calls the thruth matcher, which uses the SpaceCharge service GetPosOffsets function. The behavior of this function changes based on is2DdriftSimHack. I believe setting it to true is correct, which implies the previous production may not have an accurate truth matching. @jzennamo if you could weigh on this it would be useful! Thank you!

Regarding the CAF differences, they are all related to stub variables. I have narrowed down the problem to a use of SpaceCharge::GetPosOffsets here which should be GetCalPosOffsets. I'm confirming with Gray, after that, this PR is ready.

✔️ CI build for LArSoft Succeeded on slf7 for e26:prof -- details available through the CI dashboard

✔️ CI build for LArSoft Succeeded on slf7 for c14:prof -- details available through the CI dashboard

❌ CI build for SBND Failed at phase build SBND on slf7 for c14:prof -- details available through the CI dashboard

🚨 For more details about the failed phase, check the build SBND phase logs

parent CI build details are available through the CI dashboard

⚠️ CI build for SBND Warning at phase ci_tests SBND on slf7 for e26:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests SBND phase logs

parent CI build details are available through the CI dashboard

Summary of CI Warnings

fcl_checks

• Shows that SCE is now turned ON by default in fcl files. Expected. ✅
• Shows that simulation fcl files use is2DdriftSimHack: true. Expected. ✅

G4 Stage

• Shows that sim::OpDetBacktrackerRecord is missing, because lite fcls are now the standard (as they are used in production). Expected. ✅

Detsim Stage

• Shows that sim::SimPhotonsLite, sim::OpDetBacktrackerRecord and others are missing, because lite fcls are now the standard (as they are used in production). Expected. ✅

CAF stage

• Small differences observed in shower bestplane and track producer, but those are not reproducible on my end. In any case, it's give more resonable values now compared to defaults in the reference files. ✅

Approved