test: evidence-sampled subhalo recovery acceptance test (phase 3) - #243
Merged
Conversation
…72 phase 3) Grids the Matern (coefficient, scale) evidence of BOTH potential-correction methods — one-shot via the fixed-curvature fast path, iterative to convergence under identity damping — and asserts the input subhalo is localized at each method's evidence maximum, with a ridge reference proving the family contains high-fidelity recoveries. no_run-excluded (manual validation, ~1-2h by design). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 3 of PyAutoLabs/PyAutoLens#672: the quantitative acceptance test of the potential-correction implementation proposed by the upstream author — regularization hyper-parameters determined by maximizing the Bayesian evidence, for BOTH the one-shot (single-step) method and the iterative method, with subhalo recovery asserted at each method's evidence maximum.
scripts/imaging/subhalo_recovery_evidence.py(new): on the standard 120×120 mock ofsubhalo_recovery.py, (1) a 13×12 grid over Matern(coefficient, scale)of the one-shot joint inversion's evidence viadense_util.log_evidence_from_fixed_curvature(per-sample cost = two reg-matrix builds + three Choleskys — the curvature matrix, data vector and mapping are regularization-independent and cached), and (2) a 5×5 grid where the LM engine runs to convergence from a cold start (identity damping + stall guards, PyAutoLens#676) and returns the converged Laplace evidence. Evidence surfaces are saved to npz before any assertion.Findings (full analysis + review report on the issue): both methods localize the input subhalo at their evidence maxima (peaks 0.36"/0.16" from truth), but within the Matern family evidence-max ≠ map-fidelity-max — the +190-nat preference for shorter scales decomposes as +401 prior-misfit relief (a compact cuspy NFW dkappa misfits a long-scale smooth prior) +102 chi² −313 Occam, and the shorter-scale prior dilutes map correlation (0.13 at the max vs 0.82 on the hand-calibrated ridge). The acceptance criterion therefore asserts localization at the evidence max (distance thresholds + corr noise floor) plus a ridge reference proving the family contains high-fidelity solutions (corr > 0.5 at c=2000, s=4).
Runtime is one to a few hours by design (validation, not smoke): excluded from the automated runners via
config/build/no_run.yaml.Scripts Changed
scripts/imaging/subhalo_recovery_evidence.py— new evidence-sampled acceptance test (both methods, grids + acceptance assertions; results npz saved pre-assert;gc.collect()per iterative point).config/build/no_run.yaml— SLOW exclusion for the new script (manual validation, ~1-2h by design).Upstream PR
PyAutoLabs/PyAutoLens#676 — the iterative leg requires the identity-damping default and stall guards (a Marquardt-damped cold start under-converges at every grid point; without the guards each point pays a ~10× rejection storm).
Test Plan
🤖 Generated with Claude Code