Skip to content

test: evidence-sampled subhalo recovery acceptance test (phase 3) - #243

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/potential-correction-evidence
Jul 31, 2026
Merged

test: evidence-sampled subhalo recovery acceptance test (phase 3)#243
Jammy2211 merged 1 commit into
mainfrom
feature/potential-correction-evidence

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Summary

Phase 3 of PyAutoLabs/PyAutoLens#672: the quantitative acceptance test of the potential-correction implementation proposed by the upstream author — regularization hyper-parameters determined by maximizing the Bayesian evidence, for BOTH the one-shot (single-step) method and the iterative method, with subhalo recovery asserted at each method's evidence maximum.

scripts/imaging/subhalo_recovery_evidence.py (new): on the standard 120×120 mock of subhalo_recovery.py, (1) a 13×12 grid over Matern (coefficient, scale) of the one-shot joint inversion's evidence via dense_util.log_evidence_from_fixed_curvature (per-sample cost = two reg-matrix builds + three Choleskys — the curvature matrix, data vector and mapping are regularization-independent and cached), and (2) a 5×5 grid where the LM engine runs to convergence from a cold start (identity damping + stall guards, PyAutoLens#676) and returns the converged Laplace evidence. Evidence surfaces are saved to npz before any assertion.

Findings (full analysis + review report on the issue): both methods localize the input subhalo at their evidence maxima (peaks 0.36"/0.16" from truth), but within the Matern family evidence-max ≠ map-fidelity-max — the +190-nat preference for shorter scales decomposes as +401 prior-misfit relief (a compact cuspy NFW dkappa misfits a long-scale smooth prior) +102 chi² −313 Occam, and the shorter-scale prior dilutes map correlation (0.13 at the max vs 0.82 on the hand-calibrated ridge). The acceptance criterion therefore asserts localization at the evidence max (distance thresholds + corr noise floor) plus a ridge reference proving the family contains high-fidelity solutions (corr > 0.5 at c=2000, s=4).

Runtime is one to a few hours by design (validation, not smoke): excluded from the automated runners via config/build/no_run.yaml.

Scripts Changed

  • scripts/imaging/subhalo_recovery_evidence.py — new evidence-sampled acceptance test (both methods, grids + acceptance assertions; results npz saved pre-assert; gc.collect() per iterative point).
  • config/build/no_run.yaml — SLOW exclusion for the new script (manual validation, ~1-2h by design).

Upstream PR

PyAutoLabs/PyAutoLens#676 — the iterative leg requires the identity-damping default and stall guards (a Marquardt-damped cold start under-converges at every grid point; without the guards each point pays a ~10× rejection storm).

Test Plan

  • Full end-to-end solo run passes all acceptance assertions (log + npz archived on the issue/task worktree).
  • Script excluded from smoke/release runners (no_run.yaml); smoke count unchanged 21/21.

🤖 Generated with Claude Code

…72 phase 3)

Grids the Matern (coefficient, scale) evidence of BOTH potential-correction
methods — one-shot via the fixed-curvature fast path, iterative to
convergence under identity damping — and asserts the input subhalo is
localized at each method's evidence maximum, with a ridge reference proving
the family contains high-fidelity recoveries. no_run-excluded (manual
validation, ~1-2h by design).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Jul 31, 2026
@Jammy2211
Jammy2211 merged commit 86d0c82 into main Jul 31, 2026
3 checks passed
@Jammy2211
Jammy2211 deleted the feature/potential-correction-evidence branch July 31, 2026 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pending-release PR queued for the next release build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant