diff --git a/active.md b/active.md index beeb57b8..cd9c39b9 100644 --- a/active.md +++ b/active.md @@ -11,18 +11,6 @@ Disjoint from the registered mge-lane-death claim (research/mge-lane-death, never created). This is the canonical copy of the 2026-08-17 human-approved inference programme. -## constant-zeroth-repair -- issue: https://github.com/PyAutoLabs/PyAutoArray/issues/448 -- status: remote-handoff (prepared 2026-08-17 for a cloud/phone session; NO local worktree claim) -- worktree: none (remote session clones PyAutoLabs/PyAutoArray and branches itself) -- repos: - - PyAutoArray: feature/constant-zeroth-repair (to be created by the remote session) -- prompt: active/constant_zeroth_broken_dead_code.md -- note: Phase 8C of the inference programme (autolens_profiling#134). CPU-only, numpy-only - tests — deliberately chosen for a no-GPU remote session. PyAutoArray's other claim - (version-stamp-sync-guards) touches only `autoarray/__init__.py` + `files/release.sh`; - this task touches `autoarray/inversion/regularization/` — file-disjoint. Merge is human. - ## positions-lh-penalty-accumulation - issue: https://github.com/PyAutoLabs/PyAutoLens/issues/699 - status: pr-open (https://github.com/PyAutoLabs/PyAutoLens/pull/700) diff --git a/active/constant_zeroth_broken_dead_code.md b/active/constant_zeroth_broken_dead_code.md deleted file mode 100644 index cbd83c52..00000000 --- a/active/constant_zeroth_broken_dead_code.md +++ /dev/null @@ -1,59 +0,0 @@ -# ConstantZeroth regularization is broken twice over — dead code presenting as a feature - -Type: bug -Target: autoarray -Repos: -- @PyAutoArray -Difficulty: small -Autonomy: supervised -Priority: normal -Status: draft - -Found during the reg-logdet investigation (autolens_workspace_developer#104 -follow-up), and **independently confirmed by a second reviewer who actually ran -it**. `al.reg.ConstantZeroth` has never worked through its class API — it raises -before returning a matrix — yet it is a public, exported scheme -(`autoarray/inversion/regularization/__init__.py`). - -Two distinct defects in `autoarray/inversion/regularization/constant_zeroth.py`: - -1. **Shape bug.** `constant_zeroth_regularization_matrix_from` builds the - neighbour term `const` as an `S x S` matrix (`S` = number of mesh pixels) but - then builds the zeroth term as `xp.eye(P)` where `P = neighbors.shape[1]` is - the **neighbour count** (e.g. 4), not `S`. `const + zeroth` therefore - broadcasts `900x900 + 4x4` and raises. `constant_zeroth.py:68-72`: - ```python - reg_coeff = coefficient_zeroth**2.0 - zeroth = xp.eye(P) * reg_coeff # P is the neighbour count, should be S - return const + zeroth - ``` - The zeroth-order term is meant to be a full `S x S` scaled identity - (`+lam_z^2` on every pixel's diagonal), which is precisely the term that would - lift the graph-Laplacian null mode and make this scheme well-conditioned. - -2. **Missing-argument bug.** `ConstantZeroth.regularization_matrix_from` (the - class API path) does not pass `neighbors_sizes` to - `constant_zeroth_regularization_matrix_from`, so a correctly-shaped call raises - `TypeError` before reaching the shape bug. Verify the exact call site and - signature. - -**Why this matters beyond "a broken scheme":** in the reg-logdet investigation, -`ConstantZeroth` was hypothesised to be the *already-correct* answer — a scheme -that adds a genuine model-scaled zeroth-order term (`+lam_z^2 * I`) lifting the -null mode, immune to the `1e-8`-below-the-noise-floor conditioning collapse that -afflicts `Constant`/`Adapt`. That hypothesis is **dead on arrival** because the -scheme itself is dead. Fixing it would resurrect a genuinely useful, -well-conditioned regularization option — and is a prerequisite for "just point -users at the zeroth-order variant" ever being a real answer. - -Task: reproduce both failures on clean main FIRST (a two-line call through the -class API, then through the function with correct args). Then fix the `eye(P)` → -`eye(S)`/`S x S` shape and thread `neighbors_sizes`. Check the sibling -`adapt_split_zeroth.py` and `brightness_zeroth.py` for the same `eye(P)` / missing --arg pattern — they may share the copy-paste. Add a unit test that builds the -matrix and asserts shape `(S, S)` and positive-definiteness (the whole point of a -zeroth-order term is that the result has NO null mode). Numpy-only test per repo -policy. - -Do NOT bundle this with the reg-logdet log-det change or the Adapt double-square -probe — it is an independent, self-contained defect. diff --git a/complete/2026/08/constant-zeroth-repair.md b/complete/2026/08/constant-zeroth-repair.md new file mode 100644 index 00000000..6f67f3f4 --- /dev/null +++ b/complete/2026/08/constant-zeroth-repair.md @@ -0,0 +1,124 @@ +- issue: https://github.com/PyAutoLabs/PyAutoArray/issues/448 (CLOSED completed) +- pr: https://github.com/PyAutoLabs/PyAutoArray/pull/449 — MERGED 2026-08-18 as `74cf5a0`, 2 files, +99/-1 +- classification: library (PyAutoArray) — bug, Phase 8C of the inference programme (autolens_profiling#134) +- branch: `feature/constant-zeroth-repair` (remote cloud session; the issue's suggested branch, not the session's `claude/…` default) +- worktree: none — remote-handoff task by design (CPU-only, numpy-only tests); no local claim was ever made + +## What was wrong + +`al.reg.ConstantZeroth` was dead code presenting as a public feature — two +independent defects in `autoarray/inversion/regularization/constant_zeroth.py`, +both reproduced on clean main (`7d1906e`) before any fix (tracebacks on #448): + +1. **`eye(P)` shape bug** — the zeroth term was `xp.eye(P)` with + `P = neighbors.shape[1]` (neighbour-column count, e.g. 4), not `S` (mesh + pixel count), so `const + zeroth` raised a broadcast `ValueError` whenever + `S != P`. +2. **Missing `neighbors_sizes` at the class API** — + `ConstantZeroth.regularization_matrix_from` omitted the required argument, + raising `TypeError` before the shape bug was even reached. + +## Fix + +Zeroth term is now the full `S x S` scaled identity +(`xp.eye(S) * coefficient_zeroth**2`, matching `zeroth.py`'s `eye(pixels)` +semantics); the class API threads `neighbors_sizes=linear_obj.neighbors.sizes` +mirroring `Constant`. Siblings `adapt_split_zeroth.py` / `brightness_zeroth.py` +checked clean — their zeroth terms are `xp.diag` over per-pixel weights, no +copy-paste pattern. Four numpy-only tests added +(`test_autoarray/inversion/regularizations/test_constant_zeroth.py`): shape +`(S, S)` with `S != P`, no null mode (`min eigvalsh >= lambda_z^2`), reduction +to Constant `+ lambda_z^2 * I`, class-API smoke. Constant/Adapt/AdaptSplit +numerics and `autoarray/__init__.py` untouched. + +**Null-mode-lift verification** (9-pixel rectangular fixture, +`lambda_n = lambda_z = 1`): Constant min eigenvalue `1e-8` (jitter-floor null +mode, condition number `6e8`) → ConstantZeroth min eigenvalue `1.0 = lambda_z^2` +(condition number `7.0`); element-wise residual against `Constant + lambda_z^2 I` +exactly `0.0`. Recorded in the PR body for Phase 8C to cite. The fix makes the +scheme WORK; it does not recommend it over AdaptSplit. + +## Traps and findings + +- **No numerics or unique-identifier impact.** The class API raised + unconditionally, so no fit anywhere ever produced a likelihood through it; + `__init__` and instance attributes are unchanged, so PyAutoFit identifiers + (built from model composition, not method bodies) are stable — no output + paths orphaned. +- **One silent-wrongness edge existed in the old code:** with `P == 1` the + `(S,S) + (1,1)` broadcast *succeeded* and added `lambda_z^2` to every element + instead of the diagonal. Pathological mesh, unreachable via the class API, + now correct — but any hand-rolled direct util call on such a mesh would + (correctly) change value. +- **`zeroth.py` observed but not touched:** its class API drops `xp` + (`zeroth_regularization_matrix_from(..., pixels=...)` without `xp=xp`) — a + latent JAX-backend drop, out of scope here, worth a hygiene glance. +- **Remote-container environment traps:** `autonerves` requires Python >=3.12 + while the container default is 3.11 (use `uv venv --python 3.12`); the full + suite needs the dev extras `numba` and `pynufft` or 4 tests fail on import; + the 3 `test__nufft_pynufft__*` transformer tests fail regardless on modern + scipy (`scipy.linalg.pinv2` removed, inside pynufft) — pre-existing on clean + main, unrelated. Final run: 988 passed, 59 skipped, those 3 env failures. + Repo CI (3.12/3.13) was fully green on the PR. + +## Original prompt + +# ConstantZeroth regularization is broken twice over — dead code presenting as a feature + +Type: bug +Target: autoarray +Repos: +- @PyAutoArray +Difficulty: small +Autonomy: supervised +Priority: normal +Status: draft + +Found during the reg-logdet investigation (autolens_workspace_developer#104 +follow-up), and **independently confirmed by a second reviewer who actually ran +it**. `al.reg.ConstantZeroth` has never worked through its class API — it raises +before returning a matrix — yet it is a public, exported scheme +(`autoarray/inversion/regularization/__init__.py`). + +Two distinct defects in `autoarray/inversion/regularization/constant_zeroth.py`: + +1. **Shape bug.** `constant_zeroth_regularization_matrix_from` builds the + neighbour term `const` as an `S x S` matrix (`S` = number of mesh pixels) but + then builds the zeroth term as `xp.eye(P)` where `P = neighbors.shape[1]` is + the **neighbour count** (e.g. 4), not `S`. `const + zeroth` therefore + broadcasts `900x900 + 4x4` and raises. `constant_zeroth.py:68-72`: + ```python + reg_coeff = coefficient_zeroth**2.0 + zeroth = xp.eye(P) * reg_coeff # P is the neighbour count, should be S + return const + zeroth + ``` + The zeroth-order term is meant to be a full `S x S` scaled identity + (`+lam_z^2` on every pixel's diagonal), which is precisely the term that would + lift the graph-Laplacian null mode and make this scheme well-conditioned. + +2. **Missing-argument bug.** `ConstantZeroth.regularization_matrix_from` (the + class API path) does not pass `neighbors_sizes` to + `constant_zeroth_regularization_matrix_from`, so a correctly-shaped call raises + `TypeError` before reaching the shape bug. Verify the exact call site and + signature. + +**Why this matters beyond "a broken scheme":** in the reg-logdet investigation, +`ConstantZeroth` was hypothesised to be the *already-correct* answer — a scheme +that adds a genuine model-scaled zeroth-order term (`+lam_z^2 * I`) lifting the +null mode, immune to the `1e-8`-below-the-noise-floor conditioning collapse that +afflicts `Constant`/`Adapt`. That hypothesis is **dead on arrival** because the +scheme itself is dead. Fixing it would resurrect a genuinely useful, +well-conditioned regularization option — and is a prerequisite for "just point +users at the zeroth-order variant" ever being a real answer. + +Task: reproduce both failures on clean main FIRST (a two-line call through the +class API, then through the function with correct args). Then fix the `eye(P)` → +`eye(S)`/`S x S` shape and thread `neighbors_sizes`. Check the sibling +`adapt_split_zeroth.py` and `brightness_zeroth.py` for the same `eye(P)` / missing +-arg pattern — they may share the copy-paste. Add a unit test that builds the +matrix and asserts shape `(S, S)` and positive-definiteness (the whole point of a +zeroth-order term is that the result has NO null mode). Numpy-only test per repo +policy. + +Do NOT bundle this with the reg-logdet log-det change or the Adapt double-square +probe — it is an independent, self-contained defect. diff --git a/complete/index.md b/complete/index.md index 055a0feb..6ff26619 100644 --- a/complete/index.md +++ b/complete/index.md @@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema: only then grep a dated bucket. Curators: edit the band between the CURATED markers; everything below GENERATED is rebuilt. -1001 records across 7 buckets. +1002 records across 7 buckets. ## Highlights @@ -38,6 +38,7 @@ _(curate hard-won records here — survives regeneration.)_ - [compile-axis-triage-drift](2026/08/compile-axis-triage-drift.md) - [compile-warm-baseline-dashboard](2026/08/compile-warm-baseline-dashboard.md) - [conductor-discovery-lifecycle-split](2026/08/conductor-discovery-lifecycle-split.md) — closed on merge +- [constant-zeroth-repair](2026/08/constant-zeroth-repair.md) — CLOSED completed - [correct-circular-sersic-hazard](2026/08/correct-circular-sersic-hazard.md) — Corrected the circular Sersic hazard in the actual fitted ell_comps coordinates. The q-angle structural findin… - [covariance-interpolator-rng-seed](2026/08/covariance-interpolator-rng-seed.md) — auto-closed by the merge - [crashed-run-poisons-resume](2026/08/crashed-run-poisons-resume.md) — A run interrupted while writing output left a half-written JSON file, diff --git a/dashboard.md b/dashboard.md index 07fe108f..72803c8e 100644 --- a/dashboard.md +++ b/dashboard.md @@ -8,7 +8,7 @@ Tasks only — the organism's health lives with the Heart (`/health`), not here. | Where | Count | |-------|------:| -| [In flight](#in-flight) (`active/`) | 12 | +| [In flight](#in-flight) (`active/`) | 11 | | [Parked](#parked) (`parked.md`) | 5 | | [Planned](#planned) (`planned.md`) | 7 | | [Backlog](#backlog) (`draft/`) | 139 | @@ -48,7 +48,6 @@ Live on GitHub: [open issues](https://github.com/search?q=org%3APyAutoLabs+is%3A Issued — each has an open GitHub issue and usually a branch. The full record for each is in [`active.md`](active.md). -- [ConstantZeroth regularization is broken twice over — dead code presenting](active/constant_zeroth_broken_dead_code.md) — [issue #448](https://github.com/PyAutoLabs/PyAutoArray/issues/448) — remote-handoff (prepared 2026-08-17 for a cloud/phone session; NO local worktree claim) - [Address ECEB editorial comments on ECLIPSE-C](active/euclid_eceb_editorial_revision.md) - [Commit the inference-methods programme plan + knowledge ledger into autolens_profiling](active/inference_programme_ledger.md) — [issue #134](https://github.com/PyAutoLabs/autolens_profiling/issues/134) — pr-open (https://github.com/PyAutoLabs/autolens_profiling/pull/135) - [JAX-native posterior sampler wave — ranked shortlist from the 2026-07-16](active/jax_native_posterior_sampler_wave.md) — [issue #113](https://github.com/PyAutoLabs/autolens_workspace_developer/issues/113) — PARKED 2026-07-24 — stage (a) POSITIVE: warm-started gradient SMC SAMPLES (acc 0.80->0.17 across tempering, einstein_radius…