From c0407e38ad90054d6c76eb407388e907eaa17fbd Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 24 Aug 2026 17:32:21 +0000 Subject: [PATCH 1/9] =?UTF-8?q?prompt:=20route=20test-mode-bypass-assertio?= =?UTF-8?q?n-ties=20(#1519)=20=E2=86=92=20start=5Flibrary?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- active.md | 20 +++++++++++ ...test_mode_bypass_ordered_assertion_ties.md | 1 + dashboard.html | 22 ++++++------ dashboard.md | 34 +++++++++---------- 4 files changed, 49 insertions(+), 28 deletions(-) rename {draft/bug/autofit => active}/test_mode_bypass_ordered_assertion_ties.md (99%) diff --git a/active.md b/active.md index b041750d..77ca87ee 100644 --- a/active.md +++ b/active.md @@ -1,5 +1,25 @@ # Active Tasks +## test-mode-bypass-assertion-ties +- issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1519 +- issued: 2026-08-24 +- prompt: active/test_mode_bypass_ordered_assertion_ties.md +- status: library-dev +- worktree: ~/Code/PyAutoLabs-wt/test-mode-bypass-assertion-ties +- repos: +- summary: | + TEST_MODE=2/3 bypass evaluates at the prior medians, so a model with identical + priors plus an ordering assertion ties and raises FitException. Verified on + PyAutoFit main d3625a8 that this reproduces at THREE sites, not the one the + prompt names: the instantiation outside the try (abstract_search.py:1007), the + uniformly-scaled fake samples (:1112, so every stored sample fails too), and + SamplesSummary.max_log_likelihood (interface.py:122, recover="raise") which + makes result.max_log_likelihood_instance raise SamplesException. TEST_MODE=3 is + therefore broken as well. Chosen fix (human-approved): share TEST_MODE=1's + existing deterministic valid-point search (medians, then seeded prior draws) + so the bypass evaluates AND stores an assertion-valid vector, fixing all three. + Next step: /start_library. + ## transformed-message-factor-gradient-unpack - issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1501 (issued 2026-08-19) - issued: 2026-08-19 diff --git a/draft/bug/autofit/test_mode_bypass_ordered_assertion_ties.md b/active/test_mode_bypass_ordered_assertion_ties.md similarity index 99% rename from draft/bug/autofit/test_mode_bypass_ordered_assertion_ties.md rename to active/test_mode_bypass_ordered_assertion_ties.md index 83db6ace..b8e67430 100644 --- a/draft/bug/autofit/test_mode_bypass_ordered_assertion_ties.md +++ b/active/test_mode_bypass_ordered_assertion_ties.md @@ -9,6 +9,7 @@ Autonomy: supervised Priority: normal Status: formalised — STILL REPRODUCES; see the 2026-08-09 note before grading this against main Filed: 2026-07-17 (backfilled from git) +Issued: 2026-08-24 ## 2026-08-09 — do NOT mistake the adjacent FitException catch for this fix diff --git a/dashboard.html b/dashboard.html index 35724b62..2627cf01 100644 --- a/dashboard.html +++ b/dashboard.html @@ -138,7 +138,7 @@

PyAutoMindDashboard

Intent. Priority. Flow.

Every task the Mind is holding. Tap a task's 📋 and its /start_dev command is on your clipboard — paste it into a Claude Code chat to route Claude straight to that task. Recent is the same work by date — what has been happening rather than what to do next.

- +

markdown version

Start here

Highest priority (filed as high) — showing 12 of 17

@@ -162,6 +162,7 @@

In flight Issued — each has an open GitHub issue and usually a branch.

@PyAutoFit TransformedMessage.factor_gradient crashes on first callissue #1501 — issued 2026-08-19HOLD — do not start dev. Fix-or-delete hangs off the PyAutoFit#1498 logpdf-contract

+

Parked markdown version

3 task(s) @@ -180,7 +181,7 @@

Planned

latent-nan-guard-honest-run — planned 2026-07-22

Backlog markdown version

-

153 filed prompts, not started — sorted most-pickable first (priority, then size). 25 of them belong to an epic and are listed only under Epics below.

+

152 filed prompts, not started — sorted most-pickable first (priority, then size). 25 of them belong to an epic and are listed only under Epics below.

feature — 29 @@ -214,14 +215,13 @@

Backlog

Teach repos_sync --write to stamp organ config surfaces✨ featurepyautomindhardsupervisedlow

-bug — 30 +bug — 29

Fix release JAX runtime compatibility and likelihood parity🐛 bughealth_fixestoo-largesupervisedhigh

Fix JIT quick-update visualization output regressions🐛 bughealth_fixestoo-largesupervisedhigh

Fix release result/sample parameter-path regressions🐛 bughealth_fixestoo-largesupervisedhigh

LogGaussianPrior misreports its own support as (-inf, inf)🐛 bugautofitsmallsupervisednormal

-

TEST_MODE bypass crashes on ordered-parameter assertion ties🐛 bugautofitsmallsupervisednormal

@@ -348,6 +348,12 @@

Backlog 2026-08-24 issued +TEST_MODE bypass crashes on ordered-parameter assertion ties + + + +2026-08-24 +issued Properly time and profile the smoke/release script surface @@ -393,7 +399,7 @@

Backlog The reconstruction noise map describes a different estimator than the… - + 2026-08-22 filed Remove pynufft + legacy TransformerNUFFTPyNUFFT @@ -633,12 +639,6 @@

Backlog autolens_workspace_developer: broad stale-API rot (56 symbols, no CI) - -2026-08-04 -filed -aplt.Output stale-API drift in the remaining workspace repos - -

Epics markdown version

diff --git a/dashboard.md b/dashboard.md index 97d02e9b..65f41dfc 100644 --- a/dashboard.md +++ b/dashboard.md @@ -8,10 +8,10 @@ Every task the Mind is holding, on one page: what is in flight, what is parked, | Where | Count | |-------|------:| -| [In flight](#in-flight) (`active/`) | 2 | +| [In flight](#in-flight) (`active/`) | 3 | | [Parked](#parked) (`parked.md`) | 3 | | [Planned](#planned) (`planned.md`) | 6 | -| [Backlog](#backlog) (`draft/`) | 153 | +| [Backlog](#backlog) (`draft/`) | 152 | ## Start here @@ -159,6 +159,14 @@ Issued — each has an open GitHub issue and usually a branch. The full record f
+
📋 TEST_MODE bypass crashes on ordered-parameter assertion tiesissue #1519 — issued 2026-08-24 — library-dev + +``` +/start_dev active/test_mode_bypass_ordered_assertion_ties.md +``` + +
+ ## Parked Started or scoped, not currently in flight — resume by moving the row back to `active.md`. Full detail in [`parked.md`](parked.md). @@ -251,7 +259,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned. ## Backlog -**153** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **25** of them belong to an epic and are listed only under [Epics](#epics) below. +**152** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **25** of them belong to an epic and are listed only under [Epics](#epics) below.
feature — 29 @@ -491,7 +499,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-bug — 30 +bug — 29
📋 Fix release JAX runtime compatibility and likelihood parity — health_fixes · too-large · supervised · high @@ -541,14 +549,6 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-
📋 TEST_MODE bypass crashes on ordered-parameter assertion ties — autofit · small · supervised · normal - -``` -/start_dev draft/bug/autofit/test_mode_bypass_ordered_assertion_ties.md -``` - -
-
📋 point.py JAX-vmap parity assert is non-deterministic under the smoke env — autolens · small · supervised · normal ``` @@ -1329,6 +1329,7 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | Date | Event | Task | |------|-------|------| | 2026-08-24 | filed | The weekly smoke run's timings land in results-* under no… | +| 2026-08-24 | issued | TEST_MODE bypass crashes on ordered-parameter assertion ties | | 2026-08-24 | issued | Properly time and profile the smoke/release script surface | | 2026-08-23 | filed | pynufft removal: unswept downstream residue (1 hard break + stale… | | 2026-08-23 | filed | Phase 3: stop installing pynufft in Hands/Heart CI and PyAutoCTI… | @@ -1337,12 +1338,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-22 | filed | multi_dataset/jax_likelihood scripts hang to the timeout cap (XLA… | | 2026-08-22 | filed | Untrack the generated FITS test artifacts in autoarray | | 2026-08-22 | filed | The reconstruction noise map describes a different estimator than the… | -| 2026-08-22 | filed | Remove pynufft + legacy TransformerNUFFTPyNUFFT |
… 10 more (40 left) | Date | Event | Task | |------|-------|------| +| 2026-08-22 | filed | Remove pynufft + legacy TransformerNUFFTPyNUFFT | | 2026-08-22 | filed | Point-source JSON datasets record no resolution regime | | 2026-08-22 | filed | Is Intel macOS a supported platform, and what is the numpy-only… | | 2026-08-22 | filed | Defer the eager scipy.sparse import in derivative_util (~0.10 s of… | @@ -1352,12 +1353,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-20 | filed | Numba CPU likelihood phase 1: batched MGE convolution +… | | 2026-08-19 | filed | status.sh --repos sources a file that no longer exists | | 2026-08-19 | filed | jax 0.11 breaks beta/gamma message log_partition under jit… | -| 2026-08-19 | filed | autolens_workspace_test jax_likelihood pins: 4 scripts fail smoke on… |
… 10 more (30 left) | Date | Event | Task | |------|-------|------| +| 2026-08-19 | filed | autolens_workspace_test jax_likelihood pins: 4 scripts fail smoke on… | | 2026-08-19 | filed | autofit_profiling: bootstrap the repo + general PyAutoFit profiling… | | 2026-08-19 | filed | autoreduce 0.9 on PyPI never got the Python 3.12 floor | | 2026-08-19 | issued | @PyAutoFit TransformedMessage.factor_gradient crashes on first… | @@ -1367,12 +1368,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-19 | filed | PyAutoConf rename leftovers in Brain functional surfaces | | 2026-08-19 | filed | Explore: dashboardify the Brain's operational surfaces with pasteable… | | 2026-08-19 | filed | Deduplicate repos_sync.py's check/write pairs | -| 2026-08-19 | filed | Bug in autocti_workspace: the dataset_1d results/database example… |
… 10 more (20 left) | Date | Event | Task | |------|-------|------| +| 2026-08-19 | filed | Bug in autocti_workspace: the dataset_1d results/database example… | | 2026-08-18 | parked | single-source-density-design | | 2026-08-18 | parked | prior-message-collapse-design | | 2026-08-18 | filed | @PyAutoFit TransformedMessage.logpdf/pdf omit the transform… | @@ -1382,12 +1383,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-14 | filed | Three jax_likelihood pins are stale by ~1.24e-4 and fail the smoke… | | 2026-08-09 | found | isothermal-ell-sph-oversampling-at-the-cusp | | 2026-08-08 | parked | pyautoreduce-slacs1430-acs-comparison | -| 2026-08-08 | filed | Regenerate autolens_workspace markdown/ so the MGE pages show… |
… 10 more (10 left) | Date | Event | Task | |------|-------|------| +| 2026-08-08 | filed | Regenerate autolens_workspace markdown/ so the MGE pages show… | | 2026-08-07 | filed | autofit.plot functions accept **kwargs and silently discard them | | 2026-08-07 | filed | Regenerate setup_notebook-drifted notebooks in… | | 2026-08-06 | filed | Triage: Convolver "No blurring_image provided" warning in canonical… | @@ -1397,7 +1398,6 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-04 | filed | dataset/imaging/jwst_lw is untracked because the gitignore was never… | | 2026-08-04 | filed | cosmos_web_ring stores boolean masks as float64, wasting ~3.4 MB of… | | 2026-08-04 | filed | autolens_workspace_developer: broad stale-API rot (56 symbols, no CI) | -| 2026-08-04 | filed | aplt.Output stale-API drift in the remaining workspace repos |
From 1876aed7d4506cc0630d2bb9dd82982514eee33a Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 24 Aug 2026 17:43:30 +0000 Subject: [PATCH 2/9] prompt: register test-mode-bypass-assertion-ties library repo + implementation state --- active.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/active.md b/active.md index 77ca87ee..86ed7027 100644 --- a/active.md +++ b/active.md @@ -5,8 +5,9 @@ - issued: 2026-08-24 - prompt: active/test_mode_bypass_ordered_assertion_ties.md - status: library-dev -- worktree: ~/Code/PyAutoLabs-wt/test-mode-bypass-assertion-ties +- environment: web-github (no local worktree; PyAutoBrain/skills/WORKFLOW.md) - repos: + - PyAutoFit: feature/test-mode-bypass-assertion-ties - summary: | TEST_MODE=2/3 bypass evaluates at the prior medians, so a model with identical priors plus an ordering assertion ties and raises FitException. Verified on @@ -18,7 +19,12 @@ therefore broken as well. Chosen fix (human-approved): share TEST_MODE=1's existing deterministic valid-point search (medians, then seeded prior draws) so the bypass evaluates AND stores an assertion-valid vector, fixing all three. - Next step: /start_library. + Implemented and pushed: PyAutoFit feature/test-mode-bypass-assertion-ties + (d49275c33). Shared _test_mode_valid_parameter_vector now picks the bypass + point (medians, then seeded prior draws) so modes 2 AND 3 store an + assertion-valid vector. Verified: repro failed both modes before, passes + after; 5 new tests fail without the source change; full test_autofit + 2016 passed / 34 skipped. Next step: /ship_library (PR not yet opened). ## transformed-message-factor-gradient-unpack - issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1501 (issued 2026-08-19) From 3d2b45ed021665fd3558c8e9dd22316596d843b6 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 24 Aug 2026 17:45:59 +0000 Subject: [PATCH 3/9] =?UTF-8?q?prompt:=20test-mode-bypass-assertion-ties?= =?UTF-8?q?=20=E2=86=92=20library-shipped=20(PyAutoFit#1520)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- active.md | 12 ++++++++++-- dashboard.html | 2 +- dashboard.md | 2 +- 3 files changed, 12 insertions(+), 4 deletions(-) diff --git a/active.md b/active.md index 86ed7027..73cde864 100644 --- a/active.md +++ b/active.md @@ -4,7 +4,8 @@ - issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1519 - issued: 2026-08-24 - prompt: active/test_mode_bypass_ordered_assertion_ties.md -- status: library-dev +- status: library-shipped, awaiting-merge +- library-pr: https://github.com/PyAutoLabs/PyAutoFit/pull/1520 (pending-release) - environment: web-github (no local worktree; PyAutoBrain/skills/WORKFLOW.md) - repos: - PyAutoFit: feature/test-mode-bypass-assertion-ties @@ -24,7 +25,14 @@ point (medians, then seeded prior draws) so modes 2 AND 3 store an assertion-valid vector. Verified: repro failed both modes before, passes after; 5 new tests fail without the source change; full test_autofit - 2016 passed / 34 skipped. Next step: /ship_library (PR not yet opened). + 2016 passed / 34 skipped. Shipped as PyAutoFit#1520 (pending-release). + GATE CAVEAT: pyauto-heart is unreachable from this web-github session, so + the readiness gate ran in the WORKFLOW.md fallback form (full library suite + as the gate) — this is NOT a Heart GREEN verdict; confirm from a + Heart-reachable environment before merge. Workspace impact: API Changes are + "none — internal", but the workspace clones are absent here so the + reference.md grep was NOT run; option (iii) is inferred, not measured. + Next step: /prm (CI green → merge → close-out). ## transformed-message-factor-gradient-unpack - issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1501 (issued 2026-08-19) diff --git a/dashboard.html b/dashboard.html index 2627cf01..24b994b1 100644 --- a/dashboard.html +++ b/dashboard.html @@ -162,7 +162,7 @@

In flight Issued — each has an open GitHub issue and usually a branch.

@PyAutoFit TransformedMessage.factor_gradient crashes on first callissue #1501 — issued 2026-08-19HOLD — do not start dev. Fix-or-delete hangs off the PyAutoFit#1498 logpdf-contract

- +

TEST_MODE bypass crashes on ordered-parameter assertion tiesissue #1519 — issued 2026-08-24library-shipped, awaiting-merge

Parked markdown version

3 task(s) diff --git a/dashboard.md b/dashboard.md index 65f41dfc..3c5dec10 100644 --- a/dashboard.md +++ b/dashboard.md @@ -159,7 +159,7 @@ Issued — each has an open GitHub issue and usually a branch. The full record f
-
📋 TEST_MODE bypass crashes on ordered-parameter assertion tiesissue #1519 — issued 2026-08-24 — library-dev +
📋 TEST_MODE bypass crashes on ordered-parameter assertion tiesissue #1519 — issued 2026-08-24 — library-shipped, awaiting-merge ``` /start_dev active/test_mode_bypass_ordered_assertion_ties.md From 815f417f0f26d664a65c2385860e58645c3ed7e3 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 24 Aug 2026 17:52:14 +0000 Subject: [PATCH 4/9] =?UTF-8?q?prompt:=20ship=20test-mode-bypass-assertion?= =?UTF-8?q?-ties=20(#1519)=20=E2=86=92=20complete?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- active.md | 34 ----- ...test_mode_bypass_ordered_assertion_ties.md | 79 ---------- .../08/test-mode-bypass-assertion-ties.md | 140 ++++++++++++++++++ complete/index.md | 3 +- dashboard.html | 17 +-- dashboard.md | 20 +-- 6 files changed, 156 insertions(+), 137 deletions(-) delete mode 100644 active/test_mode_bypass_ordered_assertion_ties.md create mode 100644 complete/2026/08/test-mode-bypass-assertion-ties.md diff --git a/active.md b/active.md index 73cde864..b041750d 100644 --- a/active.md +++ b/active.md @@ -1,39 +1,5 @@ # Active Tasks -## test-mode-bypass-assertion-ties -- issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1519 -- issued: 2026-08-24 -- prompt: active/test_mode_bypass_ordered_assertion_ties.md -- status: library-shipped, awaiting-merge -- library-pr: https://github.com/PyAutoLabs/PyAutoFit/pull/1520 (pending-release) -- environment: web-github (no local worktree; PyAutoBrain/skills/WORKFLOW.md) -- repos: - - PyAutoFit: feature/test-mode-bypass-assertion-ties -- summary: | - TEST_MODE=2/3 bypass evaluates at the prior medians, so a model with identical - priors plus an ordering assertion ties and raises FitException. Verified on - PyAutoFit main d3625a8 that this reproduces at THREE sites, not the one the - prompt names: the instantiation outside the try (abstract_search.py:1007), the - uniformly-scaled fake samples (:1112, so every stored sample fails too), and - SamplesSummary.max_log_likelihood (interface.py:122, recover="raise") which - makes result.max_log_likelihood_instance raise SamplesException. TEST_MODE=3 is - therefore broken as well. Chosen fix (human-approved): share TEST_MODE=1's - existing deterministic valid-point search (medians, then seeded prior draws) - so the bypass evaluates AND stores an assertion-valid vector, fixing all three. - Implemented and pushed: PyAutoFit feature/test-mode-bypass-assertion-ties - (d49275c33). Shared _test_mode_valid_parameter_vector now picks the bypass - point (medians, then seeded prior draws) so modes 2 AND 3 store an - assertion-valid vector. Verified: repro failed both modes before, passes - after; 5 new tests fail without the source change; full test_autofit - 2016 passed / 34 skipped. Shipped as PyAutoFit#1520 (pending-release). - GATE CAVEAT: pyauto-heart is unreachable from this web-github session, so - the readiness gate ran in the WORKFLOW.md fallback form (full library suite - as the gate) — this is NOT a Heart GREEN verdict; confirm from a - Heart-reachable environment before merge. Workspace impact: API Changes are - "none — internal", but the workspace clones are absent here so the - reference.md grep was NOT run; option (iii) is inferred, not measured. - Next step: /prm (CI green → merge → close-out). - ## transformed-message-factor-gradient-unpack - issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1501 (issued 2026-08-19) - issued: 2026-08-19 diff --git a/active/test_mode_bypass_ordered_assertion_ties.md b/active/test_mode_bypass_ordered_assertion_ties.md deleted file mode 100644 index b8e67430..00000000 --- a/active/test_mode_bypass_ordered_assertion_ties.md +++ /dev/null @@ -1,79 +0,0 @@ -# TEST_MODE bypass crashes on ordered-parameter assertion ties - -Type: bug -Target: PyAutoFit -Repos: -- @PyAutoFit -Difficulty: small -Autonomy: supervised -Priority: normal -Status: formalised — STILL REPRODUCES; see the 2026-08-09 note before grading this against main -Filed: 2026-07-17 (backfilled from git) -Issued: 2026-08-24 - -## 2026-08-09 — do NOT mistake the adjacent FitException catch for this fix - -Checked by the draft/ sweep against PyAutoFit main (`3b960609`). The bypass path -in `abstract_search.py` **now catches `exc.FitException`** and continues with the -`-1e99` sentinel, logging "TEST MODE 2: likelihood verification raised -FitException … treating as a resample-rejected instance". That reads exactly like -this prompt's suggested fix. **It is not.** The bug below still reproduces. - -The catch wraps only the likelihood call. The model instantiation is on the line -*before* the `try`: - -```python -if call_likelihood: - instance = model.instance_from_vector(vector=parameter_vector) # <-- outside - try: - log_likelihood = float(analysis.log_likelihood_function(instance)) - except exc.FitException as e: - ... -``` - -and `instance_from_vector` → `instance_for_arguments` → `check_assertions` -(`autofit/mapper/prior_model/abstract.py:193`) is precisely what raises -`exc.FitException("N assertions failed!")` when an ordering assertion ties at the -prior medians. `ignore_assertions` defaults to `False` and the bypass does not -pass it. So the assertion exception escapes the guard entirely and still -hard-fails the run. - -The upside: the fix is now a one-liner rather than the "catch and retry with a -perturbation" design sketched below. Two options, both cheap and both -deterministic: - -- move the `instance_from_vector` call inside the existing `try` — the sentinel - path already does the right thing for a rejected instance; or -- pass `ignore_assertions=True` at the bypass instantiation, on the grounds that - a verification eval at the medians is not a sampled point and assertions exist - to steer sampling. - -The second is probably the better semantics (a tied median is not a pathological -model), but it changes what the verification eval attests to — pick deliberately. -Prefer either over adding perturbation logic. - -`Difficulty:` stays small. The § Blocks note below still holds. - ---- - -Found during the CTI resurrection epic (Phase 4, 2026-07-17). `PYAUTO_TEST_MODE=2/3` -bypass evaluates the model at the **prior medians**. A model whose components have -identical priors plus an ordering assertion (the standard idiom for breaking -exchange degeneracy, e.g. PyAutoCTI trap models with -`model.add_assertion(trap_0.release_timescale < trap_1.release_timescale)`) -ties exactly at the medians, so the bypass evaluation raises -`autofit.exc.FitException: GreaterThanLessThanAssertion` and the script crashes. - -Real samplers resample assertion-failing points gracefully — this is purely a -bypass-path artifact, and it makes every ordered-trap CTI workspace script -un-smokeable at TEST_MODE=2 (reproduced with a bare -`model.instance_from_prior_medians()`; TEST_MODE=1 passes). - -Suggested fix: at the bypass evaluation, catch `FitException` from assertions -and retry with a small deterministic perturbation of the unit-cube point (or a -seeded random draw), mirroring what a real sampler does. Keep it deterministic -so smoke runs stay reproducible. - -Blocks: autocti_workspace smoke coverage of `modeling/start_here.py`-class -scripts (CTI epic Phase 5); the workspace documents the artifact in its -AGENTS.md meanwhile. diff --git a/complete/2026/08/test-mode-bypass-assertion-ties.md b/complete/2026/08/test-mode-bypass-assertion-ties.md new file mode 100644 index 00000000..5d17f9de --- /dev/null +++ b/complete/2026/08/test-mode-bypass-assertion-ties.md @@ -0,0 +1,140 @@ +## test-mode-bypass-assertion-ties +- issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1519 (closed) +- completed: 2026-08-24 +- library-pr: https://github.com/PyAutoLabs/PyAutoFit/pull/1520 (merged 438f56fac) +- summary: The PYAUTO_TEST_MODE=2/3 bypass evaluated the model at the prior + medians, so a model with identical priors plus an ordering assertion (the + exchange-degeneracy idiom, e.g. PyAutoCTI trap models) tied exactly there and + check_assertions hard-failed the run. The bypass now picks its point through a + shared `_test_mode_valid_parameter_vector` — prior medians first, then + `default_rng(seed=0)` prior draws, each candidate validated — so the vector it + evaluates AND stores satisfies the assertions. +- key-finding: **the prompt's own suggested fix would not have worked.** The + 2026-08-09 note had narrowed this to a one-liner (move `instance_from_vector` + inside the existing try, or pass `ignore_assertions=True`). Reading main + d3625a8 found THREE sites, and both one-liners fix only the first: + 1. `abstract_search.py:1007` — instantiation outside the FitException guard; + 2. `_build_fake_samples:1112` — the perturbed samples are the median vector + scaled UNIFORMLY (1.001/0.999/1.002), and a uniform scale preserves an + ordering tie, so every stored sample fails the same assertion; + 3. `SamplesSummary.max_log_likelihood` (`interface.py:122`) is `@to_instance()` + with `recover="raise"` — it extends SamplesInterface directly so it does NOT + inherit Samples' next-valid recovery — so `result.max_log_likelihood_instance` + raises SamplesException, and `Result.instance` catches only AttributeError. +- key-finding: **TEST_MODE=3 was broken too and nobody had noticed.** It never + calls instance_from_vector in the bypass, so it survived the fit and died at + the first `result.max_log_likelihood_instance`. Confirmed by reproduction, not + just by reading. Fixing the stored vector fixes modes 2 and 3 together. +- trap: do NOT re-run a bypass reproduction without clearing `output/` first. A + bypassed fit calls `paths.completed()`, so a second run with the same + unique_tag takes `result_via_completed_fit` and replays the OLD (broken) + samples — which reads exactly like "the fix didn't work". Cost one false + negative during verification. +- trap: `af.m.MockAnalysis` maps its likelihood over the model and returns a + LIST for an `af.Collection`, which the bypass's `float()` rejects. Regression + tests needed a small float-returning analysis instead. +- behaviour-change: mode 3 now instantiates the model once (previously zero + times). A model whose constructor raises a non-FitException at the medians now + fails at fit time rather than result time — same failure, surfaced earlier. + Flagged in the PR body for downstream repos. +- verification: reproduced on clean main first (mode 2 raised FitException in the + fit; mode 3 raised SamplesException at result access), then both modes complete + and select the identical vector after. All 5 new tests in + `TestBypassToleratesAssertionTies` fail against the un-patched source. Full + suite 2016 passed / 34 skipped / 0 failed; CI green on all three legs + (unittest 3.12, unittest 3.13, unittest-nojax) plus Docs. +- gate-caveat: shipped from a web-github session where `pyauto-heart` is + unreachable, so the readiness gate ran in the WORKFLOW.md fallback form (full + library suite as the gate). No Heart verdict was recorded for this task; CI + green at merge is the stronger confirmation that stands in its place. The + workspace-impact grep was likewise not run (workspace clones absent) — API + Changes are "none, internal", so option (iii) was inferred, not measured. +- follow-up: `autocti_workspace` documents this artifact in its AGENTS.md as a + workaround. Delete that note now the fix has shipped — the testmode-env-drift + precedent ("delete the trap, don't document it"). Separate repo, separate task. +- follow-up: re-enable autocti_workspace smoke coverage of the + `modeling/start_here.py`-class scripts (CTI epic Phase 5) that this unblocks. +- environment: web-github; no worktree was ever created, so there is none to + remove. PyAutoFit was worked in a session clone at /home/user/pyautofit. + +## Original prompt + +# TEST_MODE bypass crashes on ordered-parameter assertion ties + +Type: bug +Target: PyAutoFit +Repos: +- @PyAutoFit +Difficulty: small +Autonomy: supervised +Priority: normal +Status: formalised — STILL REPRODUCES; see the 2026-08-09 note before grading this against main +Filed: 2026-07-17 (backfilled from git) +Issued: 2026-08-24 + +## 2026-08-09 — do NOT mistake the adjacent FitException catch for this fix + +Checked by the draft/ sweep against PyAutoFit main (`3b960609`). The bypass path +in `abstract_search.py` **now catches `exc.FitException`** and continues with the +`-1e99` sentinel, logging "TEST MODE 2: likelihood verification raised +FitException … treating as a resample-rejected instance". That reads exactly like +this prompt's suggested fix. **It is not.** The bug below still reproduces. + +The catch wraps only the likelihood call. The model instantiation is on the line +*before* the `try`: + +```python +if call_likelihood: + instance = model.instance_from_vector(vector=parameter_vector) # <-- outside + try: + log_likelihood = float(analysis.log_likelihood_function(instance)) + except exc.FitException as e: + ... +``` + +and `instance_from_vector` → `instance_for_arguments` → `check_assertions` +(`autofit/mapper/prior_model/abstract.py:193`) is precisely what raises +`exc.FitException("N assertions failed!")` when an ordering assertion ties at the +prior medians. `ignore_assertions` defaults to `False` and the bypass does not +pass it. So the assertion exception escapes the guard entirely and still +hard-fails the run. + +The upside: the fix is now a one-liner rather than the "catch and retry with a +perturbation" design sketched below. Two options, both cheap and both +deterministic: + +- move the `instance_from_vector` call inside the existing `try` — the sentinel + path already does the right thing for a rejected instance; or +- pass `ignore_assertions=True` at the bypass instantiation, on the grounds that + a verification eval at the medians is not a sampled point and assertions exist + to steer sampling. + +The second is probably the better semantics (a tied median is not a pathological +model), but it changes what the verification eval attests to — pick deliberately. +Prefer either over adding perturbation logic. + +`Difficulty:` stays small. The § Blocks note below still holds. + +--- + +Found during the CTI resurrection epic (Phase 4, 2026-07-17). `PYAUTO_TEST_MODE=2/3` +bypass evaluates the model at the **prior medians**. A model whose components have +identical priors plus an ordering assertion (the standard idiom for breaking +exchange degeneracy, e.g. PyAutoCTI trap models with +`model.add_assertion(trap_0.release_timescale < trap_1.release_timescale)`) +ties exactly at the medians, so the bypass evaluation raises +`autofit.exc.FitException: GreaterThanLessThanAssertion` and the script crashes. + +Real samplers resample assertion-failing points gracefully — this is purely a +bypass-path artifact, and it makes every ordered-trap CTI workspace script +un-smokeable at TEST_MODE=2 (reproduced with a bare +`model.instance_from_prior_medians()`; TEST_MODE=1 passes). + +Suggested fix: at the bypass evaluation, catch `FitException` from assertions +and retry with a small deterministic perturbation of the unit-cube point (or a +seeded random draw), mirroring what a real sampler does. Keep it deterministic +so smoke runs stay reproducible. + +Blocks: autocti_workspace smoke coverage of `modeling/start_here.py`-class +scripts (CTI epic Phase 5); the workspace documents the artifact in its +AGENTS.md meanwhile. diff --git a/complete/index.md b/complete/index.md index 805d0990..a4b30ded 100644 --- a/complete/index.md +++ b/complete/index.md @@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema: only then grep a dated bucket. Curators: edit the band between the CURATED markers; everything below GENERATED is rebuilt. -1098 records across 7 buckets. +1099 records across 7 buckets. ## Highlights @@ -209,6 +209,7 @@ _(curate hard-won records here — survives regeneration.)_ - [tenant-firewall-drift-aug](2026/08/tenant-firewall-drift-aug.md) — issue #198; 9 → OK, and a green `--check` is only evidence for the organs actually checked out - [tenant-firewall-hygiene-extras](2026/08/tenant-firewall-hygiene-extras.md) — auto-closed by the merge - [tenth-sample-hardcoded-index](2026/08/tenth-sample-hardcoded-index.md) — results/database/aggregator tutorials hardcoded stored-sample index 9 ("the tenth +- [test-mode-bypass-assertion-ties](2026/08/test-mode-bypass-assertion-ties.md) — The PYAUTO_TEST_MODE=2/3 bypass evaluated the model at the prior - [test-mode-fit-exception-finalization](2026/08/test-mode-fit-exception-finalization.md) — `PYAUTO_TEST_MODE=1` no longer finalizes a model point whose reconstruction raises `FitException`; it substitu… - [test-mode-samples-info-hook-contract](2026/08/test-mode-samples-info-hook-contract.md) — `NonLinearSearch._test_mode_samples_info()`'s docstring told subclasses - [test-performance-board](2026/08/test-performance-board.md) diff --git a/dashboard.html b/dashboard.html index 24b994b1..539a0f4d 100644 --- a/dashboard.html +++ b/dashboard.html @@ -138,7 +138,7 @@

PyAutoMindDashboard

Intent. Priority. Flow.

Every task the Mind is holding. Tap a task's 📋 and its /start_dev command is on your clipboard — paste it into a Claude Code chat to route Claude straight to that task. Recent is the same work by date — what has been happening rather than what to do next.

-
  • 3In flight
  • 3Parked
  • 6Planned
  • 152Backlog
+
  • 2In flight
  • 3Parked
  • 6Planned
  • 152Backlog

markdown version

Start here

Highest priority (filed as high) — showing 12 of 17

@@ -162,7 +162,6 @@

In flight Issued — each has an open GitHub issue and usually a branch.

@PyAutoFit TransformedMessage.factor_gradient crashes on first callissue #1501 — issued 2026-08-19HOLD — do not start dev. Fix-or-delete hangs off the PyAutoFit#1498 logpdf-contract

-

TEST_MODE bypass crashes on ordered-parameter assertion tiesissue #1519 — issued 2026-08-24library-shipped, awaiting-merge

Parked markdown version

3 task(s) @@ -348,12 +347,6 @@

Backlog 2026-08-24 issued -TEST_MODE bypass crashes on ordered-parameter assertion ties - - - -2026-08-24 -issued Properly time and profile the smoke/release script surface @@ -399,7 +392,7 @@

Backlog The reconstruction noise map describes a different estimator than the… - + 2026-08-22 filed Remove pynufft + legacy TransformerNUFFTPyNUFFT @@ -639,6 +632,12 @@

Backlog autolens_workspace_developer: broad stale-API rot (56 symbols, no CI) + +2026-08-04 +filed +aplt.Output stale-API drift in the remaining workspace repos + +

Epics markdown version

diff --git a/dashboard.md b/dashboard.md index 3c5dec10..83beb1bf 100644 --- a/dashboard.md +++ b/dashboard.md @@ -8,7 +8,7 @@ Every task the Mind is holding, on one page: what is in flight, what is parked, | Where | Count | |-------|------:| -| [In flight](#in-flight) (`active/`) | 3 | +| [In flight](#in-flight) (`active/`) | 2 | | [Parked](#parked) (`parked.md`) | 3 | | [Planned](#planned) (`planned.md`) | 6 | | [Backlog](#backlog) (`draft/`) | 152 | @@ -159,14 +159,6 @@ Issued — each has an open GitHub issue and usually a branch. The full record f
-
📋 TEST_MODE bypass crashes on ordered-parameter assertion tiesissue #1519 — issued 2026-08-24 — library-shipped, awaiting-merge - -``` -/start_dev active/test_mode_bypass_ordered_assertion_ties.md -``` - -
- ## Parked Started or scoped, not currently in flight — resume by moving the row back to `active.md`. Full detail in [`parked.md`](parked.md). @@ -1329,7 +1321,6 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | Date | Event | Task | |------|-------|------| | 2026-08-24 | filed | The weekly smoke run's timings land in results-* under no… | -| 2026-08-24 | issued | TEST_MODE bypass crashes on ordered-parameter assertion ties | | 2026-08-24 | issued | Properly time and profile the smoke/release script surface | | 2026-08-23 | filed | pynufft removal: unswept downstream residue (1 hard break + stale… | | 2026-08-23 | filed | Phase 3: stop installing pynufft in Hands/Heart CI and PyAutoCTI… | @@ -1338,12 +1329,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-22 | filed | multi_dataset/jax_likelihood scripts hang to the timeout cap (XLA… | | 2026-08-22 | filed | Untrack the generated FITS test artifacts in autoarray | | 2026-08-22 | filed | The reconstruction noise map describes a different estimator than the… | +| 2026-08-22 | filed | Remove pynufft + legacy TransformerNUFFTPyNUFFT |
… 10 more (40 left) | Date | Event | Task | |------|-------|------| -| 2026-08-22 | filed | Remove pynufft + legacy TransformerNUFFTPyNUFFT | | 2026-08-22 | filed | Point-source JSON datasets record no resolution regime | | 2026-08-22 | filed | Is Intel macOS a supported platform, and what is the numpy-only… | | 2026-08-22 | filed | Defer the eager scipy.sparse import in derivative_util (~0.10 s of… | @@ -1353,12 +1344,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-20 | filed | Numba CPU likelihood phase 1: batched MGE convolution +… | | 2026-08-19 | filed | status.sh --repos sources a file that no longer exists | | 2026-08-19 | filed | jax 0.11 breaks beta/gamma message log_partition under jit… | +| 2026-08-19 | filed | autolens_workspace_test jax_likelihood pins: 4 scripts fail smoke on… |
… 10 more (30 left) | Date | Event | Task | |------|-------|------| -| 2026-08-19 | filed | autolens_workspace_test jax_likelihood pins: 4 scripts fail smoke on… | | 2026-08-19 | filed | autofit_profiling: bootstrap the repo + general PyAutoFit profiling… | | 2026-08-19 | filed | autoreduce 0.9 on PyPI never got the Python 3.12 floor | | 2026-08-19 | issued | @PyAutoFit TransformedMessage.factor_gradient crashes on first… | @@ -1368,12 +1359,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-19 | filed | PyAutoConf rename leftovers in Brain functional surfaces | | 2026-08-19 | filed | Explore: dashboardify the Brain's operational surfaces with pasteable… | | 2026-08-19 | filed | Deduplicate repos_sync.py's check/write pairs | +| 2026-08-19 | filed | Bug in autocti_workspace: the dataset_1d results/database example… |
… 10 more (20 left) | Date | Event | Task | |------|-------|------| -| 2026-08-19 | filed | Bug in autocti_workspace: the dataset_1d results/database example… | | 2026-08-18 | parked | single-source-density-design | | 2026-08-18 | parked | prior-message-collapse-design | | 2026-08-18 | filed | @PyAutoFit TransformedMessage.logpdf/pdf omit the transform… | @@ -1383,12 +1374,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-14 | filed | Three jax_likelihood pins are stale by ~1.24e-4 and fail the smoke… | | 2026-08-09 | found | isothermal-ell-sph-oversampling-at-the-cusp | | 2026-08-08 | parked | pyautoreduce-slacs1430-acs-comparison | +| 2026-08-08 | filed | Regenerate autolens_workspace markdown/ so the MGE pages show… |
… 10 more (10 left) | Date | Event | Task | |------|-------|------| -| 2026-08-08 | filed | Regenerate autolens_workspace markdown/ so the MGE pages show… | | 2026-08-07 | filed | autofit.plot functions accept **kwargs and silently discard them | | 2026-08-07 | filed | Regenerate setup_notebook-drifted notebooks in… | | 2026-08-06 | filed | Triage: Convolver "No blurring_image provided" warning in canonical… | @@ -1398,6 +1389,7 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-04 | filed | dataset/imaging/jwst_lw is untracked because the gitignore was never… | | 2026-08-04 | filed | cosmos_web_ring stores boolean masks as float64, wasting ~3.4 MB of… | | 2026-08-04 | filed | autolens_workspace_developer: broad stale-API rot (56 symbols, no CI) | +| 2026-08-04 | filed | aplt.Output stale-API drift in the remaining workspace repos |
From 3cbc79c0f4c9036adb0193b29aa55cee4c3ff008 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 24 Aug 2026 18:08:09 +0000 Subject: [PATCH 5/9] prompt: file test-mode-bypass-assertion-ties follow-ups (autocti docs note + Phase 5 smoke) --- dashboard.html | 41 +++++----- dashboard.md | 81 ++++++++++++------- ...tmode_assertion_workaround_note_removal.md | 22 +++++ ...se5_smoke_reenable_ordered_trap_scripts.md | 24 ++++++ 4 files changed, 120 insertions(+), 48 deletions(-) create mode 100644 draft/docs/autocti/testmode_assertion_workaround_note_removal.md create mode 100644 draft/test/autocti/phase5_smoke_reenable_ordered_trap_scripts.md diff --git a/dashboard.html b/dashboard.html index 539a0f4d..213a6533 100644 --- a/dashboard.html +++ b/dashboard.html @@ -138,7 +138,7 @@

PyAutoMindDashboard

Intent. Priority. Flow.

Every task the Mind is holding. Tap a task's 📋 and its /start_dev command is on your clipboard — paste it into a Claude Code chat to route Claude straight to that task. Recent is the same work by date — what has been happening rather than what to do next.

-
  • 2In flight
  • 3Parked
  • 6Planned
  • 152Backlog
+
  • 2In flight
  • 3Parked
  • 6Planned
  • 154Backlog

markdown version

Start here

Highest priority (filed as high) — showing 12 of 17

@@ -180,7 +180,7 @@

Planned

latent-nan-guard-honest-run — planned 2026-07-22

Backlog markdown version

-

152 filed prompts, not started — sorted most-pickable first (priority, then size). 25 of them belong to an epic and are listed only under Epics below.

+

154 filed prompts, not started — sorted most-pickable first (priority, then size). 26 of them belong to an epic and are listed only under Epics below.

feature — 29 @@ -292,10 +292,11 @@

Backlog

Refactor Agent witness map lacks PyAutoNerves test suite🧹 maintenancepyautobrainlowsafelow

-docs — 14 +docs — 15

Split lensing regimes: multi_galaxy / group / cluster (epic plan)📖 docsautolenstoo-largesupervisedhigh

Advanced workspace guide: Preloads (PyAutoArray)📖 docsworkspacestoo-largesupervisedhigh

+

Regenerate autolens_workspace markdown/ so the MGE pages show sigma_min📖 docsautolens_workspacesmallsupervisednormal

Propagate the shear_galaxy-at-(0,0) idiom to group/ and cluster/📖 docsworkspacessmallsupervisednormal

@@ -309,14 +310,6 @@

Backlog

Phase 2: Make workspace READMEs assistant-first📖 docsworkspaces

-refactor — 5 - - -

Split Fitness.batch_size into lh_batch_size and latent_batch_size♻️ refactorautofitsmallsupervisednormal

-

Remove the dead EDEN packaging tooling from PyAutoFit♻️ refactorpyautofitmediumsupervisednormal

-

Deduplicate repos_sync.py's check/write pairs♻️ refactorpyautomindmediumsafelow

-
-
test — 5

Re-baseline the MGE imaging JIT profiling regression value🧪 testautolens_workspace_developertoo-largesupervisedhigh

@@ -325,6 +318,14 @@

Backlog

The new workspace smoke-test GitHub Actions (added via feature/smoke-test-ci) surfaced🧪 testworkspacestoo-largesupervisednormal

+refactor — 5 + + +

Split Fitness.batch_size into lh_batch_size and latent_batch_size♻️ refactorautofitsmallsupervisednormal

+

Remove the dead EDEN packaging tooling from PyAutoFit♻️ refactorpyautofitmediumsupervisednormal

+

Deduplicate repos_sync.py's check/write pairs♻️ refactorpyautomindmediumsafelow

+
+
triage — 4

TRIAGE: needs manual review before routing❓ triagemediumsafehigh

@@ -351,6 +352,12 @@

Backlog +2026-08-24 +filed +Delete the TEST_MODE ordered-assertion workaround note from… + + + 2026-08-23 filed pynufft removal: unswept downstream residue (1 hard break + stale… @@ -392,7 +399,7 @@

Backlog The reconstruction noise map describes a different estimator than the… - + 2026-08-22 filed Remove pynufft + legacy TransformerNUFFTPyNUFFT @@ -632,12 +639,6 @@

Backlog autolens_workspace_developer: broad stale-API rot (56 symbols, no CI) - -2026-08-04 -filed -aplt.Output stale-API drift in the remaining workspace repos - -

Epics markdown version

@@ -680,6 +681,10 @@

Epics

Expectation Propagation Scale-Up — Scoping🔬 researchgraphical_eptoo-largesupervisedhigh

slope_hierarchy: methods write-up (NUTS headline, EP cautionary)🔬 researchgraphical_epmediumsupervisednormal

+
+CTI resurrection — Phase 5 — 1 queued prompt(s) — ⚠️ not in epics.md + +