diff --git a/autonomy_log.md b/autonomy_log.md index 387a94f3..9ce39a30 100644 --- a/autonomy_log.md +++ b/autonomy_log.md @@ -242,3 +242,4 @@ Outcome ∈ `merged-unchanged` / `amended` / `rejected` / `parked` / | 2026-08-01 | nautilus 1-core serial-pool — corrective for Heart RED "release validation FAILED (stage integrate)" (PyAutoFit#1442 / PR#1443) | corrective-PR exception; authorization = human session instruction quoted verbatim on #1442, given at launch ("fine if any blockers need sorting") rather than after the reason string was surfaced — deviation noted | regression test red-on-main/green-patched; test_autofit/non_linear 411p; py-spy deadlock stacks; control-vs-patched release-env run | corrective (PR-open; merge under the same session authorization; validation = Stage 2/3 dispatch + nightly on fresh wheels) | | 2026-08-24 | refactor-witness-map-audit (PyAutoBrain#269 / PR#271) | safe (header safe, maintenance cap safe) | tests PASS 468 (no downstream — organ repo, nothing imports Brain) / smoke n/a (organism repo, no script surface) / review CLEAN, lifted claim basis-cited by blobless tree reads of all 10 repos / heart NOT EVALUATED — pyauto-heart unreachable in this web-github session, no verdict obtained | merged-unchanged (PR#271 merge cf969f3 after #268, human-authorized via /prm; CI Brain Tests green on 2846b65; retargeted base main before merge — GitHub only auto-retargets a stacked PR on base-branch DELETION, not on merge) | | 2026-08-24 | pyautoconf-rename-leftovers (PyAutoBrain#267 / PR#268) | supervised (header supervised, bug cap supervised) | tests PASS 467 (no downstream — organ repo) / smoke n/a (organism repo, no script surface) / review CLEAN, 3 claim dispositions incl. one demoted to idle (a test count cited from a Mind record, not re-run) / heart NOT EVALUATED — pyauto-heart unreachable in this web-github session | parked at ship sign-off per the supervised contract, then PR-open on human sign-off same session; merged-unchanged (PR#268 merge 2e7ea30, human-authorized via /prm; CI Brain Tests green on 2088bad) | +| 2026-08-24 | tracer-fits-existence-guard (autolens_workspace#503) | supervised (header safe, bug cap supervised) | tests n/a (workspace repo, no test dir) / smoke reproduced-then-fixed: imaging/modeling.py + multi_galaxy/modeling.py exit 1 with the reported FileNotFoundError under profile_smoke + PYAUTO_TEST_MODE=1 before, exit 0 after; TEST_MODE=2 default green; check_sizes.sh OK / review not run — parked before ship, so no ReviewSurface / heart NOT EVALUATED — pyauto-heart unreachable in this web-github session | parked at ship sign-off per the supervised contract, then PR-open on human sign-off same session (/prm); merged-unchanged (PR#504 merge b19750c, human-authorized via /prm; CI green on 0c14d00 — 3 runs / 7 jobs) | diff --git a/complete/2026/08/tracer-fits-existence-guard.md b/complete/2026/08/tracer-fits-existence-guard.md new file mode 100644 index 00000000..8685e712 --- /dev/null +++ b/complete/2026/08/tracer-fits-existence-guard.md @@ -0,0 +1,85 @@ +- issue: https://github.com/PyAutoLabs/autolens_workspace/issues/503 +- completed: 2026-08-24 +- workspace-pr: autolens_workspace#504 (merged b19750cad64d2b5a0d809c132086371dccde27bc -> main) +- what shipped: the "Loading From Output Folder" teaching section in `scripts/imaging/modeling.py` guarded on `files/tracer.json` and then, inside that branch, read `image/tracer.fits`. Split into two independent guards, one per file, so each read is gated on the file it actually reads. Same fix applied to `scripts/multi_galaxy/modeling.py`; both notebook mirrors regenerated. +- the actual bug: under TEST_MODE the reduced search writes `files/tracer.json` but NOT the visualisation output `image/tracer.fits`, so the guard passed and the read raised FileNotFoundError. Long-standing (present since cfd5334c, 2026-04-14), not a regression. Never blocked a release — the `release` job's `needs:` excludes `run_smoke_tests` — but left the smoke gate permanently red, so a genuine workspace regression would have looked identical to this known failure. +- guard shape is now canonical: guard-each-read-on-its-own-file, matching what `scripts/guides/results/start_here.py` already did. `draft/maintenance/workspaces/read_through_issues.md` asks for this section to be ADDED to the group/cluster/interferometer examples — this is the shape to propagate, or the same bug ships to those scripts. +- scope decision: `scripts/multi_galaxy/modeling.py` was NOT in the reported scope but carried the byte-identical block and is also on `smoke_tests.txt`. Fixing only `imaging/` would have left the smoke gate red anyway, so both went in one PR. +- trap for reproducing: the smoke profile's DEFAULT `PYAUTO_TEST_MODE=2` bypasses the sampler and writes no `tracer.json` at all, so the guard is never entered and the bug does NOT surface. `PYAUTO_TEST_MODE=1` (reduced search) is the mode that writes `tracer.json` but not `tracer.fits`. Anyone re-checking this under the plain smoke default will wrongly conclude it is already fixed. +- validation: reproduced-then-fixed under profile_smoke.yaml + PYAUTO_TEST_MODE=1 — both scripts exit 1 with the reported FileNotFoundError before, exit 0 after. CI green on the merged head 0c14d00 across 3 workflow runs / 7 jobs (Smoke Tests changes+3.12+3.13, Navigator Check x3, Script Size Guard); mergeable_state clean at merge. +- heart: NOT EVALUATED — `pyauto-heart` is unreachable from a web-github session, so leg 4 of the ship gate has no verdict. Stated on the PR and issue rather than assumed green. +- environment note: run entirely in a cloud web-github session — no worktree, no `activate.sh`. autolens needs Python >= 3.12 (the PyPI 2026.7.29.1.post1 sdist raises a RuntimeError below that as a deliberate floor guard) while the session default was 3.11; a uv venv on /usr/bin/python3.12 was needed. `.github/scripts/run_smoke.py` could not be used (needs a sibling PyAutoHands checkout on PYTHONPATH), so profile_smoke.yaml's env was applied by hand. Notebook regeneration DID work by cloning PyAutoHands read-only and running `autohands/generate.py autolens` with the venv's bin on PATH — it rewrote only the two changed notebooks, no catalogue or unrelated drift. +- gotcha for future cloud sessions: an `add_repo` clone arrives with `remote.origin.fetch = +refs/heads/main:refs/remotes/origin/main`, so `git push -u` sets the branch remote but can never create the `origin/` tracking ref — the stop-hook git check then reports pushed commits as unpushed. Fix is `git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'` then fetch. +- follow-up not done (deliberately out of scope): the hard-coded `pixel_scales=0.1` / `0.05` in these blocks are the same literal pattern autolens_workspace#502 replaced with `dataset.pixel_scales` elsewhere. +- autonomy: `--auto` launch, effective level supervised (prompt header safe, bug work-type cap supervised). Parked at ship sign-off per the contract; human typed /prm, which authorized PR-open, merge and the full close-out. + +## Original prompt + +# Bug: fix the tracer.fits existence guard in autolens_workspace imaging modeling.py + +Type: bug +Target: workspaces +Repos: +- autolens_workspace +Difficulty: small +Autonomy: safe +Priority: normal +Status: formalised +Filed: 2026-08-22 (backfilled from git) +Issued: 2026-08-24 + +Bug: fix the tracer.fits existence guard in autolens_workspace imaging modeling.py. The script crashes with FileNotFoundError because the guard checks the wrong file. It tests whether files/tracer.json exists, then reads image/tracer.fits, which TEST_MODE never writes. This fails the release smoke gate. + + +## Evidence (2026-08-23, gathered during the 2026.8.23.1 release) + +`autolens_workspace/scripts/imaging/modeling.py:641`. The guard and the read +name **different files**: + +```python +if (result_path / "files" / "tracer.json").exists(): + tracer = from_json(file_path=result_path / "files" / "tracer.json") + + tracer_fits = al.Array2D.from_fits( + file_path=result_path / "image" / "tracer.fits", hdu=0, pixel_scales=0.1 + ) +``` + +Under TEST_MODE the search is reduced, so `files/tracer.json` IS written but the +visualisation output `image/tracer.fits` is not. The guard passes and the read +raises: + +``` +FileNotFoundError: .../output/test_mode/imaging/simple/modeling//image/tracer.fits +FAIL: imaging/modeling.py +``` + +- Present since `cfd5334c` (2026-04-14); the script is untouched since 2026-08-04, + so this is long-standing, not a regression from the regime-stamp release. +- **TEST_MODE-specific.** The same script passes at release fidelity — Heart's + Stage 3 integrate ran `run_scripts (3.12, autolens, imaging)` green + (672p/0f), because a full-fidelity fit does write `tracer.fits`. +- Reproduced in PyAutoHands `release.yml` run 32542888112, job + `run_smoke_tests (3.12, autolens_workspace)`. + +## Why it matters (and why it is NOT urgent) + +It does not block releases: the `release` job's `needs:` is +`[resolve_mode, release_test_pypi, version_number]` and excludes +`run_smoke_tests`, so the publish succeeds regardless — run 32542888112 is marked +`failure` yet published 2026.8.22.1. The real cost is that it leaves the smoke +gate permanently red, so a genuine workspace regression would look identical to +this known failure and be ignored. + +## Traps + +- Do **not** "fix" this by deleting the result-loading block. It is a documented + teaching section, and `draft/maintenance/workspaces/read_through_issues.md` + separately asks for the equivalent section to be **added** to the other + `modeling.py` examples (group, cluster, interferometer). Coordinate with that + prompt — whatever guard shape is chosen here should be the one propagated + there, or the same bug ships to four more scripts. +- Guarding on `image/tracer.fits` alone would silently skip the `tracer.json` + load in test mode. Prefer guarding each read on the file it actually reads. + + diff --git a/complete/index.md b/complete/index.md index 2e3fad43..9cb396f4 100644 --- a/complete/index.md +++ b/complete/index.md @@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema: only then grep a dated bucket. Curators: edit the band between the CURATED markers; everything below GENERATED is rebuilt. -1123 records across 7 buckets. +1124 records across 7 buckets. ## Highlights @@ -233,6 +233,7 @@ _(curate hard-won records here — survives regeneration.)_ - [test-mode-samples-info-hook-contract](2026/08/test-mode-samples-info-hook-contract.md) — `NonLinearSearch._test_mode_samples_info()`'s docstring told subclasses - [test-performance-board](2026/08/test-performance-board.md) - [testmode-assertion-note-removal](2026/08/testmode-assertion-note-removal.md) — Follow-up to test-mode-bypass-assertion-ties (PyAutoFit#1520, merged +- [tracer-fits-existence-guard](2026/08/tracer-fits-existence-guard.md) - [transformed-message-semantics-doc](2026/08/transformed-message-semantics-doc.md) - [undo-community-file-declutter](2026/08/undo-community-file-declutter.md) — Undid the 2026-08-19 community-file declutter (#248 Mind, #32 Memory): - [uniform-prior-bounds-numpy-path](2026/08/uniform-prior-bounds-numpy-path.md) — auto-closed by the merge's `Closes` line diff --git a/dashboard.html b/dashboard.html index 204763e8..1d7eda95 100644 --- a/dashboard.html +++ b/dashboard.html @@ -185,7 +185,7 @@

PyAutoMindDashboard

Intent. Priority. Flow.

Every task the Mind is holding. Tap a task's 📋 and its /start_dev command is on your clipboard — paste it into a Claude Code chat to route Claude straight to that task. Recent is the same work by date — what has been happening rather than what to do next.

- +

Last updated 2026-08-24. This page is generated from active/, draft/ and the registry files, so it is only as current as they are. dashboard_refresh.yml re-renders it on every push to main — that heals a stale page, but not a stale prompt: a task that shipped without its prompt advancing to complete/ keeps rendering here as pickable backlog. Reconciling those is the refresh below.

Fix JIT quick-update visualization output regressions🐛 bughealth_fixestoo-largesupervisedhigh

Quick wins (small enough, and safe enough to run unattended)

Defer the eager scipy.sparse import in derivative_util (~0.10 s of import)🧹 maintenancelibrariessmallsafenormal

-

Bug: fix the tracer.fits existence guard in autolens_workspace imaging modeling.py🐛 bugworkspacessmallsafenormal

The weekly smoke run's timings land in results-* under no discoverable name🧹 maintenancepyautoheartsmallsafelow

In flight markdown version

Issued — each has an open GitHub issue and usually a branch.

@@ -253,7 +252,7 @@

Planned

latent-nan-guard-honest-run — planned 2026-07-22

Backlog markdown version

-

140 filed prompts, not started — sorted most-pickable first (priority, then size). 25 of them belong to an epic and are listed only under Epics below.

+

139 filed prompts, not started — sorted most-pickable first (priority, then size). 25 of them belong to an epic and are listed only under Epics below.

feature — 27

Numba CPU likelihood phase 1: batched MGE convolution + operated-matrix caching✨ featureautoarraymediumsupervisedhigh

@@ -285,7 +284,7 @@

Backlog

Teach repos_sync --write to stamp organ config surfaces✨ featurepyautomindhardsupervisedlow

-bug — 24 +bug — 23

Fix release JAX runtime compatibility and likelihood parity🐛 bughealth_fixestoo-largesupervisedhigh

Fix JIT quick-update visualization output regressions🐛 bughealth_fixestoo-largesupervisedhigh

Fix release result/sample parameter-path regressions🐛 bughealth_fixestoo-largesupervisedhigh

@@ -295,7 +294,6 @@

Backlog

generate.py deletes notebooks/ before rejecting an unknown project🐛 bugpyautohandssmallsupervisednormal

autoreduce 0.9 on PyPI never got the Python 3.12 floor🐛 bugpyautoreducesmallsupervisednormal

aplt.Output stale-API drift in the remaining workspace repos🐛 bugworkspacessmallsupervisednormal

-

Bug: fix the tracer.fits existence guard in autolens_workspace imaging modeling.py🐛 bugworkspacessmallsafenormal

Three jax_likelihood pins are stale by ~1.24e-4 and fail the smoke gate…🐛 bugworkspacessmallsupervisednormal

PROBE: is Adapt's 4th-power coefficient dependence (double square) intentional?🐛 bugautoarraymediumsupervisednormal

JIT cache not hit in modeling_visualization delaunay/rectangular scripts🐛 bugautolensmediumsupervisednormal

@@ -502,12 +500,6 @@

Backlog -2026-08-22 -filed -Bug: fix the tracer.fits existence guard in autolens_workspace… - - - 2026-08-21 filed Rectangular mesh split: Bilinear (fast CPU default) vs RTU… @@ -699,6 +691,12 @@

Backlog aplt.Output stale-API drift in the remaining workspace repos + +2026-08-04 +filed +Nightly release has been blocked 8 nights running — triage the streak + +

Epics markdown version

diff --git a/dashboard.md b/dashboard.md index ca7f71f3..7b4a4b79 100644 --- a/dashboard.md +++ b/dashboard.md @@ -45,7 +45,7 @@ anything you could not verify. | [In flight](#in-flight) (`active/`) | 2 | | [Parked](#parked) (`parked.md`) | 3 | | [Planned](#planned) (`planned.md`) | 5 | -| [Backlog](#backlog) (`draft/`) | 140 | +| [Backlog](#backlog) (`draft/`) | 139 | ## Start here @@ -157,14 +157,6 @@ anything you could not verify.
-
📋 Bug: fix the tracer.fits existence guard in autolens_workspace imaging modeling.py — workspaces · small · safe · normal - -``` -/start_dev draft/bug/workspaces/bug_fix_the_tracer_fits_existence_guard.md -``` - -
-
📋 The weekly smoke run's timings land in results-* under no discoverable name — pyautoheart · small · safe · low ``` @@ -277,7 +269,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned. ## Backlog -**140** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **25** of them belong to an epic and are listed only under [Epics](#epics) below. +**139** filed prompts, not started. Each section is sorted most-pickable first (priority, then size). **25** of them belong to an epic and are listed only under [Epics](#epics) below.
feature — 27 @@ -501,7 +493,7 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-bug — 24 +bug — 23
📋 Fix release JAX runtime compatibility and likelihood parity — health_fixes · too-large · supervised · high @@ -575,14 +567,6 @@ Scoped but not started; some are not yet prompt files. Full detail in [`planned.
-
📋 Bug: fix the tracer.fits existence guard in autolens_workspace imaging modeling.py — workspaces · small · safe · normal - -``` -/start_dev draft/bug/workspaces/bug_fix_the_tracer_fits_existence_guard.md -``` - -
-
📋 Three jax_likelihood pins are stale by ~1.24e-4 and fail the smoke gate… — workspaces · small · supervised · normal ``` @@ -1272,15 +1256,14 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-22 | filed | Point-source JSON datasets record no resolution regime | | 2026-08-22 | filed | Is Intel macOS a supported platform, and what is the numpy-only… | | 2026-08-22 | filed | Defer the eager scipy.sparse import in derivative_util (~0.10 s of… | -| 2026-08-22 | filed | Bug: fix the tracer.fits existence guard in autolens_workspace… | | 2026-08-21 | filed | Rectangular mesh split: Bilinear (fast CPU default) vs RTU… | | 2026-08-20 | filed | Numba CPU likelihood phase 2: kernel-CDF numba fast path (the 49-88%… | +| 2026-08-20 | filed | Numba CPU likelihood phase 1: batched MGE convolution +… |
… 10 more (30 left) | Date | Event | Task | |------|-------|------| -| 2026-08-20 | filed | Numba CPU likelihood phase 1: batched MGE convolution +… | | 2026-08-19 | filed | status.sh --repos sources a file that no longer exists | | 2026-08-19 | filed | jax 0.11 breaks beta/gamma message log_partition under jit… | | 2026-08-19 | filed | autolens_workspace_test jax_likelihood pins: 4 scripts fail smoke on… | @@ -1290,12 +1273,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-19 | filed | Release board: local run_logs enrichment | | 2026-08-19 | filed | RTD organism docs currency: Nerves page, organ-count drift, hands.md… | | 2026-08-19 | filed | Deduplicate repos_sync.py's check/write pairs | +| 2026-08-19 | filed | Bug in autocti_workspace: the dataset_1d results/database example… |
… 10 more (20 left) | Date | Event | Task | |------|-------|------| -| 2026-08-19 | filed | Bug in autocti_workspace: the dataset_1d results/database example… | | 2026-08-18 | parked | single-source-density-design | | 2026-08-18 | parked | prior-message-collapse-design | | 2026-08-18 | filed | @PyAutoFit TransformedMessage.logpdf/pdf omit the transform… | @@ -1305,12 +1288,12 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-14 | filed | Three jax_likelihood pins are stale by ~1.24e-4 and fail the smoke… | | 2026-08-09 | found | isothermal-ell-sph-oversampling-at-the-cusp | | 2026-08-08 | parked | pyautoreduce-slacs1430-acs-comparison | +| 2026-08-08 | filed | Regenerate autolens_workspace markdown/ so the MGE pages show… |
… 10 more (10 left) | Date | Event | Task | |------|-------|------| -| 2026-08-08 | filed | Regenerate autolens_workspace markdown/ so the MGE pages show… | | 2026-08-07 | filed | Regenerate setup_notebook-drifted notebooks in… | | 2026-08-06 | filed | Triage: Convolver "No blurring_image provided" warning in canonical… | | 2026-08-06 | filed | Rewrite PyAutoCTI docs/api — 55 of 89 autosummary entries are dead | @@ -1320,6 +1303,7 @@ The 50 newest things to happen to the work in hand, newest first — issued, par | 2026-08-04 | filed | cosmos_web_ring stores boolean masks as float64, wasting ~3.4 MB of… | | 2026-08-04 | filed | autolens_workspace_developer: broad stale-API rot (56 symbols, no CI) | | 2026-08-04 | filed | aplt.Output stale-API drift in the remaining workspace repos | +| 2026-08-04 | filed | Nightly release has been blocked 8 nights running — triage the streak |
diff --git a/draft/bug/workspaces/bug_fix_the_tracer_fits_existence_guard.md b/draft/bug/workspaces/bug_fix_the_tracer_fits_existence_guard.md deleted file mode 100644 index e22e112b..00000000 --- a/draft/bug/workspaces/bug_fix_the_tracer_fits_existence_guard.md +++ /dev/null @@ -1,67 +0,0 @@ -# Bug: fix the tracer.fits existence guard in autolens_workspace imaging modeling.py - -Type: bug -Target: workspaces -Repos: -- autolens_workspace -Difficulty: small -Autonomy: safe -Priority: normal -Status: formalised -Filed: 2026-08-22 (backfilled from git) - -Bug: fix the tracer.fits existence guard in autolens_workspace imaging modeling.py. The script crashes with FileNotFoundError because the guard checks the wrong file. It tests whether files/tracer.json exists, then reads image/tracer.fits, which TEST_MODE never writes. This fails the release smoke gate. - - -## Evidence (2026-08-23, gathered during the 2026.8.23.1 release) - -`autolens_workspace/scripts/imaging/modeling.py:641`. The guard and the read -name **different files**: - -```python -if (result_path / "files" / "tracer.json").exists(): - tracer = from_json(file_path=result_path / "files" / "tracer.json") - - tracer_fits = al.Array2D.from_fits( - file_path=result_path / "image" / "tracer.fits", hdu=0, pixel_scales=0.1 - ) -``` - -Under TEST_MODE the search is reduced, so `files/tracer.json` IS written but the -visualisation output `image/tracer.fits` is not. The guard passes and the read -raises: - -``` -FileNotFoundError: .../output/test_mode/imaging/simple/modeling//image/tracer.fits -FAIL: imaging/modeling.py -``` - -- Present since `cfd5334c` (2026-04-14); the script is untouched since 2026-08-04, - so this is long-standing, not a regression from the regime-stamp release. -- **TEST_MODE-specific.** The same script passes at release fidelity — Heart's - Stage 3 integrate ran `run_scripts (3.12, autolens, imaging)` green - (672p/0f), because a full-fidelity fit does write `tracer.fits`. -- Reproduced in PyAutoHands `release.yml` run 32542888112, job - `run_smoke_tests (3.12, autolens_workspace)`. - -## Why it matters (and why it is NOT urgent) - -It does not block releases: the `release` job's `needs:` is -`[resolve_mode, release_test_pypi, version_number]` and excludes -`run_smoke_tests`, so the publish succeeds regardless — run 32542888112 is marked -`failure` yet published 2026.8.22.1. The real cost is that it leaves the smoke -gate permanently red, so a genuine workspace regression would look identical to -this known failure and be ignored. - -## Traps - -- Do **not** "fix" this by deleting the result-loading block. It is a documented - teaching section, and `draft/maintenance/workspaces/read_through_issues.md` - separately asks for the equivalent section to be **added** to the other - `modeling.py` examples (group, cluster, interferometer). Coordinate with that - prompt — whatever guard shape is chosen here should be the one propagated - there, or the same bug ships to four more scripts. -- Guarding on `image/tracer.fits` alone would silently skip the `tracer.json` - load in test mode. Prefer guarding each read on the file it actually reads. - -