|
24 | 24 | separate /repo_cleanup sweep so a destructive branch delete never rides a code diff. |
25 | 25 | - repos: |
26 | 26 | - PyAutoHands: feature/hands-hygiene-leftovers |
| 27 | + |
| 28 | +## jax-grad-local-vs-ci-assertions |
| 29 | +- issue: https://github.com/PyAutoLabs/autolens_workspace_test/issues/260 (issued 2026-08-22) |
| 30 | +- session: https://claude.ai/code/session_01VEHLT33XpVcRt5YCJGLRMJ (web-github; no local worktree yet) |
| 31 | +- status: library-dev (RECLASSIFIED from workspace-dev — the fix is upstream in PyAutoArray; |
| 32 | + autolens_workspace_test needs no change and no tolerance change) |
| 33 | +- worktree: ~/Code/PyAutoLabs-wt/jax-grad-local-vs-ci-assertions |
| 34 | +- prompt: active/jax_grad_local_assertions_fail_but_pass_in_ci.md |
| 35 | +- classification: workspace (single repo) — routes to /start_workspace |
| 36 | +- strategy: investigate-first. Bug Agent: severity=critical, scope=single-repo, |
| 37 | + type=wrong-result, confidence=LOW. Reproduce and confirm root cause BEFORE patching. |
| 38 | + Brain sizing disagreement: declared medium, derived large. |
| 39 | +- control: imaging/jax_grad/lp.py is the discriminator — the only script known to PASS in |
| 40 | + CI and FAIL locally. Every A/B runs against it first (~41s). |
| 41 | +- ROOT CAUSE FOUND 2026-08-22 (no laptop needed; findings on issue #260): |
| 42 | + PyAutoArray `util/dataset_util.py:72` `should_simulate()` is existence-only and asymmetric — |
| 43 | + it force-regenerates under PYAUTO_SMALL_DATASETS=1 but never under full_datasets. `dataset/**` |
| 44 | + is gitignored, so CI always simulates fresh and CANNOT hit this; locally the dir persists and |
| 45 | + is never refreshed. Any prior smoke run (SMALL_DATASETS=1 is the default for every OTHER |
| 46 | + script) rewrites the FITS at 15x15, and the next jax_grad run silently loads them under |
| 47 | + full-resolution settings. All three failures reproduced exactly from a clean checkout |
| 48 | + (pixelization eager/jit matches the report to 11 s.f.; regularization's tolerance vector |
| 49 | + matches exactly). Full chain proven: fresh=PASS -> one SMALL run -> stale=FAIL -> rm -rf =PASS. |
| 50 | +- numpy FALSIFIED: lp.py byte-identical across numpy 2.2.6 / 2.4.6 / 2.5.2 (and across |
| 51 | + 1-core vs 4-core). The likelihood runs through JAX/XLA; numpy only does the FD bookkeeping. |
| 52 | +- NO tolerance change is warranted — all three asserts did their job on a genuinely invalid |
| 53 | + dataset. assert_eager_jit_consistent's rtol=1e-10 is vindicated, not under-specified. |
| 54 | +- FIX SHIPPED to a branch 2026-08-22: PyAutoArray `claude/jax-grad-assertions-ci-hontn3` @ 5ef6eea. |
| 55 | + `should_simulate` now also regenerates on the small->full transition, inferring the regime from |
| 56 | + data.fits's shape (the cap emits EXACTLY 16x16). Design went through an adversarial review that |
| 57 | + corrected three things: no pixel-scale check (not in the FITS header), `==` not `<=` (the cap |
| 58 | + cannot emit 12x12), and data.fits BY NAME not first-FITS-in-dir (PSFs are legitimately 11x11 and |
| 59 | + dataset/cluster/test/psf.fits is 5760 B — a glob would have deleted real data every run). |
| 60 | + Verified: control-tested, 1168 tests pass, 12 new tests over all four regime transitions, and |
| 61 | + end-to-end — poison then full-datasets run now PASSES where it raised. Steady state 0.64 ms. |
| 62 | + NO PR opened (not requested); branch is pushed and ready. |
| 63 | +- SCOPE: imaging manifestation only. Point-source/weak-lensing (JSON, no FITS) and interferometer |
| 64 | + (shape-invariant under the cap, so it fails SILENTLY) remain exposed; stated in the docstring. |
| 65 | +- follow-ups filed (not absorbed): |
| 66 | + - PyAutoNerves#153 — stamp the regime at the single FITS writer funnel (fitsable.py:89); the only |
| 67 | + discriminant that can catch the silent interferometer case. Kept out of the fix deliberately: it |
| 68 | + changes a header card on every FITS the stack writes (round-trip tests, file-hash pins). |
| 69 | + - PyAutoArray#470 — separate live defect: the small-datasets branch rmtree's |
| 70 | + dataset/point_source/simple, which is COMMITTED and allowlisted in .gitignore:13, replacing it |
| 71 | + with capped-solver garbage. Recoverable via git checkout but violates the allowlist invariant. |
| 72 | +- follow-up (separate defect, not this bug): autolens_workspace_test |
| 73 | + `.github/scripts/smoke_install.sh:9` `pip install "jax<0.7" "jaxlib<0.7"` downgrades jax to |
| 74 | + 0.6.2 and conflicts with autonerves' jax>=0.7,<0.11; the run only lands on the intended |
| 75 | + 0.10.2 because the next line's [optional] extras pull it back up. CI is right by accident. |
| 76 | +- out-of-bounds: moving lp.py's evaluation point, adding skip_indices, or widening a |
| 77 | + tolerance without a measured basis. All three mask the trap instead of removing it. |
| 78 | +- repos: |
| 79 | + - PyAutoArray: claude/jax-grad-assertions-ci-hontn3 |
0 commit comments