Skip to content

Commit fd3b4fd

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/pyautofit-cli-noise-fixes-vqs7mg
# Conflicts: # complete/index.md # dashboard.md
2 parents 5e79734 + f649d88 commit fd3b4fd

10 files changed

Lines changed: 547 additions & 147 deletions
Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# cli-noise-autonerves-batch
2+
3+
**Completed:** 2026-08-18 · **Type:** maintenance · **Target:** PyAutoNerves
4+
**PRs:** PyAutoNerves#149 (fixes), PyAutoMind#232 (implementation notes) — both
5+
merged 2026-08-18. No GitHub issue (small `Autonomy: safe` batch, driven
6+
straight from the draft prompt in a remote session).
7+
8+
## What shipped
9+
10+
The three autonerves-rooted CLI-noise sources from the 2026-08-06
11+
`/cli_noise_clean` audit, all fixed in one PyAutoNerves PR with regression
12+
tests:
13+
14+
1. **fits leak**`fitsable.ndarray_via_fits_from` called `fits.open` without
15+
closing, emitting `ResourceWarning: unclosed file` in every downstream repo
16+
that loads FITS. Now `with fits.open(...)`. The same fix was applied to
17+
`header_obj_from`, which had the identical unclosed-handle pattern a few
18+
lines below the one the audit named — an astropy `Header` stays valid after
19+
the file closes, so the `with` block is safe there too.
20+
2. **pytest collection**`test_test_mode.py` imported the real API functions
21+
`test_mode_level`/`test_mode_samples` by bare name, so pytest collected them
22+
as tests (`PytestReturnNotNoneWarning`, an ERROR in a future pytest). The
23+
unused `test_mode_level` import was dropped; `test_mode_samples` is aliased
24+
to `_test_mode_samples`, with a comment so nobody "cleans up" the alias.
25+
3. **`check_version` false positive** — fix option (b) from the prompt:
26+
`check_version` now returns silently when its root (defaulting to cwd) is a
27+
package source checkout (`setup.py` or `pyproject.toml` at its top level)
28+
and no version floor is recorded. A recorded floor is still enforced even in
29+
a source checkout, and a genuine workspace missing its version keys still
30+
warns. Chosen over option (a) (per-library conftest env vars) because it is
31+
self-contained in autonerves — no changes needed in the five library repos.
32+
33+
Full `test_autonerves` suite green (157/157) under `pytest -W all`; verified
34+
`check_version()` is silent with cwd at a library repo root and that
35+
`test_mode_*` no longer appear in `pytest --collect-only`.
36+
37+
## Surface decision (the prompt's item-3 open question)
38+
39+
The prompt flagged that PyAutoArray/PyAutoNerves don't call `check_version` at
40+
all, unlike autofit/autogalaxy/autolens. Decided: **the asymmetry is intended,
41+
not drift.** `check_version` is the surface of the *workspace-facing* libraries
42+
only — users run scripts from workspace clones that import those three.
43+
autoarray and autonerves are infrastructure layers never driven from a
44+
workspace cwd directly. No library `__init__` was changed.
45+
46+
## Traps / findings
47+
48+
- The audit named one leak site (`fitsable.py:210`); the sibling
49+
`header_obj_from` had the same leak and would have kept a residual
50+
ResourceWarning trickle if only the named line were fixed. When fixing a
51+
pattern-shaped noise source, grep the module for the pattern, not the line.
52+
- The downstream ResourceWarning surfaces the audit lists
53+
(`autofit/database/aggregator/scrape.py`, `autoarray` visibilities /
54+
interferometer dataset, Galaxy/Lens runs) all route through these two
55+
helpers, so no downstream-repo changes are needed — re-run the audit after
56+
the next release picks up autonerves to confirm the stack-wide clearance.
57+
58+
## Original prompt
59+
60+
# Silence the three autonerves-rooted CLI-noise sources (fits leak, pytest collection, check_version false positive)
61+
62+
Type: maintenance
63+
Target: PyAutoNerves
64+
Repos:
65+
- PyAutoNerves
66+
Difficulty: small
67+
Autonomy: safe
68+
Priority: normal
69+
Status: implemented — fixes pushed to PyAutoNerves branch
70+
`claude/autonerves-cli-noise-h1w8sq` (2026-08-18), awaiting PR/merge
71+
72+
Filed 2026-08-06 from a full `/cli_noise_clean` audit (pytest `-W all` across
73+
all five libraries + workspace script runs). Three root causes live in
74+
autonerves; the first pollutes the entire downstream stack.
75+
76+
1. **Unclosed `fits.open` in `autonerves/fitsable.py:210`**`fits.open` is
77+
called without `with`/close, emitting `ResourceWarning: unclosed file` in
78+
every repo that loads FITS via `ndarray_via_fits_from` (surfaces at
79+
`autofit/database/aggregator/scrape.py:187`,
80+
`autoarray/structures/visibilities.py:179` ×10,
81+
`autoarray/dataset/interferometer/dataset.py:198` ×4, and throughout
82+
Galaxy/Lens runs). Fix: `with fits.open(...) as hdu_list: return
83+
ndarray_via_hdu_from(hdu_list[hdu])`. One upstream fix clears the majority
84+
of ResourceWarning noise stack-wide.
85+
2. **`test_mode_level`/`test_mode_samples` collected as pytest tests**
86+
`autonerves/test_mode.py:5,24` are real API functions, but
87+
`test_autonerves/test_test_mode.py:11-16` imports them by bare name, so
88+
pytest collects them (`PytestReturnNotNoneWarning`, becomes an ERROR in a
89+
future pytest). Fix: alias the imports (`... as _test_mode_level`) or import
90+
the module and call qualified.
91+
3. **`check_version` UserWarning on every library import from a source repo**
92+
`autonerves/workspace.py:206` (default `workspace_root=Path.cwd()`) is
93+
called unconditionally by `autofit/autogalaxy/autolens.__init__`, and fires
94+
"Cannot verify the workspace ... is compatible" whenever cwd lacks
95+
`config/general.yaml` — always true in the libraries' own repos, so every
96+
pytest run/collection emits it. Fix options: (a) each library's
97+
`test_<pkg>/conftest.py` sets `PYAUTO_SKIP_WORKSPACE_VERSION_CHECK=1`, or
98+
(b) `check_version` detects it is running from inside a library source tree
99+
and skips. Note PyAutoArray/PyAutoNerves don't call `check_version` at all —
100+
inconsistent with the other three; decide the intended surface while here.
101+
102+
## Implementation notes (2026-08-18, branch `claude/autonerves-cli-noise-h1w8sq`)
103+
104+
- **1 (fits leak)**: fixed with `with fits.open(...)` in
105+
`ndarray_via_fits_from` **and** `header_obj_from`, which had the identical
106+
unclosed-handle pattern. Regression test asserts no `ResourceWarning`.
107+
- **2 (pytest collection)**: dropped the unused bare-name `test_mode_level`
108+
import; aliased `test_mode_samples as _test_mode_samples` with a comment
109+
explaining why, so collection skips it.
110+
- **3 (check_version false positive)**: chose fix (b) — `check_version` now
111+
skips silently when the root is a package source checkout (`setup.py` or
112+
`pyproject.toml` at its top level) and no version floor is recorded. A
113+
recorded floor is still enforced even in a source checkout, and a genuine
114+
workspace missing its version keys still warns. Self-contained in
115+
autonerves; no per-library conftest changes needed.
116+
- **Surface decision** (the item-3 note): `check_version` is intentionally the
117+
surface of the *workspace-facing* libraries only (autofit / autogalaxy /
118+
autolens import it at package init because users run their scripts from
119+
workspace clones). autoarray and autonerves are infrastructure layers never
120+
driven from a workspace cwd directly, so they correctly do not call it — the
121+
asymmetry is intended, not drift. No change made to any library `__init__`.
Lines changed: 224 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,224 @@
1+
The efficacy review that `docs/agent_failure_modes.md` §9 committed to when
2+
mitigation 6 shipped (PyAutoBrain#140, live 2026-07-17): after the ship series,
3+
has the falsified-by checkpoint stage gone rote? **Verdict: not proven rote —
4+
proven unobservable, which is its own finding. Keep the stage, vocabulary
5+
unchanged; make the reviewer's engagement with each lifted claim part of the
6+
recorded verdict so a rote pass becomes ledger-visible.**
7+
8+
- completed: 2026-08-18
9+
- origin: spun out of PyAutoBrain#130 at its 2026-08-15 close (the one §9 item
10+
still open); executed as a dashboard-work research session on branch
11+
`claude/automind-falsified-by-checkpoint-cmsqsi` (PyAutoBrain + PyAutoMind)
12+
- deliverables: Outcome block under mitigation 6 + §9 close-out in
13+
PyAutoBrain `docs/agent_failure_modes.md`; follow-up prompt
14+
`draft/feature/pyautobrain/review_claim_dispositions.md`; this record
15+
16+
## Instrument validation first (the method note's D1 guard)
17+
18+
Before trusting any firing-rate number, the claim-lifter was re-exercised on
19+
2026-08-18: a probe text with three known load-bearing claims ("proven
20+
byte-identical", "no-op", "does not affect") lifts 3/3 through the live
21+
`load_bearing_claims()`, and the 5 pinning tests in
22+
`PyAutoBrain/tests/test_review_claims.py` pass (8/8 with the in-place-resolve
23+
tests). A low firing rate below is therefore a fact about the inputs, not a
24+
dead instrument.
25+
26+
## The evidence base
27+
28+
Two sources, honestly scoped:
29+
30+
1. **The autonomy log ship series, 2026-07-17 → 2026-08-01** — every `--auto`
31+
ship gate since go-live: 21 distinct tasks ran a review leg (jax-compile-time-research,
32+
inject-keck, inject-alma-simobserve, cold-compile-reduction,
33+
autotune-off-default, pix-nonfinite-localisation,
34+
interpolator-aggregator-test-mode, multistart-gradient-auto-convergence
35+
ph1+ph2, multistart-cadence-int-cast, delaunay-nan-callback,
36+
multistart-cadence-followups, python-312-floor 1A/1B/1C/1D/1E/4A/4B/4C/4D),
37+
plus one August cloud-session faculty run recorded in
38+
[[autohands-firewall-allowlist]]**22 review-leg gates**, past the ~10-ship
39+
trial window. August ships after 08-01 were largely interactive/cloud
40+
sessions the autonomy log does not row; they are not counted either way.
41+
2. **Retro-measurement of the claim matcher over real shipped history** — the
42+
ReviewSurface is ephemeral (nothing persists what it lifted per ship), so
43+
firing rate was re-derived by running the live matcher over the squash-merge
44+
messages on `origin/main` since 2026-07-17 in the two repos available to the
45+
session. This is a proxy for the branch-message surface (squash messages
46+
carry the PR body, not always the full branch log) and covers organism
47+
repos only, not the library ships — disclosed, not hidden.
48+
49+
## 1. Firing rate: neither empty nor saturated
50+
51+
| repo | main commits since 2026-07-17 | with ≥1 lifted claim |
52+
|---|---:|---:|
53+
| PyAutoBrain | 50 | 13 (26%) |
54+
| PyAutoMind | 66 | 3 (5%; registry moves dominate) |
55+
56+
Trigger distribution over the 26 lifted claim-lines: `verified` 17,
57+
`unchanged` 5, `identical` 3, `byte-identical` 2, `no-op` 1. The vocabulary is
58+
not too narrow (it fires regularly) and not too broad (74–95% of ships surface
59+
nothing).
60+
61+
The interesting shape: `verified` dominates, and it mostly lifts the author's
62+
**evidence sentence** ("Verified they actually bite: reverting only
63+
_feature.py fails 7 of the new tests"), not a bare unsupported claim. Shipped
64+
commit messages in this window conspicuously carry "Verified by/against
65+
<probe>" inline — the claim culture mitigation 6 wanted. A zero finding rate
66+
is therefore at least partly deterrence, not only non-engagement.
67+
68+
## 2. Finding rate: zero — and the record cannot say why
69+
70+
Across the 22 review-leg gates: `unverified-claim` FINDINGS raised: **0**.
71+
Grep across the whole Mind (completion records, autonomy log, active/) finds
72+
the category name only in the stage's own shipping records and this prompt.
73+
74+
FINDINGS the review leg did raise in the window were generic-correctness
75+
(2026-07-27 multistart-cadence-int-cast: `int()` truncation → infinite loop,
76+
caught by the run's own review pass 1) or came from **external** adversarial
77+
reviews (Codex gpt-5.6-sol on 07-27, Claude Opus 5 on the 07-29
78+
python-312-floor merges) — not from step 2a.
79+
80+
Distinguishing "unnecessary" from "rubber-stamped": the honest answer is the
81+
ledger cannot distinguish them, and that is the review's central finding (§5).
82+
83+
## 3. Were any load-bearing? One documented exercise; it held
84+
85+
Exactly one autonomy-log row records the adversarial claim pass operating on a
86+
lifted claim: 2026-07-21 interpolator-aggregator-test-mode — *"adversarial
87+
pass on 'no-op outside test mode' claim — gated by is_test_mode()+.exists(),
88+
proven by off-switch test"*. The claim had a falsified-by basis; correctly no
89+
finding; the outcome was unchanged. No ship was held and no correction was
90+
produced by the stage in the window. Its per-ship cost is also ≈ zero (a
91+
stdlib regex plus a few surface lines), so "earning its cost" is a low bar —
92+
but the earning is currently invisible.
93+
94+
Sharpest counter-datum: the one confirmed-wrong load-bearing claim of the
95+
window — the 2026-07-27 "5 siblings affected" count, falsified to 2 by the
96+
external Codex review — lived in an **issue comment**, outside the
97+
commit-message surface the stage reads. The stage could not have caught the
98+
one escape that actually happened. (Widening the scanned surface to issue
99+
text was considered and not recommended: issue prose is where hedged
100+
discussion belongs; the boundary-crossing record the stage guards is the
101+
branch itself.)
102+
103+
## 4. Idle-phrasing exclusion: holding, with two cheap residuals
104+
105+
No changelog/rename chatter is lifted (the pinning test's contract holds on
106+
real data). Residual false positives observed in the retro-measurement:
107+
mid-sentence fragments of wrapped prose (line-based matching lifts "…row, so
108+
the yardstick is real data from the same machine. Verified by"), and narrative
109+
`identical`/`unchanged` describing the *bug*, not the change ("a night the
110+
gate correctly stopped looked identical"). Cost is seconds of reviewer
111+
attention per ship with **no bypass pressure** — unlike the F5 refusal class,
112+
an over-lifted line cannot train bypass-by-default because nothing blocks.
113+
Within budget; no vocabulary change recommended.
114+
115+
## 5. Rubber-stamping: the signature is undetectable as instrumented
116+
117+
The rote signature — claims lifted, reviewed CLEAN, no evidence cited — was
118+
looked for and **cannot be confirmed or refuted**: the ReviewSurface is
119+
ephemeral, and 20 of the 22 gates recorded only "review CLEAN" / "review
120+
self-CLEAN". A healthy pass and a rote one write the identical ledger row.
121+
Two gates only (07-21 above; the August cloud run) left evidence the surface
122+
was engaged at all. Additionally, on autonomous ships the "reader" that
123+
enforcement was delegated to is the branch's own author (`review self-CLEAN`)
124+
— the design's reader-enforcement premise is diluted exactly where the stage
125+
matters most.
126+
127+
## Verdict
128+
129+
**Keep, vocabulary unchanged; close the observability gap.** The instrument
130+
is live and calibrated, the one documented exercise worked as designed, the
131+
false-positive cost is negligible, and the claim culture it targets has
132+
visibly moved toward evidence-inline commit messages. But a stage whose
133+
engagement leaves no trace will go rote silently if it has not already; per
134+
the campaign's own ranking (deleting beats detecting beats reminding), the
135+
fix is to move it from remind-shaped to detect-shaped: the reviewing agent's
136+
verdict gains a **one-line disposition per lifted claim** — `claim →
137+
basis-cited <what> | idle | FINDING` — carried into the ship evidence
138+
(autonomy-log review cell / PR body). A bare CLEAN over a non-empty claims
139+
surface then reads as drift in the ledger, which is the reader-enforcement
140+
the original design promised. Filed as
141+
`draft/feature/pyautobrain/review_claim_dispositions.md` (small, safe).
142+
143+
## Original prompt
144+
145+
# Has the falsified-by checkpoint stage gone rote after ten ships
146+
147+
Type: research
148+
Target: PyAutoBrain
149+
Repos:
150+
- PyAutoBrain
151+
Difficulty: small
152+
Autonomy: safe
153+
Priority: normal
154+
Status: formalised
155+
156+
## What this is
157+
158+
The efficacy review that `docs/agent_failure_modes.md` §9 committed to when
159+
mitigation 6 shipped: *"trial on the next ship series, review whether it went
160+
rote after ~10 ships."* Spun out of PyAutoBrain#130 when that issue was closed
161+
(2026-08-15) — it was the one §9 item still genuinely open, and nothing tracked
162+
it.
163+
164+
This is an investigation producing a written verdict from evidence. No code
165+
change is committed up front; a fix may follow from the finding.
166+
167+
## Background
168+
169+
Mitigation 6 (PyAutoBrain#140, merged 2026-07-17, live) made the review faculty
170+
lift **load-bearing empirical claims** out of a branch's commit messages into
171+
the `ReviewSurface` as `claims to falsify` — the trigger vocabulary is `no-op`,
172+
`byte-identical`, `does-not-affect`, `proven`, `behaviour-preserving`. `AGENTS.md`
173+
step 2a then makes an unsupported one a FINDING of kind `unverified-claim`.
174+
175+
Its design was deliberately reader-enforced rather than an author checklist, and
176+
scoped to load-bearing phrasing only, precisely so it could not decay into the
177+
"remember-to-run checklist" the campaign's own constraints ban. It targets the
178+
A5/F3 failure class — confident-wrong effect-claims.
179+
180+
## Why it needs reviewing
181+
182+
The doc's constraint list bans checklists as a mechanism, and a routine
183+
adversarial pass is the single mechanism most likely to decay into one. The
184+
worry is explicit in the shipping comment: *"the one that needs care not to
185+
become the banned checklist."* A stage that fires on every ship and is waved
186+
through every time is worse than no stage, because it also carries false
187+
assurance.
188+
189+
## What to investigate
190+
191+
Over the real ship history since 2026-07-17:
192+
193+
1. **Firing rate** — on how many ships did `claims to falsify` populate at all,
194+
versus come back empty? An always-empty surface means the vocabulary is too
195+
narrow; an always-full one means it is too broad.
196+
2. **Finding rate** — how many `unverified-claim` FINDINGS were actually raised,
197+
and what happened to each? A stage that never produces a finding across ~10
198+
ships is either unnecessary or being rubber-stamped; distinguish those two.
199+
3. **Were any load-bearing?** For each finding, did falsifying the claim change
200+
the outcome — a correction, a held ship — or was it cosmetic? This is the
201+
measure of whether the stage is earning its per-ship cost.
202+
4. **Idle-phrasing exclusion** — is the load-bearing-only scoping holding, or has
203+
the matcher started lifting incidental prose? Check for false positives of
204+
the kind that trained bypass-by-default in the guard's first hour (the F5
205+
cost column).
206+
5. **Rubber-stamping** — look for the signature: claims lifted, reviewed CLEAN,
207+
no evidence cited in the review. That is the rote failure, and it looks
208+
identical to a healthy pass unless you read what the reviewer actually did.
209+
210+
## Deliverable
211+
212+
A verdict with the numbers behind it, and one of: keep as-is, narrow/broaden the
213+
trigger vocabulary, or retire the stage. If the finding is "it went rote", say
214+
what would fire instead — per the campaign's own ranking, deleting the
215+
possibility beats detecting it, and detecting beats reminding.
216+
217+
## Method note
218+
219+
Validate the instrument before trusting it: check that the review faculty's
220+
claim-lifting still runs on a branch with known load-bearing claims before
221+
concluding anything from a low firing rate. A null result that looks like a
222+
finding (D1) is the exact failure this campaign catalogued.
223+
224+
<!-- formalised by the Intake (Conception) Agent on 2026-08-15 from user-intake -->

0 commit comments

Comments
 (0)