mind: test-mode-bypass-assertion-ties close-out + three CTI Phase 5 prompts

complete/2026/08/test-mode-bypass-assertion-ties.md

@@ -0,0 +1,140 @@
+## test-mode-bypass-assertion-ties
+- issue: https://github.com/PyAutoLabs/PyAutoFit/issues/1519 (closed)
+- completed: 2026-08-24
+- library-pr: https://github.com/PyAutoLabs/PyAutoFit/pull/1520 (merged 438f56fac)
+- summary: The PYAUTO_TEST_MODE=2/3 bypass evaluated the model at the prior
+  medians, so a model with identical priors plus an ordering assertion (the
+  exchange-degeneracy idiom, e.g. PyAutoCTI trap models) tied exactly there and
+  check_assertions hard-failed the run. The bypass now picks its point through a
+  shared `_test_mode_valid_parameter_vector` — prior medians first, then
+  `default_rng(seed=0)` prior draws, each candidate validated — so the vector it
+  evaluates AND stores satisfies the assertions.
+- key-finding: **the prompt's own suggested fix would not have worked.** The
+  2026-08-09 note had narrowed this to a one-liner (move `instance_from_vector`
+  inside the existing try, or pass `ignore_assertions=True`). Reading main
+  d3625a8 found THREE sites, and both one-liners fix only the first:
+  1. `abstract_search.py:1007` — instantiation outside the FitException guard;
+  2. `_build_fake_samples:1112` — the perturbed samples are the median vector
+     scaled UNIFORMLY (1.001/0.999/1.002), and a uniform scale preserves an
+     ordering tie, so every stored sample fails the same assertion;
+  3. `SamplesSummary.max_log_likelihood` (`interface.py:122`) is `@to_instance()`
+     with `recover="raise"` — it extends SamplesInterface directly so it does NOT
+     inherit Samples' next-valid recovery — so `result.max_log_likelihood_instance`
+     raises SamplesException, and `Result.instance` catches only AttributeError.
+- key-finding: **TEST_MODE=3 was broken too and nobody had noticed.** It never
+  calls instance_from_vector in the bypass, so it survived the fit and died at
+  the first `result.max_log_likelihood_instance`. Confirmed by reproduction, not
+  just by reading. Fixing the stored vector fixes modes 2 and 3 together.
+- trap: do NOT re-run a bypass reproduction without clearing `output/` first. A
+  bypassed fit calls `paths.completed()`, so a second run with the same
+  unique_tag takes `result_via_completed_fit` and replays the OLD (broken)
+  samples — which reads exactly like "the fix didn't work". Cost one false
+  negative during verification.
+- trap: `af.m.MockAnalysis` maps its likelihood over the model and returns a
+  LIST for an `af.Collection`, which the bypass's `float()` rejects. Regression
+  tests needed a small float-returning analysis instead.
+- behaviour-change: mode 3 now instantiates the model once (previously zero
+  times). A model whose constructor raises a non-FitException at the medians now
+  fails at fit time rather than result time — same failure, surfaced earlier.
+  Flagged in the PR body for downstream repos.
+- verification: reproduced on clean main first (mode 2 raised FitException in the
+  fit; mode 3 raised SamplesException at result access), then both modes complete
+  and select the identical vector after. All 5 new tests in
+  `TestBypassToleratesAssertionTies` fail against the un-patched source. Full
+  suite 2016 passed / 34 skipped / 0 failed; CI green on all three legs
+  (unittest 3.12, unittest 3.13, unittest-nojax) plus Docs.
+- gate-caveat: shipped from a web-github session where `pyauto-heart` is
+  unreachable, so the readiness gate ran in the WORKFLOW.md fallback form (full
+  library suite as the gate). No Heart verdict was recorded for this task; CI
+  green at merge is the stronger confirmation that stands in its place. The
+  workspace-impact grep was likewise not run (workspace clones absent) — API
+  Changes are "none, internal", so option (iii) was inferred, not measured.
+- follow-up: `autocti_workspace` documents this artifact in its AGENTS.md as a
+  workaround. Delete that note now the fix has shipped — the testmode-env-drift
+  precedent ("delete the trap, don't document it"). Separate repo, separate task.
+- follow-up: re-enable autocti_workspace smoke coverage of the
+  `modeling/start_here.py`-class scripts (CTI epic Phase 5) that this unblocks.
+- environment: web-github; no worktree was ever created, so there is none to
+  remove. PyAutoFit was worked in a session clone at /home/user/pyautofit.
+
+## Original prompt
+
+# TEST_MODE bypass crashes on ordered-parameter assertion ties
+
+Type: bug
+Target: PyAutoFit
+Repos:
+- @PyAutoFit
+Difficulty: small
+Autonomy: supervised
+Priority: normal
+Status: formalised — STILL REPRODUCES; see the 2026-08-09 note before grading this against main
+Filed: 2026-07-17 (backfilled from git)
+Issued: 2026-08-24
+
+## 2026-08-09 — do NOT mistake the adjacent FitException catch for this fix
+
+Checked by the draft/ sweep against PyAutoFit main (`3b960609`). The bypass path
+in `abstract_search.py` **now catches `exc.FitException`** and continues with the
+`-1e99` sentinel, logging "TEST MODE 2: likelihood verification raised
+FitException … treating as a resample-rejected instance". That reads exactly like
+this prompt's suggested fix. **It is not.** The bug below still reproduces.
+
+The catch wraps only the likelihood call. The model instantiation is on the line
+*before* the `try`:
+
+```python
+if call_likelihood:
+    instance = model.instance_from_vector(vector=parameter_vector)   # <-- outside
+    try:
+        log_likelihood = float(analysis.log_likelihood_function(instance))
+    except exc.FitException as e:
+        ...
+```
+
+and `instance_from_vector` → `instance_for_arguments` → `check_assertions`
+(`autofit/mapper/prior_model/abstract.py:193`) is precisely what raises
+`exc.FitException("N assertions failed!")` when an ordering assertion ties at the
+prior medians. `ignore_assertions` defaults to `False` and the bypass does not
+pass it. So the assertion exception escapes the guard entirely and still
+hard-fails the run.
+
+The upside: the fix is now a one-liner rather than the "catch and retry with a
+perturbation" design sketched below. Two options, both cheap and both
+deterministic:
+
+- move the `instance_from_vector` call inside the existing `try` — the sentinel
+  path already does the right thing for a rejected instance; or
+- pass `ignore_assertions=True` at the bypass instantiation, on the grounds that
+  a verification eval at the medians is not a sampled point and assertions exist
+  to steer sampling.
+
+The second is probably the better semantics (a tied median is not a pathological
+model), but it changes what the verification eval attests to — pick deliberately.
+Prefer either over adding perturbation logic.
+
+`Difficulty:` stays small. The § Blocks note below still holds.
+
+---
+
+Found during the CTI resurrection epic (Phase 4, 2026-07-17). `PYAUTO_TEST_MODE=2/3`
+bypass evaluates the model at the **prior medians**. A model whose components have
+identical priors plus an ordering assertion (the standard idiom for breaking
+exchange degeneracy, e.g. PyAutoCTI trap models with
+`model.add_assertion(trap_0.release_timescale < trap_1.release_timescale)`)
+ties exactly at the medians, so the bypass evaluation raises
+`autofit.exc.FitException: GreaterThanLessThanAssertion` and the script crashes.
+
+Real samplers resample assertion-failing points gracefully — this is purely a
+bypass-path artifact, and it makes every ordered-trap CTI workspace script
+un-smokeable at TEST_MODE=2 (reproduced with a bare
+`model.instance_from_prior_medians()`; TEST_MODE=1 passes).
+
+Suggested fix: at the bypass evaluation, catch `FitException` from assertions
+and retry with a small deterministic perturbation of the unit-cube point (or a
+seeded random draw), mirroring what a real sampler does. Keep it deterministic
+so smoke runs stay reproducible.
+
+Blocks: autocti_workspace smoke coverage of `modeling/start_here.py`-class
+scripts (CTI epic Phase 5); the workspace documents the artifact in its
+AGENTS.md meanwhile.

complete/2026/08/testmode-assertion-note-removal.md

@@ -0,0 +1,78 @@
+## testmode-assertion-note-removal
+- issue: https://github.com/PyAutoLabs/autocti_workspace/issues/24 (closed)
+- completed: 2026-08-24
+- workspace-pr: https://github.com/PyAutoLabs/autocti_workspace/pull/25 (merged 2933cddd)
+- workspace-pr: https://github.com/PyAutoLabs/autocti_workspace_test/pull/17 (merged bfdb876b)
+- workspace-pr: https://github.com/PyAutoLabs/autocti_assistant/pull/21 (merged 6a0f645c)
+- summary: Follow-up to test-mode-bypass-assertion-ties (PyAutoFit#1520, merged
+  438f56fac). Three CTI repos documented the now-fixed TEST_MODE bypass crash as a
+  live artifact readers must work around. Deleted per the testmode-env-drift
+  precedent ("delete the trap, don't document it") rather than updated. All three
+  branches proven ancestors of their main (0 unmerged commits each).
+- key-finding: the prompt said "if a sibling note exists" — BOTH siblings existed,
+  and the second was not a note. In autocti_workspace_test/AGENTS.md the claim was
+  a parenthetical RATIONALE for a convention ("integration scripts are ...
+  single-trap (because ordered traps tie under the bypass)"). Deleting the whole
+  bullet would have quietly repealed a test-design rule. Removed only the
+  rationale; left the convention standing and flagged it for a maintainer.
+- open-question: autocti_workspace_test's single-trap convention now has no stated
+  reason. If it existed only to dodge the bypass crash it can be dropped, and
+  multi-trap ordered models exercised — arguably better coverage, since ordered
+  traps are the realistic CTI case. If it also exists for runtime/simplicity the
+  bullet needs that reason written in. Deliberately not guessed.
+- key-finding: the third site was in a SKILL (autocti_assistant
+  skills/ac_fit_cti_model.md), i.e. what the assistant tells users — a stale
+  "known artifact" there is active advice to work around a bug that no longer
+  exists. Worse than a stale AGENTS.md note. Its .claude/skills/ copy is a
+  SYMLINK (git mode 120000), so one edit covered both discovery surfaces; do not
+  assume the mirrored skills dirs are copies.
+- deliberately-untouched: the add_assertion example at ac_fit_cti_model.md:54.
+  The ordering-assertion API is unchanged and still the right idiom for breaking
+  exchange degeneracy — only the bypass's handling of it was broken.
+- trap: autocti_assistant's `wiki-currency` check is RED ON MAIN, independently of
+  any PR. Confirmed by dispatching it on main at 960fdd1c (run 32762029277):
+  failed identically. Proof it is not the PR's: the drift-report artifacts are
+  964 bytes (PR) vs 961 (main), and the report header's `assistant_ref` line
+  differs by exactly 3 chars (refs/pull/21/merge vs refs/heads/main) — so every
+  drift FINDING is byte-identical. Artifact byte-size diffing is a cheap way to
+  prove two CI failures are the same failure when the log hides the detail.
+- trap: that workflow redirects each sub-check into drift-report.md
+  (`>> "$REPORT" 2>&1`), so the job log NEVER names which of its five checks
+  failed. Read the artifact; do not try to infer it from the log tail.
+- decision: PR#21 was merged deliberately over that known-bad base (human-
+  authorised at /prm) rather than regenerating an API baseline inside a docs PR.
+  The drift is filed separately as draft/bug/autocti/wiki_currency_baseline_drift.md.
+- gate-note: autocti_workspace has NO CI configured at all (zero workflow runs,
+  zero checks). #25 was merged on explicit human authorisation per /prm's
+  no-checks guard. autocti_workspace_test's smoke ran `changes` green and skipped
+  `smoke` by path filter — correct for a docs-only diff, not a pending check.
+- follow-up: draft/test/autocti/phase5_smoke_reenable_ordered_trap_scripts.md —
+  re-enable the smoke coverage the PyAutoFit fix unblocks (CTI epic Phase 5).
+- environment: web-github; no worktree. Clones at /home/user/autocti_{workspace,
+  workspace_test,assistant}.
+
+## Original prompt
+
+# Delete the TEST_MODE ordered-assertion workaround note from @autocti_workspace AGENTS.md
+
+Type: docs
+Target: autocti_workspace
+Repos:
+- @autocti_workspace
+- @autocti_workspace_test
+- @autocti_assistant
+Difficulty: small
+Autonomy: supervised
+Priority: normal
+Status: formalised
+Filed: 2026-08-24
+Issued: 2026-08-24
+
+Re-homed from `draft/triage/` by the filing session: intake classified this
+`triage` / `Target: PyAutoFit` / `too-large` on low confidence, which is wrong on
+all three counts — the PyAutoFit fix has already shipped, so nothing here touches
+library source. This is a prose deletion across up to three workspace repos.
+
+Delete the TEST_MODE ordered-assertion workaround note from @autocti_workspace AGENTS.md. PyAutoFit#1520 (merged 438f56fac, 2026-08-24) fixed the bypass so a model with identical priors plus an ordering assertion no longer ties at the prior medians — the bypass now picks a deterministic assertion-valid point via _test_mode_valid_parameter_vector, at TEST_MODE 2 and 3. The workspace AGENTS.md documents the crash as a live artifact and tells readers to work around it; that text is now wrong. Delete the note rather than update it — the testmode-env-drift precedent is delete the trap, do not document it. Verify the note's exact wording and location in autocti_workspace first; if a sibling note exists in autocti_workspace_test or autocti_assistant, remove those too.
+
+<!-- formalised by the Intake (Conception) Agent on 2026-08-24 from user-intake -->

complete/index.md

@@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema:
 only then grep a dated bucket. Curators: edit the band between the CURATED
 markers; everything below GENERATED is rebuilt.
 
-1100 records across 7 buckets.
+1102 records across 7 buckets.
 
 <!-- CURATED:START -->
 ## Highlights
@@ -211,9 +211,11 @@ _(curate hard-won records here — survives regeneration.)_
 - [tenant-firewall-drift-aug](2026/08/tenant-firewall-drift-aug.md) — issue #198; 9 → OK, and a green `--check` is only evidence for the organs actually checked out
 - [tenant-firewall-hygiene-extras](2026/08/tenant-firewall-hygiene-extras.md) — auto-closed by the merge
 - [tenth-sample-hardcoded-index](2026/08/tenth-sample-hardcoded-index.md) — results/database/aggregator tutorials hardcoded stored-sample index 9 ("the tenth
+- [test-mode-bypass-assertion-ties](2026/08/test-mode-bypass-assertion-ties.md) — The PYAUTO_TEST_MODE=2/3 bypass evaluated the model at the prior
 - [test-mode-fit-exception-finalization](2026/08/test-mode-fit-exception-finalization.md) — `PYAUTO_TEST_MODE=1` no longer finalizes a model point whose reconstruction raises `FitException`; it substitu…
 - [test-mode-samples-info-hook-contract](2026/08/test-mode-samples-info-hook-contract.md) — `NonLinearSearch._test_mode_samples_info()`'s docstring told subclasses
 - [test-performance-board](2026/08/test-performance-board.md)
+- [testmode-assertion-note-removal](2026/08/testmode-assertion-note-removal.md) — Follow-up to test-mode-bypass-assertion-ties (PyAutoFit#1520, merged
 - [transformed-message-semantics-doc](2026/08/transformed-message-semantics-doc.md)
 - [undo-community-file-declutter](2026/08/undo-community-file-declutter.md) — Undid the 2026-08-19 community-file declutter (#248 Mind, #32 Memory):
 - [uniform-prior-bounds-numpy-path](2026/08/uniform-prior-bounds-numpy-path.md) — auto-closed by the merge's `Closes` line