agent_failure_modes.md

Agent failure modes — the validated catalogue and structural mitigations

Status: design deliverable for PyAutoBrain#130 (build-chain campaign PyAutoBuild#155, Phase 5 — deliberately last, so every earlier phase's errors enlarge the evidence). Method: the original 2026-07-15 catalogue's checkable claims were re-measured, and the 2026-07-16 campaign session is mined as an out-of-sample test of the central hypothesis — by an agent that was reading these briefs while making the new errors, which is itself the strongest datum here.

1. The original catalogue, validated

Checked 2026-07-16: git 2.34.1 (D1's merge-tree needs ≥2.38 — confirmed); PyAutoMind/repos.yaml now carries 33 repos (was 30 — the hand-list gap C1 measures keeps growing); smoke_tests.txt is 11 entries (A2's false claim confirmed falsifiable in seconds); the use_jax defaults (A4) were re-verified during Phase 3. Nothing in A–E was refuted. The A–E taxonomy survives, with one compression (§3).

2. The out-of-sample additions (2026-07-16, one campaign session)

Same discipline — claim, truth, cost, caught by:

#	error	caught by
F1	git commit in the shared PyAutoMind checkout swept a concurrent session's staged rename (×1), then a directory-scoped commit swept two more (×2), then a concurrent session swept THIS session's staged records (×3, reverse direction) — E1 repeated three times in one day, with a fresh memory note about E1 written that same morning	git log reading, after push, each time
F2	cmd | head; $? read the pager's exit, not the command's — twice — and black --check -q's silence was read as cleanliness once	self, on re-reading output (luck)
F3	A checkpoint claim ("no DISABLE_JAX pins remain — question may be moot") built on a head -4 grep was published to a GitHub issue before being falsified	a deliberate falsifying probe (D2's habit, applied on purpose this time)
F4	Two stale claims of this session's own completed tasks blocked new work	worktree_check_conflict refused — forced the completion recording that should already have happened
F5	The PyAuto API gate refused a pure file-inspection command (pattern-matched path text as API symbols)	n/a — a refusal false positive; cost: one bypassed re-run

F1 is the headline: the trap was documented in the morning, in a memory note written specifically about it, by the agent that then walked into it three times before dinner. B1 is not an anecdote; it is the normal case.

3. The compressed taxonomy

The A–E grouping compresses to two root causes plus a hygiene class:

1. Verification aimed at the believed path (A1–A5, D1, D3, F2, F3): real work producing real evidence for a proposition nobody questioned. The agent reliably verifies — and reliably verifies the wrong thing.
2. State consulted from memory instead of from the world (B1, B2, C1, C2, E1, F1, F4): notes, recollections and stale registrations standing in for git status, repos.yaml, and the claims ledger at the decisive moment.
3. Convention/shared-state hygiene (E2–E4, F5's cost column): real but secondary; the guards that exist here already work.

4. The hypothesis, tested on the new data

"Refusals catch, reminders don't" survives — with a sharper statement and a measured cost column.

• Refusals that fired correctly this session: worktree_check_conflict ×2 (F4), the test_pre_build_skill drift guard (caught a real doc/script drift on clean main during a ship), lifecycle.py check (flagged a half-done completion), and the brand-new validate_env_profiles — which on first contact found 3 dead config patterns and 15 vacuous-JAX scripts that a day of human-authored enumeration had missed. Hit rate on things they guard: still effectively 100%.
• Informs that failed this session: the E1 memory note (three more sweeps); the agent's own live awareness of the A5 error shape (it re-committed that exact shape in F3 while mid-campaign about it). Awareness is an inform. Even self-awareness. That is the sharper statement: the dividing line is not documentation-vs-tooling, it is whether anything interrupts the action, because at the decisive moment the agent is not consulting anything — not notes, not even its own recent conclusions.
• The cost column (previously unmeasured): F5 shows refusals false-positive; the API gate misfired on path text and cost one bypass. One misfire against dozens of correct refusals across two days is a good ratio, but every new refusal proposed below must state its expected false-positive surface — a refusal that misfires often trains bypass-by-default, which is worse than no guard (the brief's own routing-around warning).

5. Ranked mitigations

Each: catalogue entries caught → why it fires at the decisive moment → cost → how it fails.

1. set -o pipefail (+ never -q on check-mode tools) as the agent-shell default. Catches F2's entire class (three instances in one day) by making the instrument loud instead of the agent careful — the purest delete-the-trap available, since the mistake becomes unmakeable rather than detectable. Cost: zero per task; failure mode: legitimately-failing pipe heads (grep-no-match) now need explicit handling — small, visible, and itself error-revealing. This is a one-line hook/profile change; do it first.

2. A Mind commit guard (refusal). In the shared PyAutoMind checkout, refuse any git commit that (a) lacks explicit -- file pathspecs, or (b) would include index entries outside those paths (another session's staged work). Catches E1 + F1 (four incidents across two days, the highest-frequency error in the whole catalogue). Fires at the decisive moment by construction — it is the action. Cost: a wrapper or PreToolUse check scoped to one repo; false-positive surface: deliberate multi-file registry commits, handled by listing the files (which is the desired behaviour anyway).

   Outcome (2026-07-22, v1.3) — the false-positive surface was the real cost, and it was bigger than predicted. Deployed as bin/mind_commit_guard.py, the guard fired live three times, every one a false positive on its own author (v1.0 a quoted gh body; v1.1 a cd away from Mind; v1.2 a cd inside a for-loop body), against zero confirmed catches of a real bad Mind commit — the -- <files> habit did that work. Each FP came from the same place: attributing a commit to a repo by parsing arbitrary shell. Since §4's own argument is that a noisy refusal trains bypass-by-default, v1.3 narrows the guard to deny only when confident: _unattributable() fails open on compounds (for/while/ case), subshells and brace groups, and any cd that is not clause-leading. The two high-confidence denials (no -- section; a directory pathspec) are unchanged. Retiring the guard outright was considered and rejected — the narrowed deny surface is cheap and still covers the exact E1/F1 shape — but note the general lesson: a refusal whose trigger requires modelling an unbounded input space will spend its budget on false positives. Prefer refusals whose precondition is decidable (mitigation 3's merged-branch check, which has true-positived).

3. Stale-claim auto-expiry (refusal-adjacent). F4 shows completed tasks linger in active.md and block; the conflict guard catches it late but correctly. Cheaper at the source: worktree_remove (already the cleanup choke-point) warns-then-offers the active.md release + completion record when the branch it removes is merged. Catches F4's class before it costs a conflict roundtrip. Cost: one prompt in an existing tool.

4. Blessed instruments for the repeated questions (C2, D1, D2): the campaign's own validate_env_profiles is the proof the pattern works — the question "is this config sane?" got one tested implementation and immediately out-performed recall. Remaining candidates, each a small task: "does this branch contribute to main?" (D1/D2's hand-rolled disaster — pyauto-gut's remit), and "which repos exist?" → repos.yaml is already canonical; the missing piece is that nothing routes to it, which is mitigation 5.

5. Make the router consult the body map, not the agent. C2's lesson ("nothing routes the agent to the artifact while deciding") generalises: wherever a conductor needs a repo list, it should call the sizing faculty / repos.yaml loader rather than accept a hand-list argument. Grep the conductors for hand-list parameters; each is a small refactor. Fires at the decisive moment because the enumeration happens inside the tool rather than in the prompt.

6. Adversarial pass as a checkpoint stage, not a virtue. It caught the three worst 07-15 errors and F3 — every time it actually ran. It cannot be a hook (it is judgment), but it can be a structural stage: the ship skills' checkpoint format gains a required "falsified-by" line per load-bearing claim (what would make this false; what was run to check). This is dangerously close to the banned checklist — the honest difference is that it is enforced by the checkpoint reader (the human sees a missing falsified-by line), not remembered by the agent. Cost: minutes per ship; failure mode: rote compliance — mitigate by keeping it to load-bearing claims only. Trial it; measure whether it goes rote.

   Outcome (2026-08-18, the committed ~10-ship efficacy review) — not proven rote; proven unobservable, which is its own finding. Shipped 2026-07-17 (PyAutoBrain#140) as the review faculty's claims to falsify surface + step-2a unverified-claim finding — reader-enforced as designed, but note that on autonomous ships the "reader" is the branch's own author (review self-CLEAN in the ledger). Across the 22 ship gates with a review leg since go-live (21 autonomy-log rows 2026-07-17→08-01 plus one August cloud-session faculty run): unverified-claim findings raised: 0; records showing the claim pass demonstrably exercised: 2 (2026-07-21 interpolator-aggregator-test-mode — the "no-op outside test mode" claim checked against its off-switch test; the 2026-08 autohands-firewall-allowlist cloud run). The other ~20 verdicts are bare "review CLEAN" — a healthy pass and a rote one write the identical ledger row, exactly the failure signature this trial was supposed to look for. The instrument itself is healthy: probe claims lift 3/3 and the 5 pinning tests are green (re-validated 2026-08-18), and over the two organism repos' real merged history since go-live it fires on 13/50 (Brain) and 3/66 (Mind) merge messages — neither always-empty nor saturated. verified does most of the lifting (17/26 lines) and mostly lifts the author's evidence sentence, not a bare claim: shipped messages in this window conspicuously carry "Verified by/against " inline, so the zero finding rate is at least partly deterrence, not only non-engagement. Idle-phrasing scoping holds at the level that matters (no changelog chatter lifted; residual false positives — mid-sentence wrap fragments, narrative identical/unchanged — cost seconds and carry no bypass pressure, unlike F5's refusal class). The one confirmed-wrong load-bearing claim in the window (the 2026-07-27 "5 siblings" count, falsified by an external Codex review) lived in an issue comment — outside the commit-message surface this stage reads. Verdict: keep, vocabulary unchanged; close the observability gap — the reviewing agent's verdict gains a one-line disposition per lifted claim (basis-cited / idle / finding) recorded in the ship evidence, so a rote pass becomes visible ledger drift per this doc's own ranking (detecting beats reminding). Implemented 2026-08-18: faculty AGENTS.md step 2a + the surface epilogue + the AUTONOMY.md review leg + the ship evidence format (record: PyAutoMind complete/2026/08/review-claim-dispositions.md); full numbers in PyAutoMind complete/2026/08/falsified-by-checkpoint-efficacy-review.md.

6. The memory system, attacked honestly

The day's evidence cuts both ways. Failures: B1, B2, and F1's note that did not fire — memory as tripwire is refuted twice over. Successes: the intake-handfix note correctly guided a live intake (consulted deliberately at task start), and the campaign-state note is what makes cross-session continuation possible at all. Verdict: memory earns its keep as consult-on-purpose context and fails reliably as guard-at-the-moment. The policy that follows: guard-shaped memories ("NEVER do X in Y") are candidates for conversion into refusals (mitigation 2 is exactly the E1 note, converted); a memory that cannot be converted should state when it will be consulted, or it is a hope, not a mechanism.

7. Estimating the unnoticed set

Method: count claims that crossed an external boundary (issue comment, commit message, PR body, user-facing summary) and were later found wrong, as a lower bound on the rate at which unverified boundary-crossing claims are wrong. This campaign: 3 such catches (A2 on 07-15; F3's published checkpoint; the Phase 1 brief's "origin READMEs correctly pin" claim, half-refuted by measurement) across roughly 40 boundary-crossing checkpoint claims — ~7% caught-late rate among claims that WERE eventually re-derived. The honest estimate is that claims never re-derived are wrong at a comparable rate: in a session producing dozens of boundary-crossing claims, expect 1–3 undetected false statements to be live in the record right now. The mitigation is not zero-defect (impossible) but boundary discipline: mitigation 6 targets exactly the boundary-crossing moment.

8. Rejected

• Documentation as primary mechanism — refuted a third time by F1, now with the strongest possible variant (same-day self-authored note).
• "Try harder" / awareness — refuted by F3: the error was made during a campaign about that error, by the agent writing about it.
• A universal pre-action checklist — banned by the brief, and F1 shows why: the decisive moment has no checklist-consulting step to hook into.
• Rejecting the refusals hypothesis — attempted per the brief; the new data strengthened it instead. Its honest weaknesses: refusals only exist where someone built one (coverage is reactive), and F5's false-positive cost is real. Neither overturns the hit-rate asymmetry.

9. Open decisions (human) + migration

• Mitigation 1 (pipefail default): approve as a harness/profile change — one line, highest ratio. 2 (Mind commit guard) and 3 (claim expiry): small tasks, file on approval. 4–5: mine the conductors, file individually. 6 (falsified-by lines): trial on the next ship series, review whether it went rote after ~10 ships — done 2026-08-18, see item 6's Outcome (verdict: keep + per-claim disposition lines; the review itself was PyAutoMind complete/2026/08/falsified-by-checkpoint-efficacy-review.md).
• Whether F5's API-gate false-positive class warrants a scope fix (only scan arguments that are Python code, not path strings) — small, filed separately.
• Each mechanism lands as its own PR behind its own plan; nothing here changes behaviour by merging this document.

Trust nothing here

Written by the agent that made F1–F3 while writing about A–E. The §7 estimate is constructed from this document's own author's error rate and should be treated as optimistic. Every §1 number is re-checkable in seconds; §2's incidents are all in the campaign's issue trail (PyAutoBuild#155/#156/ #161, PyAutoHeart#83) with commands and commit SHAs.