Skip to content

Commit 7c15eef

Browse files
committed
review: verdicts record a disposition per lifted claim
The change the 2026-08-18 falsified-by efficacy review recommended (agent_failure_modes.md item 6 Outcome): across 22 ship gates the ledger could not distinguish a healthy adversarial claim pass from a skipped one — both wrote a bare 'review CLEAN'. Now, when the ReviewSurface lifts any 'claims to falsify', the verdict carries one disposition line per claim (basis-cited / idle / FINDING unverified-claim), written by the reviewer at verdict time; a bare CLEAN over a non-empty claims surface is malformed evidence. An empty surface requires nothing, so the 74-95% of ships that lift no claims gain no busywork. Surfaces: faculty AGENTS.md step 2a + verdict mapping; _review.py human-emit epilogue (printed only when claims lifted); AUTONOMY.md autonomous-ship-gate review leg; ship_library reference.md gate evidence line. No trigger-vocabulary change (measured as neither empty nor saturated). 2 new pinning tests; suite 351 passed. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH4NizvBK2jki2Uh5TMABh
1 parent 102633e commit 7c15eef

6 files changed

Lines changed: 58 additions & 7 deletions

File tree

AUTONOMY.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,10 @@ applicability rule so "n/a" is a stated fact, never an assumption:
158158
feel stronger.
159159
3. **Review** — review-faculty verdict **CLEAN**
160160
(`agents/faculties/review/AGENTS.md`). FINDINGS → resolve and re-review, or
161-
park to a human checkpoint; BLOCKED → park.
161+
park to a human checkpoint; BLOCKED → park. When the surface lifted any
162+
`claims to falsify`, CLEAN carries one disposition line per claim
163+
(basis-cited / idle / finding — faculty AGENTS.md step 2a); a bare CLEAN
164+
over a non-empty claims surface is malformed evidence, not CLEAN.
162165
4. **Heart** — verdict **GREEN** or **STALE**, or **YELLOW whose reason set is
163166
contained in the set the human acknowledged at launch**. Heart observes
164167
organism state, not the branch (the audit confirmed its legs never see

agents/faculties/review/AGENTS.md

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,11 +62,21 @@ Heart and the agent reasons over the verdict.
6262
category `unverified-claim` — the author asserted an effect they did not
6363
show. Scope it to genuinely load-bearing claims (an effect that, if wrong,
6464
ships a bug); an idle turn of phrase is not a finding — say so and move on,
65-
so the pass does not decay into rote noise.
65+
so the pass does not decay into rote noise. **Record a disposition per
66+
claim**: the verdict carries one line per lifted claim —
67+
`claim: "<lifted line>" → basis-cited: <the test/measurement/diff that shows it> | idle | FINDING (unverified-claim)`
68+
— written by the reviewer at verdict time, never by the author. This is
69+
what makes the pass auditable: the 2026-08-18 efficacy review
70+
(`docs/agent_failure_modes.md` item 6 Outcome) found that across 22 ship
71+
gates a healthy pass and a skipped one wrote the identical ledger row.
72+
An empty surface requires nothing.
6673
3. Map the outcome to the verdict: any unresolved must-fix → **FINDINGS**
6774
(ranked list, file:line, failure scenario) — including any
6875
`unverified-claim` from step 2a; nothing → **CLEAN**; could not
69-
complete steps 1–2 → **BLOCKED** (say why).
76+
complete steps 1–2 → **BLOCKED** (say why). A CLEAN over a non-empty
77+
`claims to falsify` surface **must** carry the step-2a disposition lines —
78+
without them the evidence is malformed, not CLEAN (the ship-checkpoint
79+
reader can see the omission; that is the reader-enforcement).
7080

7181
## Run
7282

agents/faculties/review/_review.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -190,6 +190,12 @@ def emit_human(surfaces: list[dict]) -> None:
190190
print("file:line, failure scenario) | BLOCKED (could not review — say why).")
191191
print("A load-bearing claim above with no falsified-by basis in the branch is")
192192
print("a FINDING (unverified-claim) — see the faculty AGENTS.md.")
193+
if any(s.get("claims_to_falsify") for s in surfaces):
194+
print("Record ONE disposition line per lifted claim in the verdict:")
195+
print(' claim: "<lifted line>" -> basis-cited: <test/measurement/diff>'
196+
" | idle | FINDING (unverified-claim)")
197+
print("A CLEAN over a non-empty claims surface without dispositions is")
198+
print("malformed evidence, not CLEAN (faculty AGENTS.md step 2a).")
193199

194200

195201
def main(argv=None) -> int:

docs/agent_failure_modes.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -179,9 +179,11 @@ Each: catalogue entries caught → why it fires at the decisive moment → cost
179179
agent's verdict gains a one-line disposition per lifted claim
180180
(basis-cited / idle / finding) recorded in the ship evidence, so a rote
181181
pass becomes visible ledger drift per this doc's own ranking (detecting
182-
beats reminding). Filed: PyAutoMind
183-
`draft/feature/pyautobrain/review_claim_dispositions.md`; full numbers in
184-
PyAutoMind `complete/2026/08/falsified-by-checkpoint-efficacy-review.md`.
182+
beats reminding). Implemented 2026-08-18: faculty AGENTS.md step 2a + the
183+
surface epilogue + the AUTONOMY.md review leg + the ship evidence format
184+
(record: PyAutoMind `complete/2026/08/review-claim-dispositions.md`); full
185+
numbers in PyAutoMind
186+
`complete/2026/08/falsified-by-checkpoint-efficacy-review.md`.
185187

186188
## 6. The memory system, attacked honestly
187189

skills/ship_library/reference.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,8 @@ what the human validates instead of the pre-approval they didn't give:
6262
- Effective level: safe (header: <level>, cap: <work-type> → <cap>)
6363
- Plan: on the issue (#<n>), written at start, unmodified since
6464
- Gate: tests <pass counts / n-a + why> · smoke <result / n-a + why> ·
65-
review CLEAN · Heart <GREEN | YELLOW within launch ack>
65+
review CLEAN <+ one disposition per lifted claim, when the surface lifted
66+
any — faculty AGENTS.md step 2a> · Heart <GREEN | YELLOW within launch ack>
6667
- [ ] Human: plan sound in hindsight?
6768
- [ ] Human: diff matches plan (no scope creep)?
6869
- [ ] Human: merge, amend, or reject — then log the outcome

tests/test_review_claims.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,3 +58,32 @@ def test_verified_and_safe_to_delete_are_claims():
5858
claims = load_bearing_claims(text)
5959
j = " ".join(claims).lower()
6060
assert "verified" in j and "safe to delete" in j and "zero diff" in j
61+
62+
63+
def test_emit_human_demands_dispositions_when_claims_lifted(capsys):
64+
from _review import emit_human
65+
66+
surface = {
67+
"repo": "PyAutoDemo", "path": "/tmp/x", "branch": "feature/x",
68+
"base": "abc123def456", "commits_ahead": 1, "commits": ["abc fix"],
69+
"shortstat": "1 file changed", "files": ["M\tf.py"],
70+
"risk_flags": [], "claims_to_falsify": ["This change is a no-op for CI."],
71+
}
72+
emit_human([surface])
73+
out = capsys.readouterr().out
74+
assert "ONE disposition line per lifted claim" in out
75+
assert "malformed evidence" in out
76+
77+
78+
def test_emit_human_no_disposition_demand_on_empty_surface(capsys):
79+
from _review import emit_human
80+
81+
surface = {
82+
"repo": "PyAutoDemo", "path": "/tmp/x", "branch": "feature/x",
83+
"base": "abc123def456", "commits_ahead": 1, "commits": ["abc fix"],
84+
"shortstat": "1 file changed", "files": ["M\tf.py"],
85+
"risk_flags": [], "claims_to_falsify": [],
86+
}
87+
emit_human([surface])
88+
out = capsys.readouterr().out
89+
assert "disposition" not in out.lower()

0 commit comments

Comments
 (0)