Skip to content

fix(branchsync): settle wedged custody records at the kept local head - #1

Closed
hmcdaniel03 wants to merge 20 commits into
mainfrom
fm/nm-824-custody-release
Closed

fix(branchsync): settle wedged custody records at the kept local head#1
hmcdaniel03 wants to merge 20 commits into
mainfrom
fm/nm-824-custody-release

Conversation

@hmcdaniel03

Copy link
Copy Markdown
Owner

Intent

Work the maintainer's requested revisions on the ALREADY-OPEN PR kunchenguid#883 (kunchenguid/no-mistakes), which fixes issue kunchenguid#824. This is a revision round on an existing PR, not a new change: the PR stays open, the branch stays fm/nm-824-custody-release, and no second PR is ever opened. Delivery must push to origin (our fork hmcdaniel03/no-mistakes) with --force-with-lease on fm/nm-824-custody-release only, so PR kunchenguid#883 updates in place against base kunchenguid/no-mistakes:main.

The maintainer's binding request was: rebase/resolve onto latest main, close known-gap (1) fail-closed, preferably also (2) so the TUI has the same exit as the CLI, then ping. He will not rebase or resolve conflicts himself while the safety gaps remain. The PR body's own "Known gaps" list (items 1-5) is our prior review's list and is what he was quoting.

WHAT THIS ROUND DID.

Rebase: fm/nm-824-custody-release was rebased onto upstream/main (13cdf1a). Only AGENTS.md conflicted, because main extracted that section into .agents/skills/; the resolution takes main's AGENTS.md verbatim (it is now byte-identical to main's) and re-homes our prose into .agents/skills/branch-sync-and-push-safety/SKILL.md, the file that now owns it. No upstream content was dropped to win a conflict.

Gap (1), the merge blocker. In internal/branchsync/sync.go, recoverKeepLocal wrote refs/no-mistakes/recover-gate/ BEFORE four refusals that each reported "no files or refs were changed". That ref is exactly what wedges every later attempt (settlementAnchorsFree stops advertising the settlement once it names a commit the gate has moved off, PreserveRecoveryAnchor then refuses to retarget it, and nothing retires it), so those refusals could recreate the condition this settlement exists to clear while claiming to have changed nothing. DECISION: fixed by reordering the write, not by cleaning up on the refusal path. Rationale: the anchor guards exactly one operation, the compare-and-swap that moves the gate branch off the displaced head, and nothing before that swap can strand the head because refs/heads/ still names it. So the anchor write moved to immediately before the CAS, which makes the no-change claim true BY CONSTRUCTION rather than by a cleanup that could itself fail and leave the same false claim. The read-only anchor CONFLICT check deliberately stays first, since it is the cheapest refusal and must not be reached only after a staging ref exists.

Counts, which MUST be stated explicitly in the PR body because the maintainer read our earlier "four" as an admission: at the pre-fix tip, recoverKeepLocal had 8 refusal returns, 5 of them in a post-anchor-write window, and 4 of those 5 made the untrue "no files or refs were changed" claim. The 5th (the lost compare-and-swap) says "no LOCAL files or refs were changed", names the anchor, and prescribes reconciling it, so it was already honest. Both our "four" and an auditor's "eight/five" were correct about different things; the PR body must say so rather than leave it reading as an error.

The honest form of the post-fix claim, which must NOT be compressed back into "exactly one": there is now one post-write REFUSAL by design (the lost CAS, which deliberately keeps its anchor because that swap failed because the gate moved, so the pin may be the only ref still naming the displaced head) PLUS one post-success FAILURE (the custody stamp, below). The earlier "exactly one post-write window" phrasing was scoped to refusals without saying so.

Gap (2), the TUI dead end. recoverableBranchSync keyed on safety blocked_pipeline_owned_recoverable while a self-inconsistent record carries the settlement next action, so the TUI was the one operator surface with no exit. It now reaches the same settlement the CLI offers. DECISION: the new predicate keys on the advertised next_action.code return_custody_keep_local rather than on a safety code, deliberately, so it cannot drift from the branchsync predicate that decides where the settlement can actually complete; the service never advertises that action for a record the settlement would only refuse. It gets its own confirmation box rather than reusing the recovery one, because settling KEEPS the local head where recovery TAKES the preserved head, and the CLI makes that an explicit --keep-local choice.

Gap (3). recoverSettleInconsistent's pin loop used a probe that read any non-zero git exit as "absent", in the one place the settlement's data-safety argument depends on that distinction. This was NOT cosmetic: verified that with the old probe the settlement COMPLETED (custody_returned, Recovered:true) without ever proving the recorded head absent. New git.CommitPresence treats ONLY git's exit 1 (the store was read and the object is not there) as an absence. Two shapes now fail closed instead of being read as absent: an unreadable store, and a present-but-wrong-type object, since git cat-file -e exits 0 for a tree, blob or tag and collapsing that into "absent" would assert proven absence about an object that is right there. settlementAnchorsFree mirrors the same probe so the advertisement agrees with the write and a record that can only refuse is never offered the settlement. KNOWN TRADEOFF, deliberate and worth stating in the PR body: a recorded head that git cannot resolve at all now fails closed to manual reconciliation rather than settling, because absence cannot be proven for it.

Additional fix found this round, not in the published list. finishRecover's stamp failure runs AFTER the Git side already succeeded - a keep-local settlement has moved the gate branch and written the recovery anchor - and it explicitly set NextAction to nil, so a failed database write left refs changed and named no exit at all. That is the kunchenguid#824 shape one layer down, and it had zero test coverage. It now reports that the recovery's Git changes are already applied (instead of implying nothing happened) and names the SAME recovery command again via recoveryRetryAction. The PR body should say why that is legitimate in R5 terms: naming the same command is only honest because every Git step it repeats is idempotent once applied - the gate now equals the kept head so recoverKeepLocal skips its whole move, and an already-advanced worktree takes the equal/ahead path - and the tests assert the prescribed retry ACTUALLY COMPLETES rather than merely that a next action exists.

Gap (4). The --keep-local flag help claimed unconditionally that the gate compare-and-swaps onto the kept head, but the equal/ahead, user_owned, already-recovered and absent-gate-branch paths return before recoverKeepLocal is reached. Help and docs now say the CAS happens where the gate branch still names a different head, and that custody otherwise returns without changing any ref.

Gap (5). The two --run abort tests had dropped the two fixture-invariant assertions their bare-abort sibling carries. This was worse than the PR body said: a --run abort response carries no branch_sync object at all, so those tests could not have failed even with the StatePipelineOwned clause deleted - verified by deleting it and watching them still pass. They now assert the fixture's classification through the surface that does report it, and were verified to fail when that clause is removed.

TEST DISCIPLINE USED. Every fix was proven by reverting it and watching the new test fail, then restoring it: the four (now five) recover-gate refusal tests, the undetermined-absence test, the present-but-wrong-type test, the two stamp-failure tests, and the two --run abort tests. Two narrow service seams were added following the file's existing seam convention (absPathFn, stampCustodyReturnedFn) because filepath.Abs failure and the post-success database failure cannot otherwise be induced portably - chmod is a no-op on Windows and this package runs on the Windows CI leg.

CONSTRAINTS AND EXCLUSIONS THAT STILL BIND.

Do NOT implement the structural blockedPlan change offered in the PR body's "Structural note" (inverting the nil-NextAction default). It is a separate, much larger change the maintainer did not ask for; mentioning it in the body is fine, implementing it here is out of scope.

Preserve every behavior the maintainer already accepted: default --recover behavior unchanged, compare-and-swap and never a blind force for gate moves, and fail-closed refusal whenever unique unpublished pipeline commits cannot be anchored. This settlement is a shortcut for self-inconsistent records only, never for records protecting unique unpublished content.

Do not re-litigate which refusal sites are "in theme"; the maintainer's comment and the five published gaps are the scope.

The failing "PR must be raised via no-mistakes" check is expected and is NOT ours to fix - the maintainer stated it is ignored when that check is not in the main ruleset, and live HEAD mergeability governs.

DELIVERABLE STATE. The PR body's "Known gaps" section must be rewritten so it no longer advertises defects that are now fixed, and must carry the explicit counts and the one-post-write-refusal-plus-one-post-success-failure accounting described above. make lint, go test -race ./..., and make e2e were all green locally before this run.

What Changed

  • --recover --keep-local now settles a self-inconsistent custody record — a terminal run whose recorded pipeline head can no longer be verified — instead of leaving the branch with no command that can complete. branchsync advertises next_action.code: return_custody_keep_local only where the settlement can actually finish, pins any surviving copy of the recorded head under refs/no-mistakes/recover-stranded/<run> before moving anything, and moves the gate branch by the same compare-and-swap, so a concurrent gate push still wins. New git.CommitPresence backs both the advertisement and the write with one probe that treats only cat-file -e exit 1 as a proven absence, so an unreadable object store or a present-but-wrong-type object fails closed to manual reconciliation rather than being read as "absent".
  • Keep-local refusals no longer leave behind the ref they claim not to have written: refs/no-mistakes/recover-gate/<run> is now written immediately before the compare-and-swap it guards. At the pre-fix tip recoverKeepLocal had 8 refusal returns, 5 of them after the anchor write, and 4 of those 5 asserted "no files or refs were changed" while having written that ref; those 4 now precede the write, making the claim true by construction. What remains is one post-write refusal by design — the lost compare-and-swap, which deliberately keeps its anchor because it may be the only ref still naming the displaced head, and which reports the narrower "no local files or refs" claim and names the ref plus the gate directory — plus one post-success failure, the custody stamp, which now reports that the recovery's Git changes are already applied and re-names the same recovery command (idempotent once applied) instead of returning a nil next action.
  • The TUI reaches the same exit as the CLI: u offers "settle custody at local head" with its own confirmation naming the kept head and the unverifiable recorded head, keyed on the advertised return_custody_keep_local action rather than a safety code so it cannot drift from the service predicate. axi abort on an already-terminal run keeps its no-op but now names the settlement command (allowlisted custody actions only, never a fresh-run launch), and the --keep-local flag help, CLI/TUI/agent docs, and published agent guidance now say the gate compare-and-swap happens only where the gate branch still names a different head.

Risk Assessment

⚠️ Medium: A large, data-safety-critical change that lets a previously always-refusing custody record move the gate branch and stamp custody, but every settlement path is fail-closed, pins any surviving copy of the recorded head before anything moves, keeps the compare-and-swap, and is backed by behavioral regressions - leaving only an info-level message-accuracy nit.

Testing

Ran the targeted unit tests covering all five published gaps (branchsync settlement/keep-local refusals/stamp failure, the TUI exit, the fail-closed presence probe, the flag help, and the abort fixture invariants) plus full package runs of internal/branchsync, internal/cli, internal/tui and internal/git — all green. Because nothing exercised the settlement through the command surface an operator actually types, I added a focused CLI journey test that builds the #824 record on real Git repos and the real state database and walks the whole road, asserting persisted state rather than just output: worktree HEAD untouched, gate branch compare-and-swapped onto the kept head, displaced gate head still anchored at refs/no-mistakes/recover-gate/<run>, and custody_returned_at written. I proved the regression both ways by swapping internal/branchsync/sync.go back to upstream main (13cdf1a) and watching the journey fail with every surface pointing at inspect_and_reconcile_manually, then restoring it and confirming a clean worktree. For the TUI I captured the real renderer's ANSI output on both revisions and turned it into reviewer-visible before/after screenshots, showing u going from bound-to-nothing to a settlement confirmation distinct from the recovery one. The only worktree change left behind is the new test file; temporary capture harnesses were deleted.

  • Evidence: Before/after evidence page — CLI transcript + TUI renders (screenshot) (local file: /Users/huntermcdaniel/.no-mistakes/evidence/01M15F2V70W27JBB75S43SRN87/settlement-evidence.png)
  • Evidence: TUI gap (2) before/after — wedged panel and settlement confirmation (screenshot) (local file: /Users/huntermcdaniel/.no-mistakes/evidence/01M15F2V70W27JBB75S43SRN87/tui-settlement.png)
Evidence: Full CLI transcript, upstream main vs this branch, identical fixture
# `no-mistakes` CLI transcript - issue #824 wedged custody record

Fixture (built by `wedgedCustodyAbortFixture`, real Git + real state DB):

- terminal run, status `failed`, no push binding
- recorded pipeline head `aaaa...aaaa` exists in **no** object store
- local gate branch `feature/wedged` sits at a **later** run's head, so it names neither the recorded head nor the operator's head

Both columns run the identical fixture through the real cobra command tree.

## BEFORE - upstream main (13cdf1a)

\### STEP 1 - `no-mistakes axi sync --check` on the wedged record

`` `
branch_sync:
  state: pipeline_owned
  changed: false
  local:
    branch: feature/wedged
    head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    clean: true
  pipeline:
    run: "01M15M46HYDVZHJ5BPCY0T4H95"
    status: failed
    phase: pre_push
    submitted_head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: ""
    pushed_at: 0
    push_generation: 0
  target:
    kind: ""
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI531601777/001/remote.git
    ref: ""
  remote:
    observed_head: ""
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: blocked_recover_preserved_head_missing
  pr_state: none
  note: the run finished failed but its recorded pipeline head is not available in the invoking worktree or local gate; inspect and reconcile the recorded and live heads manually
  next_action:
    code: inspect_and_reconcile_manually
    command: no-mistakes axi status
error: the run finished failed but its recorded pipeline head is not available in the invoking worktree or local gate; inspect and reconcile the recorded and live heads manually
help[1]: Run `no-mistakes axi status`
`` `

\### STEP 2 - `no-mistakes axi sync --recover` (no --keep-local) still refuses

`` `
branch_sync:
  state: pipeline_owned
  changed: false
  local:
    branch: feature/wedged
    head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    clean: true
  pipeline:
    run: "01M15M46HYDVZHJ5BPCY0T4H95"
    status: failed
    phase: pre_push
    submitted_head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: ""
    pushed_at: 0
    push_generation: 0
  target:
    kind: ""
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI531601777/001/remote.git
    ref: ""
  remote:
    observed_head: ""
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: blocked_recover_preserved_head_missing
  pr_state: none
  note: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed
error: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed
`` `

\### STEP 3 - the advertised `no-mistakes axi sync --recover --keep-local` completes

`` `
branch_sync:
  state: pipeline_owned
  changed: false
  local:
    branch: feature/wedged
    head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    clean: true
  pipeline:
    run: "01M15M46HYDVZHJ5BPCY0T4H95"
    status: failed
    phase: pre_push
    submitted_head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: ""
    pushed_at: 0
    push_generation: 0
  target:
    kind: ""
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI531601777/001/remote.git
    ref: ""
  remote:
    observed_head: ""
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: blocked_recover_preserved_head_missing
  pr_state: none
  note: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed
error: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed
`` `

> Every surface loops back to `next_action.code: inspect_and_reconcile_manually` -> `no-mistakes axi status`, which re-reports the same block. There is no command that ends the state, and STEP 3 (the settlement) refuses: the branch stays `pipeline_owned` forever.

## AFTER - this branch (711ca05)

\### STEP 1 - `no-mistakes axi sync --check` on the wedged record

`` `
branch_sync:
  state: pipeline_owned
  changed: false
  local:
    branch: feature/wedged
    head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    clean: true
  pipeline:
    run: "01M15M4E093MA96MDMP0MJXSX5"
    status: failed
    phase: pre_push
    submitted_head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: ""
    pushed_at: 0
    push_generation: 0
  target:
    kind: ""
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI759232991/001/remote.git
    ref: ""
  remote:
    observed_head: ""
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: blocked_recover_preserved_head_missing
  pr_state: none
  note: "the run finished failed but its recorded pipeline head cannot be verified in the invoking worktree or local gate; return custody at the current local head, which also points the gate branch at it"
  next_action:
    code: return_custody_keep_local
    command: no-mistakes axi sync --recover --keep-local
error: "the run finished failed but its recorded pipeline head cannot be verified in the invoking worktree or local gate; return custody at the current local head, which also points the gate branch at it"
help[1]: Run `no-mistakes axi sync --recover --keep-local`
`` `

\### STEP 2 - `no-mistakes axi sync --recover` (no --keep-local) still refuses

`` `
branch_sync:
  state: pipeline_owned
  changed: false
  local:
    branch: feature/wedged
    head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    clean: true
  pipeline:
    run: "01M15M4E093MA96MDMP0MJXSX5"
    status: failed
    phase: pre_push
    submitted_head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: ""
    pushed_at: 0
    push_generation: 0
  target:
    kind: ""
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI759232991/001/remote.git
    ref: ""
  remote:
    observed_head: ""
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: blocked_recover_preserved_head_missing
  pr_state: none
  note: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed
error: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed
`` `

\### STEP 3 - the advertised `no-mistakes axi sync --recover --keep-local` completes

`` `
branch_sync:
  state: custody_returned
  changed: false
  recovered: true
  local:
    branch: feature/wedged
    head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    clean: true
  pipeline:
    run: "01M15M4E093MA96MDMP0MJXSX5"
    status: failed
    phase: ""
    submitted_head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: ""
    pushed_at: 0
    push_generation: 0
  target:
    kind: ""
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI759232991/001/remote.git
    ref: ""
  remote:
    observed_head: ""
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: custody_returned
  pr_state: none
  next_action:
    code: run_pipeline
    command: "no-mistakes axi run --intent \"<what the user set out to accomplish>\""
help[1]: "Run `no-mistakes axi run --intent \"<what the user set out to accomplish>\"`"
`` `

\### STEP 4 - `no-mistakes axi sync --check` after settling

`` `
branch_sync:
  state: custody_returned
  changed: false
  local:
    branch: feature/wedged
    head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    clean: true
  pipeline:
    run: "01M15M4E093MA96MDMP0MJXSX5"
    status: failed
    phase: ""
    submitted_head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: ""
    pushed_at: 0
    push_generation: 0
  target:
    kind: ""
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI759232991/001/remote.git
    ref: ""
  remote:
    observed_head: ""
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: custody_returned
  pr_state: none
  next_action:
    code: run_pipeline
    command: "no-mistakes axi run --intent \"<what the user set out to accomplish>\""
help[1]: "Run `no-mistakes axi run --intent \"<what the user set out to accomplish>\"`"
`` `

> The plan now advertises `return_custody_keep_local`, the plain `--recover` still refuses (the settlement is never silently taken without `--keep-local`), the advertised command completes with `state: custody_returned`, and a fresh read confirms the branch is released.

Verified persisted state after STEP 3 (asserted in the test, not just printed):

- worktree `HEAD` unchanged (settlement never touches the worktree)
- gate `refs/heads/feature/wedged` compare-and-swapped onto the kept local head
- displaced gate head still reachable at `refs/no-mistakes/recover-gate/<run>`
- `runs.custody_returned_at` recorded in the state database
Evidence: Rendered evidence page (HTML)
<!doctype html><html lang="en"><head><meta charset="utf-8">
<title>no-mistakes #824 - custody settlement evidence</title>
<style>
 :root { color-scheme: dark; }
 * { box-sizing:border-box; }
 body { margin:0; background:#0d1117; color:#c9d1d9; font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif; }
 .wrap { max-width:1500px; margin:0 auto; padding:34px 26px 70px; }
 h1 { font-size:23px; margin:0 0 6px; letter-spacing:-.01em; }
 .sub { color:#8b949e; font-size:13px; line-height:1.6; margin:0 0 6px; max-width:1000px; }
 h2 { font-size:13px; text-transform:uppercase; letter-spacing:.1em; color:#8b949e; margin:38px 0 14px; border-bottom:1px solid #21262d; padding-bottom:9px; }
 h3 { font-size:12.5px; color:#c9d1d9; margin:0 0 8px; font-weight:600; font-family:"SF Mono",Menlo,Consolas,monospace; }
 .grid { display:grid; grid-template-columns:minmax(0,1fr) minmax(0,1fr); gap:20px; align-items:start; }
 .card { background:#010409; border:1px solid #21262d; border-radius:10px; overflow:hidden; min-width:0; }
 .card > header { padding:9px 14px; font-size:11.5px; font-weight:700; letter-spacing:.05em; border-bottom:1px solid #21262d; }
 .bad > header { background:#3d1418; color:#ffb0b0; }
 .good > header { background:#0d2b1a; color:#82e5a8; }
 .card section { padding:12px 14px; border-top:1px solid #161b22; }
 .card section:first-of-type { border-top:0; }
 pre { margin:0; font-family:"SF Mono",Menlo,Consolas,monospace; font-size:11px; line-height:1.5; color:#c9d1d9; white-space:pre-wrap; overflow-wrap:anywhere; }
 .note { font-size:12.5px; color:#8b949e; line-height:1.65; margin:14px 0 0; max-width:1100px; }
 code { background:#161b22; padding:1px 5px; border-radius:4px; font-size:11.5px; font-family:"SF Mono",Menlo,Consolas,monospace; }
 .hl { color:#82e5a8; } .hlbad { color:#ffb0b0; }
 ul { font-size:12.5px; color:#8b949e; line-height:1.8; margin:12px 0 0; padding-left:20px; max-width:1100px; }
</style></head><body><div class="wrap">

<h1>issue #824 &mdash; a wedged custody record now has a working exit on every operator surface</h1>
<p class="sub">Fixture, built on real Git repositories and the real state database: a terminal run (<code>failed</code>) whose recorded pipeline head <code>aaaa&hellip;aaaa</code> exists in <strong>no</strong> object store, with the local gate branch sitting at a <em>later</em> run's head. Left column is upstream main (13cdf1a); right column is this branch (711ca05). Identical fixture, identical commands.</p>
<p class="sub">Terminal renders come from the real TUI renderer; CLI blocks are the verbatim output of the real cobra command tree. Long lines are soft-wrapped here for readability &mdash; the app itself does not wrap them (same as the pre-existing recovery box).</p>

<h2>1 &middot; CLI &mdash; the agent surface (gaps 1 &amp; 3)</h2>
<div class="grid">
  <div class="card bad"><header>BEFORE &mdash; upstream main (13cdf1a)</header>
    <section><h3>$ no-mistakes axi sync --check</h3><pre class="">branch_sync:
  state: pipeline_owned
  changed: false
  local:
    branch: feature/wedged
    head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    clean: true
  pipeline:
    run: &quot;01M15M46HYDVZHJ5BPCY0T4H95&quot;
    status: failed
    phase: pre_push
    submitted_head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: &quot;&quot;
    pushed_at: 0
    push_generation: 0
  target:
    kind: &quot;&quot;
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI531601777/001/remote.git
    ref: &quot;&quot;
  remote:
    observed_head: &quot;&quot;
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: blocked_recover_preserved_head_missing
  pr_state: none
  note: the run finished failed but its recorded pipeline head is not available in the invoking worktree or local gate; inspect and reconcile the recorded and live heads manually
  next_action:
    code: inspect_and_reconcile_manually
    command: no-mistakes axi status
error: the run finished failed but its recorded pipeline head is not available in the invoking worktree or local gate; inspect and reconcile the recorded and live heads manually
help[1]: Run `no-mistakes axi status`</pre></section>
    <section><h3>$ no-mistakes axi sync --recover --keep-local</h3><pre class="">branch_sync:
  state: pipeline_owned
  changed: false
  local:
    branch: feature/wedged
    head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    clean: true
  pipeline:
    run: &quot;01M15M46HYDVZHJ5BPCY0T4H95&quot;
    status: failed
    phase: pre_push
    submitted_head: 3fe29c6dd58bba0320ba82bdc8edeb10754cc6ca
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: &quot;&quot;
    pushed_at: 0
    push_generation: 0
  target:
    kind: &quot;&quot;
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI531601777/001/remote.git
    ref: &quot;&quot;
  remote:
    observed_head: &quot;&quot;
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: blocked_recover_preserved_head_missing
  pr_state: none
  note: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed
error: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed</pre></section>
  </div>
  <div class="card good"><header>AFTER &mdash; this branch (711ca05)</header>
    <section><h3>$ no-mistakes axi sync --check</h3><pre class="">branch_sync:
  state: pipeline_owned
  changed: false
  local:
    branch: feature/wedged
    head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    clean: true
  pipeline:
    run: &quot;01M15M4E093MA96MDMP0MJXSX5&quot;
    status: failed
    phase: pre_push
    submitted_head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: &quot;&quot;
    pushed_at: 0
    push_generation: 0
  target:
    kind: &quot;&quot;
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI759232991/001/remote.git
    ref: &quot;&quot;
  remote:
    observed_head: &quot;&quot;
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: blocked_recover_preserved_head_missing
  pr_state: none
  note: &quot;the run finished failed but its recorded pipeline head cannot be verified in the invoking worktree or local gate; return custody at the current local head, which also points the gate branch at it&quot;
  next_action:
    code: return_custody_keep_local
    command: no-mistakes axi sync --recover --keep-local
error: &quot;the run finished failed but its recorded pipeline head cannot be verified in the invoking worktree or local gate; return custody at the current local head, which also points the gate branch at it&quot;
help[1]: Run `no-mistakes axi sync --recover --keep-local`</pre></section>
    <section><h3>$ no-mistakes axi sync --recover --keep-local</h3><pre class="">branch_sync:
  state: custody_returned
  changed: false
  recovered: true
  local:
    branch: feature/wedged
    head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    clean: true
  pipeline:
    run: &quot;01M15M4E093MA96MDMP0MJXSX5&quot;
    status: failed
    phase: &quot;&quot;
    submitted_head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: &quot;&quot;
    pushed_at: 0
    push_generation: 0
  target:
    kind: &quot;&quot;
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI759232991/001/remote.git
    ref: &quot;&quot;
  remote:
    observed_head: &quot;&quot;
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: custody_returned
  pr_state: none
  next_action:
    code: run_pipeline
    command

... [1074 bytes truncated] ...

ged
    head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    clean: true
  pipeline:
    run: &quot;01M15M4E093MA96MDMP0MJXSX5&quot;
    status: failed
    phase: pre_push
    submitted_head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: &quot;&quot;
    pushed_at: 0
    push_generation: 0
  target:
    kind: &quot;&quot;
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI759232991/001/remote.git
    ref: &quot;&quot;
  remote:
    observed_head: &quot;&quot;
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: blocked_recover_preserved_head_missing
  pr_state: none
  note: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed
error: the recorded pipeline head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa is missing from the local gate; inspect the recorded and live heads before returning custody; no files or refs were changed</pre></section>
  </div>
  <div class="card good"><header>AFTER &mdash; fresh read confirms the branch is released</header>
    <section><h3>$ no-mistakes axi sync --check</h3><pre class="">branch_sync:
  state: custody_returned
  changed: false
  local:
    branch: feature/wedged
    head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    clean: true
  pipeline:
    run: &quot;01M15M4E093MA96MDMP0MJXSX5&quot;
    status: failed
    phase: &quot;&quot;
    submitted_head: cd46711c8c1618ee6c5249fc9af38382159fcd35
    current_head: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    pushed_head: &quot;&quot;
    pushed_at: 0
    push_generation: 0
  target:
    kind: &quot;&quot;
    remote: origin
    url: /var/folders/gf/nh6xqm0x1nggw7d8gnh9j8gr0000gn/T/TestWedgedCustodyRecordSettlesThroughTheCLI759232991/001/remote.git
    ref: &quot;&quot;
  remote:
    observed_head: &quot;&quot;
    freshness: pipeline_push
    observed_at: 0
  relation: unknown
  safety: custody_returned
  pr_state: none
  next_action:
    code: run_pipeline
    command: &quot;no-mistakes axi run --intent \&quot;&lt;what the user set out to accomplish&gt;\&quot;&quot;
help[1]: &quot;Run `no-mistakes axi run --intent \&quot;&lt;what the user set out to accomplish&gt;\&quot;`&quot;</pre></section>
  </div>
</div>
<ul>
  <li>Worktree <code>HEAD</code> is unchanged by the settlement (asserted against live Git).</li>
  <li>Gate <code>refs/heads/feature/wedged</code> is compare-and-swapped onto the kept local head.</li>
  <li>The displaced gate head is still reachable at <code>refs/no-mistakes/recover-gate/&lt;run&gt;</code> &mdash; nothing is dropped.</li>
  <li><code>runs.custody_returned_at</code> is recorded in the state database.</li>
</ul>

<h2>3 &middot; TUI &mdash; the surface that had no exit at all (gap 2)</h2>
<div class="grid">
  <div class="card bad"><header>BEFORE &mdash; upstream main (13cdf1a)</header><section><pre>=== branch status panel (wedged custody record) ===
<span style="color:#5c6370">╭─ </span><span style="color:#56b6c2;font-weight:600">Local branch</span> <span style="color:#5c6370">───────────────────────────────────────────────────────────────────────────────╮</span>
<span style="color:#5c6370">│</span> Local branch unchanged; the pipeline fix is not pushed yet. Do not make follow-up commits.   <span style="color:#5c6370">│</span>
<span style="color:#5c6370">╰──────────────────────────────────────────────────────────────────────────────────────────────╯</span>

=== after pressing `u` ===
(nothing happened: `u` is not bound for this state - no exit offered)
</pre></section></div>
  <div class="card good"><header>AFTER &mdash; this branch (711ca05)</header><section><pre>=== branch status panel (wedged custody record) ===
<span style="color:#5c6370">╭─ </span><span style="color:#56b6c2;font-weight:600">Local branch</span> <span style="color:#5c6370">───────────────────────────────────────────────────────────────────────────────╮</span>
<span style="color:#5c6370">│</span> Run ended terminally and its recorded pipeline head can no longer be verified, so there is nothing to recover. Settle custody at your current head to take the branch back. <span style="color:#5c6370">│</span>
<span style="color:#5c6370">╰──── </span><span style="color:#5c6370">u settle custody at local head</span> <span style="color:#5c6370">──────────────────────────────────────────────────────────╯</span>

=== after pressing `u` : settlement confirmation ===
<span style="color:#5c6370">╭─ </span><span style="color:#56b6c2;font-weight:600">Confirm custody settlement at local head</span> <span style="color:#5c6370">───────────────────────────────────────────────────╮</span>
<span style="color:#5c6370">│</span> The run ended failed and its recorded pipeline head can no longer be verified,               <span style="color:#5c6370">│</span>
<span style="color:#5c6370">│</span> so there is nothing to recover. Settling returns custody at the head you                     <span style="color:#5c6370">│</span>
<span style="color:#5c6370">│</span> already have and points the gate branch at it.                                               <span style="color:#5c6370">│</span>
<span style="color:#5c6370">│</span>                                                                                              <span style="color:#5c6370">│</span>
<span style="color:#5c6370">│</span> Local branch:   feature/wedged                                                               <span style="color:#5c6370">│</span>
<span style="color:#5c6370">│</span> Kept HEAD:      a58cf22e873e83b5f1c5bf7d89dc5694b3f062ef                                     <span style="color:#5c6370">│</span>
<span style="color:#5c6370">│</span> Recorded HEAD:  aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa (unverifiable)                      <span style="color:#5c6370">│</span>
<span style="color:#5c6370">│</span>                                                                                              <span style="color:#5c6370">│</span>
<span style="color:#5c6370">│</span> Your worktree is never touched. Any still-reachable copy of the recorded head is anchored first, and the settlement refuses rather than proceeding if one exists and cannot be anchored. The gate moves only by compare-and-swap, so a concurrent gate push wins. This is `no-mistakes sync --recover --keep-local`. <span style="color:#5c6370">│</span>
<span style="color:#5c6370">╰──── </span><span style="color:#5c6370">u/enter settle  ·  esc cancel</span> <span style="color:#5c6370">───────────────────────────────────────────────────────────╯</span>
</pre></section></div>
</div>
<p class="note">Before, the wedged record fell through to the generic &ldquo;Local branch unchanged&rdquo; copy and <code>u</code> was bound to nothing &mdash; the TUI was the one surface with no exit. After, the panel names the condition, offers <code>u settle custody at local head</code>, and <code>u</code> opens a settlement confirmation that is deliberately <em>not</em> the recovery one: it states which head is kept and which is abandoned before asking.</p>
</div></body></html>
  • Evidence: TUI evidence page (HTML) (local file: /Users/huntermcdaniel/.no-mistakes/evidence/01M15F2V70W27JBB75S43SRN87/tui-settlement.html)
Evidence: Key CLI diff: the advertised exit now completes
BEFORE (upstream main 13cdf1a) — $ no-mistakes axi sync --check
safety: blocked_recover_preserved_head_missing
next_action:
code: inspect_and_reconcile_manually
command: no-mistakes axi status
$ no-mistakes axi sync --recover --keep-local
state: pipeline_owned
error: the recorded pipeline head aaaa...aaaa is missing from the local gate; ... no files or refs were changed

AFTER (this branch 711ca05) — $ no-mistakes axi sync --check
safety: blocked_recover_preserved_head_missing
next_action:
code: return_custody_keep_local
command: no-mistakes axi sync --recover --keep-local
$ no-mistakes axi sync --recover --keep-local
state: custody_returned
recovered: true
safety: custody_returned
Evidence: Rendered --keep-local help on both CLI surfaces (gap 4)

--keep-local with --recover: keep the current local head; the worktree is never touched, surviving preserved commits stay anchored, and where the gate branch still names a different head it compare-and-swaps onto the kept head; otherwise custody returns without moving any branch ref. Also settles a record whose preserved head can no longer be verified

$ no-mistakes sync --help
Refreshes the current branch's persisted pipeline push binding and, after
confirmation, advances only a completely clean checked-out branch using one of
two guarded modes: a strict fast-forward for clean behind branches, or an
equivalent-diverged advance that first anchors the old head and then moves the
branch to the verified pipeline head with reset semantics. It never stashes,
merges genuine divergence, rebases, switches branches, or updates a remote.
--check performs the fresh proof without applying it.
--recover returns custody of a branch whose run went terminal with unpublished
pipeline commits: it anchors the preserved head, then either fast-forwards a
clean behind worktree or adopts a diverged preserved head only when proven to
carry every local change. Unproven divergence refuses. A run cancelled before
the pipeline changed anything releases the branch by itself (user_owned) and
makes --recover a no-op. --recover --keep-local keeps the current local head
instead and never touches the worktree; where the gate branch still names a
different head it is compare-and-swapped onto the kept head, and where it does
not, custody returns without moving any branch ref. That is also the settlement
for a record whose preserved head can no longer be verified.

Usage:
  no-mistakes sync [flags]

Flags:
      --check        freshly verify and show the synchronization plan without changing HEAD
  -h, --help         help for sync
      --keep-local   with --recover: keep the current local head; the worktree is never touched, surviving preserved commits stay anchored, and where the gate branch still names a different head it compare-and-swaps onto the kept head. Also settles a record whose preserved head can no longer be verified
      --recover      return custody of a branch stranded by a terminal run with unpublished pipeline commits (a no-op when cancellation already released the branch)
  -y, --yes          apply an eligible guarded synchronization without prompting


$ no-mistakes axi sync --help
Verifies the registered invoking worktree, clean exact branch, persisted
pipeline push binding, configured fork or upstream target, live remote equality,
and either strict ancestry or content-equivalent divergence. The default applies
an eligible plan without a prompt: strict fast-forward for behind branches, or an
equivalent advance that anchors the old head before moving the branch to the
verified pipeline head with reset semantics.
--check performs the same fresh read-only plan. Blocked states change nothing.
--recover performs the guarded custody return offered by
next_action.code: recover_custody; --keep-local keeps the current local head
and, where the gate branch still names a different head, moves the gate branch
to it, which is also the settlement offered by
next_action.code: return_custody_keep_local.

Usage:
  no-mistakes axi sync [flags]

Flags:
      --check        freshly verify and return the plan without changing HEAD
  -h, --help         help for sync
      --keep-local   with --recover: keep the current local head; the worktree is never touched, surviving preserved commits stay anchored, and where the gate branch still names a different head it compare-and-swaps onto the kept head; otherwise custody returns without moving any branch ref. Also settles a record whose preserved head can no longer be verified
      --recover      return custody of a branch stranded by a terminal run with unpublished pipeline commits (a no-op when cancellation already released the branch)
  • Evidence: Raw go test -v logs backing the transcript (pre-change / post-change) (local file: /Users/huntermcdaniel/.no-mistakes/evidence/01M15F2V70W27JBB75S43SRN87/raw-before.txt)
  • Outcome: ⚠️ 1 info across 1 run (14m5s)

Pipeline

Updates from git push no-mistakes

... (5 earlier update rounds omitted to keep the PR body within GitHub's 65536-char limit; full history is in the run log.)

⚠️ **Review** - 1 info

Narrowest fix: thread the pinned/strandedRef pair into recoverKeepLocal so its refusals append the same anchoredElsewhere suffix the settlement already builds; the write ordering, the CAS, and the advertisement predicate all stay as they are. This is user-visible refusal text and challenges the author's explicit accounting, so it is your call whether to correct the messages or narrow the invariant claims instead.

  • ℹ️ docs/src/content/docs/reference/cli.md:239 - The gap-(4) sentence added this round overstates one of the three no-op paths it enumerates: "...and on the paths that return before the gate is reached at all, such as a released user_owned branch, an already-returned custody record, and a local head that already contains the preserved one - custody returns without changing any ref."

The first two are exact - Recover returns at sync.go:590 and :598 before touching anything. The third is not. "A local head that already contains the preserved one" is the branch at sync.go:651, which calls anchorReachablePreserved (sync.go:1192), and that unconditionally calls custody.PreserveRecoveryHead on the invoking worktree, creating refs/no-mistakes/recover/&lt;run&gt; there. Terminalization pins that ref in the GATE, not in the operator's worktree, so on a first --keep-local for this shape the ref is genuinely created rather than found. A ref is changed.

The same wording appears one clause earlier for "where the gate branch already names the kept head": reaching recoverKeepLocal with gateHead == state.Local.Head via the behind/diverged switch means FetchRemoteRef plus PreserveRecoveryAnchor already ran on the worktree, and via the settlement means the stranded pin may already exist.

The flag help in internal/cli/sync.go is fine ("surviving preserved commits stay anchored" acknowledges it); only this reference sentence claims no ref at all. The intended meaning is clearly "no BRANCH ref moves", which the surrounding paragraph says correctly elsewhere - so the narrowest fix is to say "without moving any branch ref" here. Flagging rather than fixing because it is published product wording in the exact class of claim this round is auditing.

🔧 Fix: make keep-local refusals name the anchor they wrote
2 issues (1 error, 1 warning) still open:

  • 🚨 internal/cli/axi_abort_custody_test.go:440 - assertDivergedReleasedFixtureInvariants reads the fixture's classification by running the BARE axi abort, and its doc comment requires that it be called "before any fake daemon is started". But the bare abort path is the one abort path that requires a live daemon: runAxiAbort (internal/cli/axi_drive.go:947) calls openAxiDaemonEnvopenAxiEnvWithOptions{ensureDaemonConn: true}daemon.EnsureDaemon(p).

Concrete sequence: wedgedCustodyAbortFixture sets NM_HOME to a fresh temp dir and never starts a daemon, so daemonIsRunningViaIPC stats a missing socket and returns (false, nil); EnsureDaemon then calls daemonStartStart → (managed service bypassed under go test, internal/daemon/service.go:270) → startDetachedDaemon, which re-execs the TEST BINARY as &lt;testbin&gt; daemon run --root &lt;NM_HOME&gt;. That child hits helpers_test.go:60, sees NM_TEST_START_DAEMON unset (only setupTestRepo sets it, helpers_test.go:140; this fixture does not), returns from init(), and falls into TestMain/m.Run() — so it runs the cli test suite instead of serving the socket. waitForDaemonStartWithProcess polls health until the default 45s NM_TEST_DAEMON_START_TIMEOUT, then returns daemon launched but did not become ready. axi abort emits that as an error, executeCmd returns non-nil, and the helper's t.Fatalf(&#34;reading the fixture classification must not fail&#34;) fires.

Both callers are affected: TestRunScopedAbortNoOpEmitsNoHelpForOrdinaryDivergence (line 355) and TestRunScopedAbortNoOpEmitsNoHelpForOrdinaryDivergenceWithDaemon (line 371) stall ~45s each and then fail, and each leaves a detached test-binary child re-running the package.

The two sibling bare-abort tests in this same file start startNoActiveRunDaemon first (lines 217/317), and TestAxiRunReportsInvalidGlobalConfig (internal/cli/axi_test.go:797) clamps NM_TEST_DAEMON_START_TIMEOUT to 100ms specifically because a daemon cannot come up in this package — both confirm the requirement.

Narrowest fix: read the classification without the CLI/daemon at all - open the DB, findRepo, and call branchsync.Service{...}.InspectCached(ctx) directly (the same read terminalRunCustodyHelpWithDB performs), asserting State == StateDiverged/Safety == blocked_diverged and the inspect_and_reconcile_manually next action. That keeps both fixture invariants without needing a daemon, and works for the WithDaemon variant too, where startInactiveAbortDaemon does not even serve MethodGetActiveRun.

  • ⚠️ internal/branchsync/sync.go:1013 - keepLocalNoChangeClause DISCARDS the blanket string whenever an anchor note is present, and it is applied to the lost-compare-and-swap refusal too. That refusal's blanket is &#34;no local files or refs were changed&#34; - deliberately scoped to local because on that path the attempt has just written refs/no-mistakes/recover-gate/&lt;run&gt; in the gate. When the caller supplies an anchor note (the settlement path, where pinned is non-empty), the message becomes &#34;... ; no branch, worktree, or file changes were made; the recorded head is now anchored at refs/no-mistakes/recover-stranded/&lt;run&gt; in ...&#34;, dropping the local qualifier in the one branch where a non-local ref file was written by this very attempt.

Concrete reachable state: conflicting gate recovery anchor, recorded head still present in the gate (the TestRecoverKeepLocalSettlesConflictingGateAnchorAndPinsPreservedHead fixture), gate branch at G with G != local head and G != recorded head. recoverSettleInconsistent pins the stranded ref, delegates with a non-empty note, recoverKeepLocal takes writeGateAnchor = true, writes recover-gate/&lt;run&gt;, and a concurrent gate push then loses the CAS - producing a refusal that names the anchor it wrote and, in the same sentence, claims no file changes were made.

It also contradicts documentation written in the same commit: docs/src/content/docs/reference/cli.md:239 still ends "That refusal names the ref and the gate directory, and reports that no local files or refs were changed", which is now false for the settlement path. No test covers it - wedgedCustodyFixture pins nothing, so anchoredNote is empty in TestRecoverKeepLocalSettlementLosesConcurrentGatePushCleanly.

Narrowest fix: on the two gate-race sites, APPEND the anchor note to the existing local-scoped blanket instead of replacing it (i.e. &#34;no local files or refs were changed&#34; + anchoredNote), leaving keepLocalNoChangeClause for the genuinely pre-write refusals. This is user-visible refusal wording in the exact claim class this round is auditing, so it is your call whether to correct the message or the cli.md sentence.

🔧 Fix: keep lost-swap refusal local-scoped; drop daemon from fixture check
2 issues (1 warning, 1 info) still open:

  • ⚠️ internal/cli/sync.go:97 - Gap (4) is closed on three of four help surfaces but not on the axi sync --keep-local flag help, which still makes the exact unconditional claim the gap was raised to correct.

The intent states gap (4) as: "The --keep-local flag help claimed unconditionally that the gate compare-and-swaps onto the kept head, but the equal/ahead, user_owned, already-recovered and absent-gate-branch paths return before recoverKeepLocal is reached. Help and docs now say the CAS happens where the gate branch still names a different head, and that custody otherwise returns without changing any ref."

The human sync flag help was corrected (sync.go:61): "...and where the gate branch still names a different head it compare-and-swaps onto the kept head". The axi sync Long text was corrected too (sync.go:78-80). But sync.go:97 still reads "surviving preserved commits stay anchored and the gate branch compare-and-swaps onto the kept head" - no conditional.

That is false for every path that returns before recoverKeepLocal: Recover stamps custody via finishRecover on the equal/ahead branch (sync.go:670) without touching the gate at all, recoverSettleInconsistent returns early when the gate branch is proven absent (sync.go:859) or gateDir is empty (sync.go:847), and recoverKeepLocal itself skips its whole block when gateHead == state.Local.Head (sync.go:955). axi sync --help prints the flag line, so an agent reading only that surface gets the overstatement. No test pins any of the four help strings, so nothing catches the drift.

Narrowest fix: bring sync.go:97 in line with sync.go:61 - e.g. "...surviving preserved commits stay anchored, and where the gate branch still names a different head it compare-and-swaps onto the kept head; otherwise custody returns without moving any branch ref".

  • ℹ️ docs/src/content/docs/reference/cli.md:239 - The closing sentence of the --keep-local paragraph attributes to one refusal properties that belong to two different refusals, so on either reading part of it is false. The trailing clause added in the latest fix round (24c96a4) is what forces the ambiguity into an outright contradiction.

The sentence: "...later attempts refuse on the anchor conflict until you reconcile that ref in the gate by hand. That refusal names the ref and the gate directory, and reports that no local files or refs were changed - it keeps that narrower claim precisely because it has just written a ref in the gate..."

The two candidate referents behave differently:

  • The anchor-CONFLICT refusal (sync.go:963) formats gateHead, gateAnchor, and s.GateDir, so it does name the ref and the gate directory - but it closes with keepLocalNoChangeClause(&#34;no files or branch refs were changed&#34;, anchoredNote), never "no local files or refs", and it is a read-only refusal that has written nothing, so "it has just written a ref in the gate" is false of it.
  • The lost-CAS refusal (sync.go:1019) does say "no local files or refs were changed" and has just written the gate anchor - but it formats only custody.RecoveryGateRef(run.ID); s.GateDir never appears in it, so "and the gate directory" is false of it.

The grammatical antecedent is the anchor-conflict refusal, while the new trailing clause describes the lost-CAS refusal. The .agents/skills/branch-sync-and-push-safety/SKILL.md version of the same claim is correct - it says the lost CAS "names the ref to reconcile" with no mention of the gate directory - so only cli.md is drifted. The round-2 decision explicitly asked that cli.md be kept in step with whatever the code ends up saying, which is why this is worth naming rather than leaving.

Narrowest fix: split the two, e.g. "That later anchor-conflict refusal names the ref and the gate directory. The lost compare-and-swap itself reports that no local files or refs were changed and names the anchor to reconcile - it keeps that narrower claim precisely because it has just written a ref in the gate, and where the attempt had already anchored the preserved head it appends the same anchor note rather than dropping the local qualifier."

🔧 Fix: condition axi sync keep-local flag help on gate state
1 info still open:

  • ℹ️ internal/branchsync/sync.go:1253 - finishRecover's stamp-failure branch replaced the old neutral wording with a positive claim - "the recovery's Git changes are already applied" - but three of its callers reach it having applied nothing at all, so on those paths the new message asserts a mutation that never happened.

Concrete reachable state (one step, and the seam to induce it already exists): the wedged custody record with the gate branch deleted. recoverSettleInconsistent's pin loop finds the recorded head in no store, so pinned stays empty and nothing is written; git.ExactRefTarget then reports the gate branch absent and sync.go:860 returns s.finishRecover(ctx, run, false, true) with zero refs, files, or objects touched. If stampCustodyReturned fails there, the operator is told the recovery's Git changes are already applied. TestRecoverKeepLocalSettlementTreatsAnUnreadableGateBranchAsUnknown's "absent gate branch settles without a gate move" subtest builds exactly that fixture, and stampCustodyReturnedFn is the seam that fails the write. The same holds for sync.go:848 (gateDir == &#34;&#34;) and sync.go:1025 (recoverKeepLocal skipping its whole block because gateHead == state.Local.Head).

The two tests that pin this message both use fixtures where the gate really did move (TestKeepLocalStampFailureAfterTheGateMovedNamesACompletableRetry asserts the moved gate branch and the written recover-gate anchor; TestDefaultRecoveryStampFailureNamesItsOwnRetry uses the plain fixture where the worktree moves), so neither covers a no-op settlement.

No safety consequence: the prescribed return_custody_keep_local retry is still correct and completable, and nothing was left behind to reconcile. It matters only because this branch's whole standard is that a state's message must be true of what it actually did, and this is the one claim added by the fix rounds that runs the other way - overstating instead of understating. Narrowest correction is to scope the sentence to what is known, e.g. "any Git changes this recovery makes are already applied, but the custody return could not be recorded; re-run the same recovery to complete the record", leaving the safety code, the recoveryRetryAction next action, and both existing tests untouched. This is user-visible wording, so it is your call whether to correct it or accept the looser reading.

⚠️ **Test** - 1 info
  • ℹ️ internal/cli/axi_settle_custody_test.go - new test file written by agent: internal/cli/axi_settle_custody_test.go
  • go test -race ./internal/branchsync/ -run &#39;Settle|Settlement|KeepLocal|Stamp|Wedged|RefusalsNameAnExit|AdvertiseSettlement|LeavesNoGateAnchor|AnchorItAlreadyWrote|AnchorTheAttemptWrote&#39; (33 tests, all pass)
  • go test -race ./internal/tui/ -run &#39;Wedged|Settlement|Recover&#39; — covers TestWedgedCustodyRecordReachesTheSameSettlementExitAsTheCLI and TestPipelineOwnedStateWithoutASettlementActionOffersNoSettlement
  • go test -race ./internal/cli/ -run &#39;Custody|Abort|Settlement|Guidance&#39; — abort-surface settlement help and the gap-5 fixture-invariant assertions
  • Added and ran go test -race ./internal/cli/ -run TestWedgedCustodyRecordSettlesThroughTheCLI — new end-to-end journey: axi sync --check -> axi sync --recover (refuses) -> axi sync --recover --keep-local (settles) -> axi sync --check (released), plus live-Git and DB assertions
  • Regression proof: replaced internal/branchsync/sync.go with the 13cdf1a version and re-ran the journey test — it fails (settlement refuses, every next_action is inspect_and_reconcile_manually); restored the file and verified a clean git status
  • TUI before/after capture: rendered renderLocalBranchStatus and the u keypress against the wedged state on both 13cdf1a (branch_sync.go/keys.go/view.go swapped) and HEAD, then restored all files
  • go test -race ./internal/cli/ ./internal/tui/ ./internal/git/ -count=1 and go test -race ./internal/branchsync/ -count=1 — all four packages green
  • Captured rendered no-mistakes sync --help and no-mistakes axi sync --help to confirm the gap-4 conditional --keep-local wording on both surfaces
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

kunchenguid and others added 20 commits August 27, 2026 22:08
…chenguid#877)

* fix(pipeline): measure agent silence and bound a wedged auto-fix agent

A pipeline agent invocation that exhausted its budget reported "agent
timed out after 30m0s (agent silent for 30m0s)": the configured budget
printed twice, with nothing measured, and on the shared agent-run seam
the adapter's own error discarded. A wedged agent, a busy one, and a
crashed one were therefore indistinguishable, so two real 30-minute
timeouts on one task (one at Review, one at the CI fix step) left no
evidence to act on.

Three defects composed that failure.

The claim was fabricated. classifyAgentRun, reviewAgentError, and
testAgentError all restated their own budget as if it were an
observation. agentActivity now measures the invocation and is the single
owner of the evidence: whether output was ever seen, how long ago the
last output was, and whether a subprocess launched.

The evidence was discarded. For a native agent the adapter error carries
the killed process's exit status and its captured stderr, which is the
only account of what the process was doing, including an authentication
prompt written to stderr that nothing else surfaces. It is now appended
to every timeout diagnostic as "agent reported: ...", and testAgentError
no longer replaces it with the bare context cause.

Progress was unobservable. Adapters forward only assistant prose to
OnChunk, and a tool-using turn emits none until the very end (verified
against pi 0.84.3, which streams tool_execution_* and toolcall_* with no
text_delta), so a healthy fix round looked identical to a wedge.
nativeAgentPipe now raises a throttled LifecyclePhaseActivity for every
non-empty read of a native subprocess's stdout or stderr. The executor
records it as step activity so axi status can see a live agent, and never
writes it to the step log. Subprocess start and exit deliberately do not
count as output: start proves launch rather than work, and exit is the
deadline's own consequence.

The CI path also had no bounded outcome. A fix-agent timeout was logged
as a warning and the identical request was re-issued on the next poll,
up to auto_fix.ci attempts, each costing another full budget invisibly
until ci_timeout ended the run hours later. It now parks at an ask-user
gate carrying the measured diagnostic and, when the timed-out agent left
uncommitted work, the run worktree holding it. The run and its worktree
stay alive, so no commits are lost and no second run has to take custody
of the branch, and a further attempt is the operator's to spend. Only a
proven budget burn parks; other fix failures keep warn-and-retry.

Review deliberately still fails the run rather than parking: the Push
step commits leftover worktree changes, so an approved park would ship a
half-finished, unreviewed fix.

No timeout was raised, no retry added, no agent order or model pin
touched.

* no-mistakes(document): Document native agent timeout outcomes

* no-mistakes(ci): Fixed Greptile's retry-lifecycle finding. Retry and unknown lifecycle metadata no longer count as measured agent output, while subprocess byte activity still does. Added a behavioral regression proving retry events remain forwarded but a retry-only invocation is diagnosed as silent. Verified with make lint and race-enabled pipeline and pipeline/steps tests

* no-mistakes(review): Keep fallback notices out of agent activity measurements

* no-mistakes(review): Measure subprocess silence from latest launch

* no-mistakes(review): Redact adapter credentials from timeout findings

* no-mistakes(review): Reset activity evidence between agent attempts

* no-mistakes(review): Reset activity before fresh fixer sessions

* no-mistakes(document): Document per-attempt timeout evidence

* no-mistakes(ci): Fixed Greptile's finding by resetting the activity start timestamp at every retry or fallback boundary, preventing earlier attempts from inflating silence duration when the current attempt never launches. Added a behavioral regression covering a delayed failed attempt followed by a silent, never-launched fallback. Verified with make lint, race-enabled pipeline tests, repeated focused regressions, and the executable silent-agent axi e2e test

* no-mistakes(review): Reset activity before OpenCode format fallback

* no-mistakes(document): Document all fresh-attempt timeout resets
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
A terminal run whose recorded pipeline head can no longer be verified -
missing from every reachable object store, or contradicted by the run's
own recovery ref - left the branch pipeline_owned with no command that
could settle it: every guarded recovery refused on the same unverifiable
head, and abort of the already-terminal run was an idempotent no-op
(kunchenguid#824).

--recover --keep-local now settles that self-inconsistent record. It
stays fail-closed: every reachable copy of the recorded head is pinned
at refs/no-mistakes/recover-stranded/<run> first, and a head that still
exists but cannot be pinned refuses rather than being stranded. The gate
branch moves by the existing compare-and-swap, never a force, so a
concurrent gate push still wins.

Inspection names that exit as next_action.code: return_custody_keep_local
instead of the dead-end manual-reconciliation pointer, and terminal-run
abort responses carry the same command. The kunchenguid#814 polarity is preserved:
an unverifiable record never advertises recover_custody.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four defects in the kunchenguid#824 keep-local settlement, all in edge sub-shapes
rather than the primary flow:

- selfInconsistentCustodyRecord advertised the settlement for a terminal
  run whose head was never verified, but Recover refuses that at its
  unverified-head guard before any keep-local interception - recreating
  the same "advertised action that always refuses" wedge. That shape is
  kunchenguid#707's scope, so the predicate now excludes it and status keeps its
  honest manual-reconciliation pointer.
- A conflicting worktree recovery anchor on the locally reachable path
  refused without settlement while the predicate advertised it. It is
  now intercepted like the gate-side shapes; keeping a head that already
  contains the preserved commits can lose nothing.
- The settlement treated any gate-branch rev-parse error as "branch
  missing" and stamped custody returned without the compare-and-swap.
  Only a branch proven absent via ExactRefTarget settles without one; an
  unreadable branch refuses like the sibling keep-local sites.
- A refusal claimed "no files or refs were changed" after the pin loop
  may already have written the stranded anchor. It now names where that
  anchor exists.

Each fix has a regression test that fails without it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…help

Three findings from the review of 391b3d7:

- selfInconsistentCustodyRecord still treated every incompatible gate
  recovery ref as settleable, but SYMBOLIC evidence is refused before any
  keep-local interception. A dangling symref is invisible to for-each-ref
  while symbolic-ref still succeeds, so the predicate said "incompatible"
  while Recover's own probe saw no ref and then failed inside
  PreserveRecoveryHead's symbolic check - an un-intercepted refusal. A
  resolving symref reaches the ordinary keep-local path and refuses there.
  Symbolic gate-anchor evidence now disqualifies the record outright,
  keeping the honest manual-reconciliation pointer.
- The bare-abort no-op emitted the branch's next action unconditionally,
  so on a released branch it answered an abort by prescribing `axi run` -
  telling the operator to launch a pipeline. Abort help is now allowlisted
  to custody-settlement actions via custodySettlementHelp, applied to all
  three abort sites so the --run siblings cannot drift into the same
  overreach. The structured branch_sync object still reports the branch's
  own next_action, which is state rather than a prescription.
- cli.md claimed every listed refusal was settleable with --keep-local;
  the shapes refused before an interception are not.

The predicate's invariant is now stated once: it may name the settlement
only where Recover(keepLocal) actually reaches it. Each fix has a
regression test that fails without it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two review findings, both narrowly scoped:

- The pipeline_owned gate added to the bare-abort help was never applied
  to the two `axi abort --run <id>` sites. Their only guard was
  state.Pipeline.RunID != runID, which proves the branch resolves to that
  run, not that a run still holds it - so a released, diverged branch
  reached custodySettlementHelp and answered an abort with `git log`
  reconciliation advice the bare site had already been taught not to
  emit. Both conditions are now required, matching the bare site.

- settlementAnchorsFree predicts whether PreserveRecoveryAnchor can write
  an anchor, but that write refuses any symbolic ref first while the
  probe used ExactRefTarget alone. A dangling symref is invisible to
  for-each-ref yet visible to symbolic-ref, so the probe reported "free"
  for a ref the settlement can never write, and inspection advertised a
  settlement that refuses every time. The probe now mirrors
  recoveryAnchorCompatible, covering both the stranded and gate anchors.

Each fix has a regression test that fails without it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the merge blockers the maintainer named on kunchenguid#883, plus the three
smaller published gaps.

recoverKeepLocal wrote refs/no-mistakes/recover-gate/<run> before four
refusals that each reported "no files or refs were changed" - and that
ref is exactly what wedges every later attempt, so those refusals could
recreate the condition this settlement exists to clear. The anchor guards
one operation, the compare-and-swap that moves the gate branch off the
displaced head, and nothing before that swap can strand it because
refs/heads/<branch> still names it. So the write moved to immediately
before the swap and every refusal that can precede it now leaves nothing
behind by construction rather than by a cleanup that could itself fail.
The read-only anchor conflict check stays first. One post-write refusal
remains, the lost swap, deliberately: it failed because the gate moved,
so that pin may be the only ref still naming the displaced head.

The TUI keyed its u action on blocked_pipeline_owned_recoverable while a
self-inconsistent record carries the settlement next action, so it was
the one operator surface with no exit. It now reaches the same settlement
as the CLI, keyed on the advertised next action so it cannot drift from
the predicate deciding where the settlement can complete, behind its own
confirmation because settling KEEPS the local head where recovery takes
the preserved one.

The settlement's data-safety argument is "nothing still has this head, so
nothing can be lost", which only a proven absence supports. Its probe read
any non-zero git exit as absent, so an unreadable store and a present
non-commit object both settled the record while the head was still there.
git.CommitPresence now treats only exit 1 as absence; everything else
refuses, and settlementAnchorsFree mirrors it so the advertisement agrees
with the write.

finishRecover's stamp failure runs after the Git side already succeeded
and nilled NextAction, leaving a moved gate branch with no named exit -
the same shape one layer down. It now reports that the changes are applied
and names the same recovery command, which completes because every Git
step it repeats is idempotent once applied.

Also: --keep-local help no longer claims the gate always compare-and-swaps,
since the equal/ahead, user_owned, already-recovered and absent-gate-branch
paths return first; and the two --run abort tests regain the fixture
invariants that prove they exercise the StatePipelineOwned clause - their
responses carry no branch_sync object, so they could not have failed even
with that clause deleted.

Refs kunchenguid#824.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAsKnxZJbLEiUKMcUtCL2U
@hmcdaniel03

Copy link
Copy Markdown
Owner Author

Opened in error by the validation tooling against the wrong base; the real pull request for this work is kunchenguid#883. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants