Skip to content

feat: preserve Firstmate continuity across Claude context compaction - #20

Merged
ruby-dlee merged 4 commits into
mainfrom
fm/autocompact-graceful-g2
Jul 22, 2026
Merged

feat: preserve Firstmate continuity across Claude context compaction#20
ruby-dlee merged 4 commits into
mainfrom
fm/autocompact-graceful-g2

Conversation

@ruby-dlee

Copy link
Copy Markdown
Owner

Intent

Make Firstmate graceful around Claude Code context autocompaction. Use the verified PreCompact hook to deterministically capture a fresh local durable resume anchor before both automatic and manual compaction, then use a compact-sourced SessionStart recovery path to inject that anchor plus normal session-start reconciliation so a long autonomous primary resumes without lost durable fleet state. Keep the hook Firstmate-scoped and additive to Stop and PreToolUse, reuse stow only for periodic judgment-based conversation knowledge, empirically verify Claude Code event payloads and ordering plus manual and automatic behavior in isolated homes, add shellcheck-clean colocated tests and documentation, and hold the PR for captain merge. Do not implement compaction triggering or unrelated away-mode, dialog-detection, or permission work.

What Changed

  • Register Firstmate-scoped Claude hooks that atomically capture durable fleet state before manual or automatic compaction and restore it with normal session-start reconciliation afterward.
  • Fail closed on capture errors while preserving recovery for malformed payloads and environments without jq.
  • Add colocated shell coverage and documentation for hook behavior, recovery boundaries, and periodic stow usage.

Risk Assessment

✅ Low: Captain, the change is well-bounded, the prior fail-open paths are addressed, and the source now conforms to the stated autocompaction recovery intent.

Testing

The supplied full baseline had already passed; focused automation and isolated manual/automatic capture-to-recovery replays also passed on Claude Code 2.1.217, produced reviewer-visible hook transcripts, confirmed Firstmate-only scoping and captain-held PR preservation, and left the source worktree clean. No screenshot was applicable because the changed user surface is CLI hook output rather than a graphical UI.

Evidence: End-to-end compaction recovery transcript
Manual capture proof:
Trigger: `manual`
      compact-e2e,default,needs-decision,"PR held for captain merge [key=merge-hold]"
      compact-e2e,"https://github.com/ruby-dlee/firstmate/pull/9999"
    # MANUAL_CAPTURE_STATE_4417
    pr=https://github.com/ruby-dlee/firstmate/pull/9999

Automatic refresh proof (manual marker must be absent):
Trigger: `auto`
      compact-e2e,default,needs-decision,"PR held for captain merge [key=merge-hold]"
      post-compact-followup,Resume after held captain merge,compact-e2e,captain merge required
      compact-e2e,"https://github.com/ruby-dlee/firstmate/pull/9999"
      post-compact-followup,"- [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required"
    # AUTO_FRESH_STATE_8823
    - [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required
    pr=https://github.com/ruby-dlee/firstmate/pull/9999
PASS: stale manual marker absent after automatic atomic refresh

Compact SessionStart recovery proof:
FIRSTMATE AUTOCOMPACT RECOVERY CONTEXT
=== FRESH RESUME ANCHOR: /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/data/autocompact-resume.md ===
Trigger: `auto`
      compact-e2e,default,needs-decision,"PR held for captain merge [key=merge-hold]"
      post-compact-followup,Resume after held captain merge,compact-e2e,captain merge required
      compact-e2e,"https://github.com/ruby-dlee/firstmate/pull/9999"
      post-compact-followup,"- [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required"
    # AUTO_FRESH_STATE_8823
    - [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required
    pr=https://github.com/ruby-dlee/firstmate/pull/9999
=== NORMAL SESSION-START RECONCILIATION ===
SESSION START - /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home
# AUTO_FRESH_STATE_8823
- [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required
pr=https://github.com/ruby-dlee/firstmate/pull/9999
status tail (last 5 line(s), wake-EVENT history, not current state; full log: /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/state/compact-e2e.status):
needs-decision: PR held for captain merge [key=merge-hold]

Firstmate scoping proof:
PASS: marked secondmate primary captured SECONDMATE_DURABLE_STATE_6134
PASS: unmarked linked child worktree was silent and wrote no anchor
PASS: non-Firstmate directory was silent and wrote no anchor

Runtime used for the documented empirical event probes:
2.1.217 (Claude Code)
Evidence: Manual PreCompact resume anchor
# Autocompact resume anchor

Generated: `2026-07-22T17:25:00Z`
Trigger: `manual`
Session: `evidence-manual`
Transcript: `/isolated/manual.jsonl`

This file is the deterministic bridge across Claude Code context compaction.
It captures durable file state only and does not replace the judgment-based `stow` skill.
The compact-sourced SessionStart hook prints this anchor and then runs `bin/fm-session-start.sh` for normal lock, wake, backlog, task, and endpoint reconciliation.

## Fleet pickup snapshot

    schema: fm-bearings.v1
    home: 01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home
    generated: "2026-07-22T17:25:02Z"
    prs: "not_requested (run: /bearings include PRs)"
    in_flight[1]{id,kind,state,doing}:
      compact-e2e,ship,unknown,"backend target gone: firstmate:fm-compact-e2e"
    decisions_open[1]{id,key,verb,summary}:
      compact-e2e,default,needs-decision,"PR held for captain merge [key=merge-hold]"
    landed: []
    gates: []
    reports: []
    recorded_prs[1]{id,url}:
      compact-e2e,"https://github.com/ruby-dlee/firstmate/pull/9999"
    bodies: []
    paths[1]{id,worktree,home,status,report}:
      compact-e2e,/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-primary,"-",/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/state/compact-e2e.status,/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/data/compact-e2e/report.md
    actions[1]{id,watch,steer}:
      compact-e2e,bin/fm-peek.sh fm-compact-e2e,bin/fm-send.sh fm-compact-e2e '<instruction>'
    endpoints[1]{id,backend,target,exists,agent}:
      compact-e2e,tmux,"firstmate:fm-compact-e2e",true,not_checked
    omitted[1]{surface,reveal}:
      live PR discovery + checks,"--include-prs"

## Backlog at capture

    # MANUAL_CAPTURE_STATE_4417
    ## In flight
    - [ ] compact-e2e - Preserve durable work through compaction

## In-flight metadata at capture

### compact-e2e

    window=firstmate:fm-compact-e2e
    worktree=/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-primary
    kind=ship
    mode=no-mistakes
    pr=https://github.com/ruby-dlee/firstmate/pull/9999
Evidence: Fresh automatic PreCompact resume anchor
# Autocompact resume anchor

Generated: `2026-07-22T17:25:02Z`
Trigger: `auto`
Session: `evidence-auto`
Transcript: `/isolated/auto.jsonl`

This file is the deterministic bridge across Claude Code context compaction.
It captures durable file state only and does not replace the judgment-based `stow` skill.
The compact-sourced SessionStart hook prints this anchor and then runs `bin/fm-session-start.sh` for normal lock, wake, backlog, task, and endpoint reconciliation.

## Fleet pickup snapshot

    schema: fm-bearings.v1
    home: 01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home
    generated: "2026-07-22T17:25:02Z"
    prs: "not_requested (run: /bearings include PRs)"
    in_flight[1]{id,kind,state,doing}:
      compact-e2e,ship,unknown,"backend target gone: firstmate:fm-compact-e2e"
    decisions_open[1]{id,key,verb,summary}:
      compact-e2e,default,needs-decision,"PR held for captain merge [key=merge-hold]"
    landed: []
    gates[1]{id,title,blocked_by,reason}:
      post-compact-followup,Resume after held captain merge,compact-e2e,captain merge required
    reports: []
    recorded_prs[1]{id,url}:
      compact-e2e,"https://github.com/ruby-dlee/firstmate/pull/9999"
    bodies[1]{id,body}:
      post-compact-followup,"- [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required"
    paths[1]{id,worktree,home,status,report}:
      compact-e2e,/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-primary,"-",/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/state/compact-e2e.status,/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/data/compact-e2e/report.md
    actions[1]{id,watch,steer}:
      compact-e2e,bin/fm-peek.sh fm-compact-e2e,bin/fm-send.sh fm-compact-e2e '<instruction>'
    endpoints[1]{id,backend,target,exists,agent}:
      compact-e2e,tmux,"firstmate:fm-compact-e2e",true,not_checked
    omitted[1]{surface,reveal}:
      live PR discovery + checks,"--include-prs"

## Backlog at capture

    # AUTO_FRESH_STATE_8823
    ## In flight
    - [ ] compact-e2e - Preserve durable work through compaction
    ## Queued
    - [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required

## In-flight metadata at capture

### compact-e2e

    window=firstmate:fm-compact-e2e
    worktree=/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-primary
    kind=ship
    mode=no-mistakes
    pr=https://github.com/ruby-dlee/firstmate/pull/9999
Evidence: Compact-sourced SessionStart recovery output
FIRSTMATE AUTOCOMPACT RECOVERY CONTEXT
Treat the fresh durable anchor and session-start digest below as authoritative over the lossy compaction summary.
Resume the in-flight work directly after reconciling the drained wake queue and live endpoints.

=== FRESH RESUME ANCHOR: /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/data/autocompact-resume.md ===
# Autocompact resume anchor

Generated: `2026-07-22T17:25:02Z`
Trigger: `auto`
Session: `evidence-auto`
Transcript: `/isolated/auto.jsonl`

This file is the deterministic bridge across Claude Code context compaction.
It captures durable file state only and does not replace the judgment-based `stow` skill.
The compact-sourced SessionStart hook prints this anchor and then runs `bin/fm-session-start.sh` for normal lock, wake, backlog, task, and endpoint reconciliation.

## Fleet pickup snapshot

    schema: fm-bearings.v1
    home: 01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home
    generated: "2026-07-22T17:25:02Z"
    prs: "not_requested (run: /bearings include PRs)"
    in_flight[1]{id,kind,state,doing}:
      compact-e2e,ship,unknown,"backend target gone: firstmate:fm-compact-e2e"
    decisions_open[1]{id,key,verb,summary}:
      compact-e2e,default,needs-decision,"PR held for captain merge [key=merge-hold]"
    landed: []
    gates[1]{id,title,blocked_by,reason}:
      post-compact-followup,Resume after held captain merge,compact-e2e,captain merge required
    reports: []
    recorded_prs[1]{id,url}:
      compact-e2e,"https://github.com/ruby-dlee/firstmate/pull/9999"
    bodies[1]{id,body}:
      post-compact-followup,"- [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required"
    paths[1]{id,worktree,home,status,report}:
      compact-e2e,/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-primary,"-",/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/state/compact-e2e.status,/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/data/compact-e2e/report.md
    actions[1]{id,watch,steer}:
      compact-e2e,bin/fm-peek.sh fm-compact-e2e,bin/fm-send.sh fm-compact-e2e '<instruction>'
    endpoints[1]{id,backend,target,exists,agent}:
      compact-e2e,tmux,"firstmate:fm-compact-e2e",true,not_checked
    omitted[1]{surface,reveal}:
      live PR discovery + checks,"--include-prs"

## Backlog at capture

    # AUTO_FRESH_STATE_8823
    ## In flight
    - [ ] compact-e2e - Preserve durable work through compaction
    ## Queued
    - [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required

## In-flight metadata at capture

### compact-e2e

    window=firstmate:fm-compact-e2e
    worktree=/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-primary
    kind=ship
    mode=no-mistakes
    pr=https://github.com/ruby-dlee/firstmate/pull/9999

=== NORMAL SESSION-START RECONCILIATION ===

================================================================================
SESSION START - /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home
================================================================================

LOCK
--------------------------------------------------------------------------------
lock acquired: harness pid 72683

BOOTSTRAP
--------------------------------------------------------------------------------
TASKS_AXI: available

WAKE QUEUE
--------------------------------------------------------------------------------
●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
●  WATCHER DOWN - SUPERVISION IS OFF
●  1 task(s) in flight, but no watcher has a fresh beacon (last beat: never, grace 300s).
●  Trust the emitted supervision protocol for this harness; do not use shell & for watcher repair.
●  This is a supervision warning only; the guarded operation WILL still run.
●  resume supervision according to the session-start block for this harness; do not use shell &.
●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
================================================================================
SUPERVISION OPERATING INSTRUCTIONS - primary harness: codex
================================================================================
Current state:
- Lock: held by this session; this session owns normal supervision unless away mode says otherwise.
- Away mode: inactive.
- X mode: inactive; use the default watcher cadence.
- After every handled wake, resume this emitted harness protocol instead of following a hardcoded background-arm recipe.

Mode: Codex foreground checkpoint.

When this session owns supervision and away mode is not active:
1. Drain first with `bin/fm-wake-drain.sh`.
2. Source `/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/config/x-mode.env` first when X mode is active.
3. Run one foreground watcher checkpoint with `bin/fm-watch-checkpoint.sh --seconds "${FM_CODEX_WATCH_CHECKPOINT:-180}"`.
4. If the command prints `signal:`, `stale:`, `check:`, or `heartbeat`, drain queued wakes, handle that wake, then start the next checkpoint.
5. If the command prints `checkpoint:` or exits 124 with no wake, drain queued wakes anyway, process any queued user message now visible to Codex, then start the next checkpoint.
6. Never use shell `&` or Codex background tasks for firstmate watcher supervision.
7. Do not run `bin/fm-watch-arm.sh` as Codex's normal supervision command.
   If it is ever shelled anyway, a backgrounded, piped, or bundled anti-pattern is denied automatically by the PreToolUse seatbelt (`bin/fm-arm-pretool-check.sh`) registered in `.codex/hooks.json`.

Codex cannot reason while a foreground tool call is running.
The bounded checkpoint returns control regularly so user messages and queued wakes can be handled without relying on background-task wake semantics.


================================================================================
CONTEXT
================================================================================

data/projects.md
--------------------------------------------------------------------------------
ABSENT

data/secondmates.md
--------------------------------------------------------------------------------
ABSENT

data/captain.md
--------------------------------------------------------------------------------
ABSENT

data/learnings.md
--------------------------------------------------------------------------------
ABSENT

================================================================================
FLEET STATE
================================================================================

data/backlog.md
--------------------------------------------------------------------------------
# AUTO_FRESH_STATE_8823
## In flight
- [ ] compact-e2e - Preserve durable work through compaction
## Queued
- [ ] post-compact-followup - Resume after held captain merge blocked-by: compact-e2e - captain merge required

In-flight tasks (state/*.meta)
--------------------------------------------------------------------------------

--- compact-e2e ---
window=firstmate:fm-compact-e2e
worktree=/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-primary
kind=ship
mode=no-mistakes
pr=https://github.com/ruby-dlee/firstmate/pull/9999
endpoint: alive (backend=tmux window=firstmate:fm-compact-e2e)
status tail (last 5 line(s), wake-EVENT history, not current state; full log: /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KY58PHJ9WM8X67PQF5AB2CM3/isolated-home/state/compact-e2e.status):
needs-decision: PR held for captain merge [key=merge-hold]

Orphan status logs (state/*.status without matching .meta)
--------------------------------------------------------------------------------
(none)

AFK
--------------------------------------------------------------------------------
absent

================================================================================
NEXT STEP
================================================================================
Follow the supervision operating instructions block above for harness 'codex'.
This script never starts supervision itself.

The digest above is complete for this session start. Do NOT re-read
data/projects.md, data/secondmates.md, data/captain.md, data/learnings.md,
data/backlog.md, or state/*.meta now - they were just printed in full.
Do NOT bulk-read state/*.status now either: their bounded tails were just
printed with full log paths for targeted follow-up when older wake-event
history is actually needed. Re-reading everything defeats the entire point
of this command. Re-read a file only if this digest flagged it ABSENT (then
rebuild or create it per AGENTS.md), its contents looked unparseable/corrupt,
or an individual full status log is needed for older wake-event history.

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed (2) ✅
  • 🚨 bin/fm-autocompact.sh:66 - The required criterion says to “deterministically capture a fresh local durable resume anchor before both automatic and manual compaction” and inject recovery on compact-sourced SessionStart, but command -v jq ... || exit 0 silently disables both phases. Because jq is not an unconditional bootstrap dependency, a valid primary can compact without an anchor or recovery. Confirm this must fail closed during capture and still surface recovery context after compaction.
  • 🚨 bin/fm-autocompact.sh:182 - The required fresh-anchor guarantee is contradicted because the rendering group returns only its final command’s status. An earlier printf or sed failure can therefore be masked by the successful line-182 test, after which an incomplete temporary file is published and compaction proceeds. Ensure every rendering failure prevents the atomic move.

🔧 Fix: Captain, harden jq-free autocompact recovery
1 warning still open:

  • ⚠️ bin/fm-autocompact.sh:199 - Once an in-scope recover command is running, unreadable or empty stdin and malformed or missing event fields still exit successfully before recover_context. Because the registered hook is already compact-scoped, these transport/parser failures suppress both the readable anchor and normal session-start digest after the compaction boundary. Emit recovery with a warning on parse failure; reserve silent no-op behavior for a successfully parsed non-compact event.

🔧 Fix: Captain, preserve recovery on malformed payloads
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit "$rc"
  • Pre-supplied successful baseline: command -v tmux &gt;/dev/null || { echo &#34;tmux is required for e2e tests&#34; &gt;&amp;2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo &#34;== $t ==&#34;; bash &#34;$t&#34; || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit &#34;$rc&#34;
  • Focused automation: bash tests/fm-autocompact.test.sh
  • Manual replay: piped a manual PreCompact payload through FM_ROOT_OVERRIDE=&lt;isolated-primary&gt; FM_HOME=&lt;isolated-home&gt; ./bin/fm-autocompact.sh capture
  • Automatic replay: piped an automatic PreCompact payload through the same isolated capture path and verified the prior manual marker disappeared
  • Recovery replay: piped a compact SessionStart payload through FM_GATE_REFUSE_BYPASS=1 FM_ROOT_OVERRIDE=&lt;isolated-primary&gt; FM_HOME=&lt;isolated-home&gt; ./bin/fm-autocompact.sh recover
  • Scoping replay in a marked secondmate home, an unmarked linked child worktree, and a non-Firstmate directory
  • Runtime check: claude --version returned Claude Code 2.1.217
  • Cleanup verification: git status --short returned clean
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@ruby-dlee
ruby-dlee merged commit 7fe76a0 into main Jul 22, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant