diff --git a/.agents/skills/harness-adapters/SKILL.md b/.agents/skills/harness-adapters/SKILL.md index 03a9b2893e..702c693201 100644 --- a/.agents/skills/harness-adapters/SKILL.md +++ b/.agents/skills/harness-adapters/SKILL.md @@ -202,13 +202,15 @@ Its broader dark-TRUECOLOR placeholder handling and dark-theme tradeoff are docu That styled capture is internal to the boolean detector only. `fm-peek` and every other human or LLM-facing capture path stays plain `tmux capture-pane` with no escape codes. -**Primary-session guard fact (verified 2026-07-04, Claude Code 2.1.201; preserved 2026-07-08, Claude Code 2.1.204; Stop-owned auto-arm revalidated 2026-07-24, Claude Code 2.1.219).** +**Primary-session guard fact.** +[`docs/turnend-guard.md`](../../../docs/turnend-guard.md) owns the current mechanism, and [`docs/verification/supervision.md`](../../../docs/verification/supervision.md#turn-end-guard) owns dated evidence. This is separate from the per-task crewmate turn-end hook above (that one just `touch`es a marker file in a task's own `.claude/settings.local.json`). The firstmate PRIMARY's own `.claude/settings.json` registers two Stop hooks: `bin/fm-turnend-guard.sh --claude` and the Stop-owned auto-arm `bin/fm-claude-stop-autoarm.sh` (`asyncRewake: true`, `timeout: 28800`), and exiting the guard with status 2 plus stderr reliably forces the model to continue. -Claude Code's stdin payload to a Stop hook carries a `stop_hook_active` boolean that is `true` when the current stop attempt follows ANY stop-hook-driven continuation, including `asyncRewake` rewakes; the primary guard therefore ignores it in `--claude` mode and uses the cooperative claim/epoch check plus a bounded re-block budget instead, while the codex-mode default still treats it as a one-block loop guard. +Claude Code's stdin payload to a Stop hook carries a `stop_hook_active` boolean that is `true` when the current stop attempt follows ANY stop-hook-driven continuation, including `asyncRewake` rewakes; the primary guard therefore ignores it in `--claude` mode. +The current owner above defines its shared session-ownership boundary, one-shot escalation after two identical no-claim blocks, and separate bounded progression for verified automatic failures; the codex-mode default still treats `stop_hook_active` as a one-block loop guard. A project-level `.claude/settings.json` only takes effect when Claude Code's project root is that exact directory - it does not walk up from a subdirectory looking for one, so firstmate launches the primary from the repo root. After those settings are loaded, hook command resolution is still cwd-sensitive because Claude Code runs commands through `/bin/sh` against the session's current cwd; keep the tracked commands anchored through `"$CLAUDE_PROJECT_DIR"/bin/...` and see `docs/turnend-guard.md` for the verified Stop-hook details. -Claude Code's primary watcher protocol is Stop-owned: the auto-arm hook fires on every Stop and foregrounds `bin/fm-watch-arm.sh` when the home is eligible and still needs supervision, and its exit-2 `asyncRewake` rewake is the wake; the model drains and handles wakes but never runs a routine re-arm command. +Claude Code's primary watcher protocol is Stop-owned: the auto-arm hook fires on every Stop and foregrounds `bin/fm-watch-arm.sh` when the home is eligible and still needs supervision, and its exit-2 `asyncRewake` rewake is the wake; the model presents and handles wakes, runs the drain's printed post-handling acknowledgement, and never runs a routine re-arm command. ## codex (VERIFIED 2026-06-11, codex-cli 0.139.0) @@ -287,8 +289,8 @@ The follow-up was verified in the interactive TUI; `opencode run` can exit befor | Interrupt | single Escape | Pi has no permission system, so crewmates are always autonomous. -Pi's `packages/coding-agent/docs/settings.md` UI and display section documents `regular` as the `tuiMode` default and `fullscreen` as experimental; fullscreen can bury steers by rewriting scrollback, so Firstmate avoids it when the installed CLI supports the override. -`fm-spawn.sh --help` owns the executable-pinning and version-safe launch mechanics. +Pi 0.83 removed the former `--tui-mode` startup option and its `tuiMode` setting, while remaining an interactive terminal application by default; `fm-spawn` therefore omits the obsolete option so Pi-family crews can start on current installations. +`fm-spawn.sh --help` owns the executable-pinning and launch mechanics. `pi-signed` is the signed wrapper identity verified on version 0.82.0 and exposes the same CLI and TUI behavior as Pi. Firstmate records `pi-signed` without normalization and refuses rather than falling back to `pi` when that wrapper is unavailable. The observed signed process tree is an exact `pi-signed` wrapper parent with the Pi application as its child, while tmux reports the foreground command as the exact `pi-launcher` name for both selected executables. diff --git a/AGENTS.md b/AGENTS.md index bd40813bf7..337c3a3fa3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -120,7 +120,7 @@ state/ runtime records and signals; gitignored .status-presentation-cursor .status-presentation-lock fleet-wide per-task status identity/byte-offset manifest and serialization lock preventing already-presented status lines from being replayed as new; owned by fm-classify-lib.sh, with each task's row retired by teardown .afk durable away-mode flag; present = sub-supervisor may inject escalations (set by /afk, cleared on user return) .watch.lock .wake-queue.lock watcher singleton and queue serialization locks - .claude-autoarm.lock .claude-autoarm-epoch .claude-autoarm-failure-notified .claude-autoarm-failure-alarmed .turnend-claude-blocks .turnend-claude-blocks.lock Claude Stop auto-arm single-flight, epoch, failure-episode, attended-alarm, guard-budget, and budget-lock records; never touch + .claude-autoarm.lock .claude-autoarm-epoch .claude-autoarm-entry-trace .claude-autoarm-entry-trace.lock .claude-autoarm-failure-notified .claude-autoarm-failure-alarmed .turnend-claude-blocks .turnend-claude-blocks.lock .turnend-claude-escalated Claude Stop auto-arm single-flight, epoch, bounded entry diagnostics, failure episode, attended alarm, guard budget, budget lock, and one-shot escalation records; never touch .cursor-park-owner .cursor-park-owner.lock .turnend-cursor-blocks Cursor stop-hook owner record, publication and commit lock, and bounded repair-nag budget; never touch .hash-* .count-* .stale-* .stale-since-* .paused-* .wedge-escalations-* .seen-* .hb-surfaced-* .last-* .heartbeat-streak watcher internals; never touch .watch-triage.log watcher's absorbed-wake debug log (size-capped); never relied on, safe to delete diff --git a/bin/fm-claude-stop-autoarm.sh b/bin/fm-claude-stop-autoarm.sh index 806be1bfab..ab73b20f3f 100755 --- a/bin/fm-claude-stop-autoarm.sh +++ b/bin/fm-claude-stop-autoarm.sh @@ -64,6 +64,9 @@ OWNER_LOCK="$STATE/.claude-autoarm.lock" EPOCH="$STATE/.claude-autoarm-epoch" FAILURE_NOTICE="$STATE/.claude-autoarm-failure-notified" FAILURE_ALARM="$STATE/.claude-autoarm-failure-alarmed" +ENTRY_TRACE="$STATE/.claude-autoarm-entry-trace" +ENTRY_TRACE_LOCK="$STATE/.claude-autoarm-entry-trace.lock" +ENTRY_TRACE_MAX_LINES=256 AUTOARM_ATTEMPTS=${FM_CLAUDE_AUTOARM_ATTEMPTS:-2} case "$AUTOARM_ATTEMPTS" in 1|2|3) : ;; @@ -81,6 +84,31 @@ esac # shellcheck source=bin/fm-hook-host-lib.sh . "$SCRIPT_DIR/fm-hook-host-lib.sh" +# The bounded volatile entry trace distinguishes a hook that never ran from one +# that took a pre-claim gate. Trace I/O is strictly best-effort and never waits, +# prints, or changes the hook result. Concurrent appends may briefly exceed the +# bound; the next successful trimming claim restores it. +trace_entry_event() { # + local event=$1 count tmp + [ -d "$STATE" ] || return 0 + if [ -e "$ENTRY_TRACE" ] && { [ ! -f "$ENTRY_TRACE" ] || [ -L "$ENTRY_TRACE" ]; }; then + return 0 + fi + printf 'at=%s pid=%s event=%s\n' "$(date +%s)" "${BASHPID:-$$}" "$event" \ + >> "$ENTRY_TRACE" 2>/dev/null || return 0 + fm_lock_try_acquire "$ENTRY_TRACE_LOCK" || return 0 + count=$(awk 'END { print NR }' "$ENTRY_TRACE" 2>/dev/null || true) + case "$count" in ''|*[!0-9]*) count=0 ;; esac + if [ "$count" -gt "$ENTRY_TRACE_MAX_LINES" ]; then + tmp="$ENTRY_TRACE.tmp.${BASHPID:-$$}" + tail -n "$ENTRY_TRACE_MAX_LINES" "$ENTRY_TRACE" > "$tmp" 2>/dev/null \ + && mv -f "$tmp" "$ENTRY_TRACE" 2>/dev/null + rm -f "$tmp" 2>/dev/null || true + fi + fm_lock_release "$ENTRY_TRACE_LOCK" + return 0 +} + # Consume the Stop payload once. The decisions below are state-based; the # payload is read so a slow writer can never wedge on a full pipe, and its host # is inspected before anything else runs. @@ -93,9 +121,13 @@ PAYLOAD=$(cat 2>/dev/null || true) # (docs/turnend-guard.md "Harness integrations"). Cursor's own park adapter owns # its turn boundary, so stand down on a Cursor-delivered payload. fm_hook_payload_is_foreign_host "$PAYLOAD" && exit 0 +trace_entry_event entry # --- scope: genuine primary checkout only ----------------------------------- -fm_primary_scope_matches "$FM_ROOT" "$STATE" || exit 0 +if ! fm_primary_scope_matches "$FM_ROOT" "$STATE"; then + trace_entry_event gate-scope + exit 0 +fi # --- identity: only the lock-owning session's hooks may arm ------------------ # A prior session may have died after leaving its numeric harness pid in .lock. @@ -107,40 +139,61 @@ RECOVER_SESSION_LOCK=0 if ! fm_session_lock_owned_by_self "$STATE"; then LOCK_PID=$(cat "$STATE/.lock" 2>/dev/null || true) case "$LOCK_PID" in - ''|*[!0-9]*) exit 0 ;; + '') trace_entry_event gate-lock-missing; exit 0 ;; + *[!0-9]*) trace_entry_event gate-lock-malformed; exit 0 ;; esac - fm_harness_pid_alive "$LOCK_PID" && exit 0 + if fm_harness_pid_alive "$LOCK_PID"; then + trace_entry_event gate-live-session-owner + exit 0 + fi RECOVER_SESSION_LOCK=1 fi # --- AFK: the away daemon owns the watcher and triage; never rewake ---------- -[ -e "$STATE/.afk" ] && exit 0 +if [ -e "$STATE/.afk" ]; then + trace_entry_event gate-afk + exit 0 +fi -# --- need: in-flight work or an X-mode relay poll ---------------------------- +# --- need: work, relay polling, process sources, or queued wake delivery ----- need_supervision() { fm_supervision_needed "$STATE" "$GRACE" } -need_supervision || exit 0 +if ! need_supervision; then + trace_entry_event gate-no-supervision + exit 0 +fi # --- stale session-lock recovery --------------------------------------------- # Delegate the claim to fm-lock.sh so its live-owner refusal and write semantics # remain the single acquisition owner, then re-verify current-session identity # before touching any auto-arm state. if [ "$RECOVER_SESSION_LOCK" -eq 1 ]; then - "$SCRIPT_DIR/fm-lock.sh" >/dev/null 2>&1 || exit 0 - fm_session_lock_owned_by_self "$STATE" || exit 0 + if ! "$SCRIPT_DIR/fm-lock.sh" >/dev/null 2>&1; then + trace_entry_event gate-lock-recovery-failed + exit 0 + fi + if ! fm_session_lock_owned_by_self "$STATE"; then + trace_entry_event gate-identity-unresolved + exit 0 + fi fi # --- single-flight owner claim ------------------------------------------------ # Claude runs one background process per firing with no dedupe. Exactly one # owner foregrounds the arm and translates its close; every other firing exits # 0 so one watcher cycle maps to at most one exit-2 rewake. -fm_lock_try_acquire "$OWNER_LOCK" || exit 0 +if ! fm_lock_try_acquire "$OWNER_LOCK"; then + trace_entry_event gate-owner-lock-held + exit 0 +fi if ! fm_lock_set_role "$OWNER_LOCK" autoarm; then + trace_entry_event gate-owner-role-failed fm_lock_release "$OWNER_LOCK" exit 0 fi trap 'fm_lock_release "$OWNER_LOCK"' EXIT +trace_entry_event claimed write_epoch() { # local outcome=$1 seq tmp diff --git a/bin/fm-guard.sh b/bin/fm-guard.sh index 21d6da3ed8..26bceef946 100755 --- a/bin/fm-guard.sh +++ b/bin/fm-guard.sh @@ -1,12 +1,12 @@ #!/usr/bin/env bash # Watcher liveness and worktree-tangle guard, called by supervision scripts, by -# fm-wake-drain.sh after it empties queued wakes, and by fm-session-start.sh in +# fm-wake-drain.sh after it presents queued wakes, and by fm-session-start.sh in # read-only advisory mode whenever session-lock ownership was not verified. # First, always warn if the firstmate primary checkout (FM_ROOT) is on a named # non-default branch, because that means firstmate-on-itself work landed in the # primary instead of an isolated worktree. -# Then, if a task is in flight (a state/.meta exists) or X-mode relay -# polling is active (state/x-watch.check.sh exists) and supervision is not +# Then, if a task is in flight, a process-event source is registered, X-mode +# Relay polling is active, or wake delivery is pending and supervision is not # healthy, prints a loud, clearly delimited banner so the agent cannot skim past # it in the tool output of whatever it was doing - the one channel every harness # has. Supervision health is MODEL-AWARE (fm_watcher_supervision_verdict in @@ -37,7 +37,6 @@ STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}" CONFIG="${FM_CONFIG_OVERRIDE:-$FM_HOME/config}" WATCH="$SCRIPT_DIR/fm-watch.sh" GRACE=${FM_GUARD_GRACE:-300} -queue_pending=false READ_ONLY=${FM_GUARD_READ_ONLY:-0} case "$READ_ONLY" in 1|true|TRUE|yes|YES) READ_ONLY=1 ;; *) READ_ONLY=0 ;; esac CONTINUE_LINE=${FM_GUARD_CONTINUE_LINE:-This is a supervision warning only; the guarded operation WILL still run.} @@ -150,12 +149,13 @@ fi # Compute supervision need and watcher-beacon freshness via the shared # grace-based predicate (bin/fm-supervision-lib.sh). Act when work, an event -# source, or an X-mode relay poll needs supervision. +# source, an X-mode relay poll, or pending wake delivery needs supervision. fm_supervision_status "$STATE" "$GRACE" in_flight=$FM_SUP_IN_FLIGHT sources=$FM_SUP_SOURCES needed=$FM_SUP_NEEDED beacon_desc=$FM_SUP_BEACON_DESC +queue_pending=$FM_SUP_QUEUE_PENDING fm_watcher_supervision_verdict "$STATE" "$WATCH" "$GRACE" "$FM_HOME" "$FM_ROOT" watcher_healthy=$FM_WATCHER_VERDICT_OK watcher_down_reason=$FM_WATCHER_VERDICT_REASON @@ -167,8 +167,6 @@ if [ "$needed" = false ]; then exit 0 fi -[ -s "$FM_WAKE_QUEUE" ] && queue_pending=true - # No fresh watcher with tasks in flight is the dangerous state: emit a prominent, # bordered banner FIRST so it reads as an alarm, not a buried stderr line. Later # calls in the same episode get a one-line reminder only. @@ -207,6 +205,8 @@ if [ "$watcher_healthy" = false ]; then printf '● %s task(s) in flight, but %s.\n' "$in_flight" "$watcher_cause" elif [ "$sources" -gt 0 ]; then printf '● %s process-event source(s) registered, but %s.\n' "$sources" "$watcher_cause" + elif "$queue_pending"; then + printf '● Durable queued wake delivery pending, but %s.\n' "$watcher_cause" else printf '● X-mode relay polling needs supervision, but %s.\n' "$watcher_cause" fi diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index cfb25f0058..981ee6df32 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -108,11 +108,9 @@ # overrides it for this spawn (either kind). A non-flag string containing # whitespace is treated as a RAW launch command - the escape hatch for verifying # new adapters. For pi and pi-signed, fm-spawn resolves the selected executable -# name from PATH once, probes that concrete path with --help, and launches the -# same path. It adds --tui-mode regular only when that help advertises the flag; -# a failed or inconclusive probe omits it so older Pi versions remain launchable. -# A missing selected executable refuses before endpoint creation, and pi-signed -# never falls back to pi. +# name from PATH once and launches the same path without the removed --tui-mode +# option. A missing selected executable refuses before endpoint creation, and +# pi-signed never falls back to pi. # config/secondmate-harness may also carry an optional model and effort as extra # whitespace-separated tokens (" [] []"). For a # --secondmate spawn, those tokens apply only when this spawn also resolves its @@ -151,7 +149,6 @@ # Launch templates live in launch_template() below; placeholders replaced before launch: # __BRIEF__ absolute path to data//brief.md # __PIBIN__ quoted concrete Pi-family executable path resolved from PATH -# __PITUIMODE__ optional --tui-mode regular when that executable advertises it # __TURNEND__ absolute path to state/.turn-ended (for harnesses whose # turn-end signal rides the launch command, e.g. codex -c notify=[...]) # __PIEXT__ absolute path to state/.pi-ext.ts (pi turn-end extension, @@ -1087,15 +1084,6 @@ resolve_pi_executable() { esac } -# Pi's CLI surface is version-dependent, so probe the resolved executable's help -# before composing the optional regular-TUI flag. An absent or inconclusive probe -# omits the flag so older Pi versions can still spawn. -pi_supports_tui_mode() { - local executable=$1 help - help=$("$executable" --help 2>&1) || return 1 - printf '%s\n' "$help" | grep -Eq -- '(^|[[:space:]])--tui-mode([[:space:]=]|$)' -} - # The verified launch command per adapter. The knowledge half of each adapter # (busy-state source, exit command, dialogs, quirks) lives in the harness-adapters skill. launch_template() { @@ -1121,7 +1109,7 @@ launch_template() { ;; opencode) printf '%s' 'OPENCODE_CONFIG_CONTENT='\''{"permission":{"*":"allow"}}'\'' opencode __MODELFLAG__--prompt "$(__OPINPUT__ encode launch-brief < __BRIEF__)"' ;; pi|pi-signed) - printf '%s' '__PIBIN____PITUIMODE__' + printf '%s' '__PIBIN__' if [ "$kind" = secondmate ]; then printf '%s' ' __MODELFLAG____EFFORTFLAG__-e __PITURNEND__ -e __PIWATCH__ "$(__OPINPUT__ encode launch-brief < __BRIEF__)"' else @@ -1233,11 +1221,6 @@ case "$HARNESS" in echo "error: $HARNESS executable not found on PATH; install it or select a different verified harness" >&2 exit 1 } - PI_TUI_MODE= - if pi_supports_tui_mode "$PI_BIN"; then - PI_TUI_MODE=' --tui-mode regular' - fi - LAUNCH=${LAUNCH//__PITUIMODE__/$PI_TUI_MODE} LAUNCH="FM_PI_HARNESS=$HARNESS $LAUNCH" ;; cursor) diff --git a/bin/fm-supervision-lib.sh b/bin/fm-supervision-lib.sh index 3bbb13bdf8..413047be16 100644 --- a/bin/fm-supervision-lib.sh +++ b/bin/fm-supervision-lib.sh @@ -3,9 +3,10 @@ # Usage: . bin/fm-supervision-lib.sh # # Reports whether a firstmate home needs supervision because it has in-flight -# work (a state/.meta exists) or an X-mode relay poll -# (state/x-watch.check.sh), and whether its watcher has a fresh liveness beacon -# (state/.last-watcher-beat, touched every poll cycle, within the grace window). +# work (a state/.meta exists), an X-mode relay poll +# (state/x-watch.check.sh), or pending wake delivery, and whether its watcher has +# a fresh liveness beacon (state/.last-watcher-beat, touched every poll cycle, +# within the grace window). # bin/fm-turnend-guard.sh uses the PID-strict fm_watcher_healthy from # bin/fm-wake-lib.sh for its block decision. bin/fm-guard.sh uses the model-aware # fm_watcher_supervision_verdict (also in bin/fm-wake-lib.sh), which owns what a @@ -25,34 +26,51 @@ fm_sup_stat_mtime() { # Populates, for the state dir at $1: # FM_SUP_IN_FLIGHT count of state/*.meta (in-flight tasks) # FM_SUP_SOURCES count of registered process-to-event sources -# FM_SUP_NEEDED true/false - in-flight work, an X-mode relay poll, or a -# registered event source (a source is a wait on an -# external process, not a task, so it has no metadata) +# FM_SUP_IDENTITY_FINGERPRINT stable fingerprint of task and source identities +# FM_SUP_NEEDED true/false - in-flight work, an X-mode relay poll, a +# registered event source, or pending wake delivery +# (a source is a wait on an external process, not a task, +# so it has no metadata) # FM_SUP_WATCHER_FRESH true/false - a watcher beacon within the grace window # FM_SUP_BEACON_DESC human-readable beacon age, for banners ("never" if absent) -# FM_SUP_QUEUE_PENDING true/false - state/.wake-queue has unread records +# FM_SUP_QUEUE_PENDING true/false - state/.wake-queue has unacknowledged records +# FM_SUP_QUEUE_FINGERPRINT stable fingerprint of the pending wake records # grace-seconds defaults to $FM_GUARD_GRACE, then 300, matching fm-guard.sh. # Always returns 0; callers read the vars, or use fm_supervision_unhealthy below. fm_supervision_status() { - local state=$1 grace=${2:-${FM_GUARD_GRACE:-300}} meta source beat m age + local state=$1 grace=${2:-${FM_GUARD_GRACE:-300}} meta source beat m age identity_records= + local LC_ALL=C FM_SUP_IN_FLIGHT=0 FM_SUP_NEEDED=false FM_SUP_WATCHER_FRESH=false FM_SUP_BEACON_DESC=never FM_SUP_QUEUE_PENDING=false + FM_SUP_QUEUE_FINGERPRINT=none for meta in "$state"/*.meta; do [ -e "$meta" ] || continue FM_SUP_IN_FLIGHT=$((FM_SUP_IN_FLIGHT + 1)) + identity_records="${identity_records}task:${#meta}:$meta;" done FM_SUP_SOURCES=0 for source in "$state"/procevent/*.source; do [ -e "$source" ] || continue FM_SUP_SOURCES=$((FM_SUP_SOURCES + 1)) + identity_records="${identity_records}source:${#source}:$source;" done + FM_SUP_IDENTITY_FINGERPRINT=$(printf '%s' "$identity_records" \ + | cksum 2>/dev/null | awk '{printf "%s-%s", $1, $2}') + [ -n "$FM_SUP_IDENTITY_FINGERPRINT" ] || FM_SUP_IDENTITY_FINGERPRINT=unavailable + if [ -s "$state/.wake-queue" ]; then + FM_SUP_QUEUE_PENDING=true + FM_SUP_QUEUE_FINGERPRINT=$(cksum < "$state/.wake-queue" 2>/dev/null \ + | awk '{printf "%s-%s", $1, $2}') + [ -n "$FM_SUP_QUEUE_FINGERPRINT" ] || FM_SUP_QUEUE_FINGERPRINT=unavailable + fi if [ "$FM_SUP_IN_FLIGHT" -gt 0 ] \ || [ -f "$state/x-watch.check.sh" ] \ - || [ "$FM_SUP_SOURCES" -gt 0 ]; then + || [ "$FM_SUP_SOURCES" -gt 0 ] \ + || [ "$FM_SUP_QUEUE_PENDING" = true ]; then FM_SUP_NEEDED=true fi @@ -68,9 +86,6 @@ fm_supervision_status() { FM_SUP_BEACON_DESC=unknown fi fi - - # shellcheck disable=SC2034 # Read by callers (fm-guard.sh) after sourcing. - [ -s "$state/.wake-queue" ] && FM_SUP_QUEUE_PENDING=true return 0 } diff --git a/bin/fm-turnend-guard.sh b/bin/fm-turnend-guard.sh index f3b4285511..2331d5f06b 100755 --- a/bin/fm-turnend-guard.sh +++ b/bin/fm-turnend-guard.sh @@ -58,10 +58,14 @@ # the first fresh exhausted-failure epoch preserves the bounded progression, # while later fresh failed epochs consume it instead of resetting it; # 3. only when neither materializes is the auto-arm genuinely absent: re-block -# with the repair banner, bounded to FM_CLAUDE_TURNEND_BLOCK_BUDGET -# (default 3) consecutive blocks per session - safely below Claude Code's -# hard 8-consecutive-block override - then allow one loud attended -# fail-open only for an already verified failure episode. +# with the repair banner. Two unchanged no-claim blocks terminate in one +# attended captain escalation instead of an unbounded exchange. A verified +# failure episode keeps its stronger FM_CLAUDE_TURNEND_BLOCK_BUDGET +# progression (default 3, safely below Claude Code's hard 8-consecutive- +# block override) and one-time automatic-mechanism alarm. +# A read-only Claude session whose matching auto-arm defers to another live +# session-lock owner is outside this recovery obligation and exits silently; +# the lock-owning session remains the sole mutable supervision owner. set -u SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -76,6 +80,7 @@ CURSOR_MODE=0 SYNC_WAIT_MS=${FM_CLAUDE_AUTOARM_SYNC_WAIT_MS:-800} EPOCH_FRESH=${FM_CLAUDE_AUTOARM_EPOCH_FRESH:-15} BLOCK_BUDGET=${FM_CLAUDE_TURNEND_BLOCK_BUDGET:-3} +UNCLAIMED_BLOCK_BUDGET=2 case "$SYNC_WAIT_MS" in ''|*[!0-9]*) SYNC_WAIT_MS=800 ;; esac case "$EPOCH_FRESH" in ''|*[!0-9]*|0) EPOCH_FRESH=15 ;; esac case "$BLOCK_BUDGET" in ''|*[!0-9]*|0) BLOCK_BUDGET=3 ;; esac @@ -94,6 +99,8 @@ done . "$SCRIPT_DIR/fm-primary-scope-lib.sh" # shellcheck source=bin/fm-hook-host-lib.sh . "$SCRIPT_DIR/fm-hook-host-lib.sh" +# shellcheck source=bin/fm-session-lock-lib.sh +. "$SCRIPT_DIR/fm-session-lock-lib.sh" # Read the whole turn-end hook payload once; never block on unreadable/absent # stdin. @@ -141,6 +148,19 @@ fi # so this exempts them while guarding every real secondmate home. fm_primary_scope_matches "$FM_ROOT" "$STATE" || exit 0 +# A lock-refused Claude session is read-only and its sibling auto-arm must defer +# to the live owner. Applying the mutable owner's backstop here would create an +# impossible recovery loop: this session cannot arm, while the guard blocks it +# because it did not arm. Keep malformed, missing, stale, and self-owned locks +# on the ordinary guarded path; only a proven foreign live owner is exempt. +if [ "$CLAUDE_MODE" -eq 1 ] && ! fm_session_lock_owned_by_self "$STATE"; then + SESSION_LOCK_PID=$(cat "$STATE/.lock" 2>/dev/null || true) + case "$SESSION_LOCK_PID" in + ''|*[!0-9]*) : ;; + *) fm_harness_pid_alive "$SESSION_LOCK_PID" && exit 0 ;; + esac +fi + # --- the actual predicate ---------------------------------------------------- # shellcheck source=bin/fm-wake-lib.sh . "$SCRIPT_DIR/fm-wake-lib.sh" @@ -150,11 +170,12 @@ BUDGET_LOCK="$STATE/.turnend-claude-blocks.lock" OWNER_LOCK="$STATE/.claude-autoarm.lock" FAILURE_NOTICE="$STATE/.claude-autoarm-failure-notified" FAILURE_ALARM="$STATE/.claude-autoarm-failure-alarmed" +ESCALATION_MARKER="$STATE/.turnend-claude-escalated" SESSION_ID=$(printf '%s' "$PAYLOAD" | jq -r '.session_id // "unknown"' 2>/dev/null || printf 'unknown') budget_reset() { [ "$CLAUDE_MODE" -eq 1 ] || return 0 fm_lock_try_acquire "$BUDGET_LOCK" || return 0 - rm -f "$BUDGET_FILE" 2>/dev/null || true + rm -f "$BUDGET_FILE" "$ESCALATION_MARKER" 2>/dev/null || true fm_lock_release "$BUDGET_LOCK" } @@ -185,6 +206,8 @@ block_stop() { printf '● %s task(s) in flight, but no live watcher holds this home lock (last beat: %s).\n' "$FM_SUP_IN_FLIGHT" "$FM_SUP_BEACON_DESC" elif [ "$FM_SUP_SOURCES" -gt 0 ]; then printf '● %s process-event source(s) registered, but no live watcher holds this home lock (last beat: %s).\n' "$FM_SUP_SOURCES" "$FM_SUP_BEACON_DESC" + elif [ "$FM_SUP_QUEUE_PENDING" = true ]; then + printf '● Durable queued wake delivery pending, but no live watcher holds this home lock (last beat: %s).\n' "$FM_SUP_BEACON_DESC" else printf '● X-mode relay polling needs supervision, but no live watcher holds this home lock (last beat: %s).\n' "$FM_SUP_BEACON_DESC" fi @@ -205,22 +228,38 @@ fi # The Stop-owned auto-arm fires on the same Stop event. Give it a brief bounded # window to prove it owns recovery for this event epoch before consuming one of # Claude's bounded continuations. -budget_account_current_epoch() { - local current_epoch outcome old_session old_count old_epoch tmp initialized +budget_account_current_epoch() { # [observe|block] + local mode=${1:-observe} current_epoch outcome old_session old_count old_epoch + local old_reblocks old_signature signature x_mode afk tmp initialized + case "$mode" in observe|block) : ;; *) return 1 ;; esac fm_lock_try_acquire "$BUDGET_LOCK" || return 1 current_epoch=$(sed -n 's/^epoch=\([0-9][0-9]*\) .*/\1/p' "$STATE/.claude-autoarm-epoch" 2>/dev/null || true) outcome=$(sed -n 's/^.*outcome=\([a-z][a-z-]*\) .*$/\1/p' "$STATE/.claude-autoarm-epoch" 2>/dev/null || true) + x_mode=0 + [ -f "$CONFIG/x-mode.env" ] && x_mode=1 + afk=0 + [ -e "$STATE/.afk" ] && afk=1 + signature="inflight=$FM_SUP_IN_FLIGHT:sources=$FM_SUP_SOURCES:identities=$FM_SUP_IDENTITY_FINGERPRINT:queue=$FM_SUP_QUEUE_FINGERPRINT:x=$x_mode:afk=$afk:epoch=${current_epoch:-none}:outcome=${outcome:-none}" initialized=0 COUNT=0 + REBLOCK_COUNT=0 + REBLOCK_SIGNATURE= if [ -f "$BUDGET_FILE" ]; then old_session=$(sed -n '1s/^session=//p' "$BUDGET_FILE" 2>/dev/null || true) old_count=$(sed -n '2s/^count=//p' "$BUDGET_FILE" 2>/dev/null || true) old_epoch=$(sed -n '3s/^epoch=//p' "$BUDGET_FILE" 2>/dev/null || true) + old_reblocks=$(sed -n '4s/^reblocks=//p' "$BUDGET_FILE" 2>/dev/null || true) + old_signature=$(sed -n '5s/^signature=//p' "$BUDGET_FILE" 2>/dev/null || true) case "$old_count" in ''|*[!0-9]*) old_count=0 ;; esac + case "$old_reblocks" in + ''|*[!0-9]*) old_reblocks=0 ;; + esac if [ "$old_session" = "$SESSION_ID" ]; then COUNT=$old_count + REBLOCK_COUNT=$old_reblocks + REBLOCK_SIGNATURE=$old_signature if [ -n "$current_epoch" ] && [ "$old_epoch" = "$current_epoch" ]; then : else @@ -241,8 +280,18 @@ budget_account_current_epoch() { *) COUNT=1 ;; esac fi + if [ "$mode" = block ]; then + if [ "${old_session:-}" = "$SESSION_ID" ] && [ "${old_signature:-}" = "$signature" ]; then + REBLOCK_COUNT=$((REBLOCK_COUNT + 1)) + else + REBLOCK_COUNT=1 + rm -f "$ESCALATION_MARKER" 2>/dev/null || true + fi + REBLOCK_SIGNATURE=$signature + fi tmp="$BUDGET_FILE.tmp.$$" - if ! printf 'session=%s\ncount=%s\nepoch=%s\n' "$SESSION_ID" "$COUNT" "$current_epoch" > "$tmp" 2>/dev/null \ + if ! printf 'session=%s\ncount=%s\nepoch=%s\nreblocks=%s\nsignature=%s\n' \ + "$SESSION_ID" "$COUNT" "$current_epoch" "$REBLOCK_COUNT" "$REBLOCK_SIGNATURE" > "$tmp" 2>/dev/null \ || ! mv -f "$tmp" "$BUDGET_FILE" 2>/dev/null; then rm -f "$tmp" 2>/dev/null || true fm_lock_release "$BUDGET_LOCK" @@ -344,6 +393,73 @@ terminal_fail_open() { return 0 } +terminal_unclaimed_escalation() { + local pid role old_session old_reblocks old_signature + [ "$REBLOCK_COUNT" -ge "$UNCLAIMED_BLOCK_BUDGET" ] || return 1 + [ ! -e "$STATE/.afk" ] || return 1 + failure_state_present && return 1 + [ ! -e "$ESCALATION_MARKER" ] || return 3 + if ! fm_lock_try_acquire "$OWNER_LOCK"; then + pid=$(cat "$OWNER_LOCK/pid" 2>/dev/null || true) + role=$(fm_lock_role "$OWNER_LOCK" 2>/dev/null || true) + if fm_pid_alive "$pid" && [ "$role" = autoarm ]; then + return 2 + fi + return 1 + fi + if ! fm_lock_set_role "$OWNER_LOCK" terminal-escalation; then + fm_lock_release "$OWNER_LOCK" + return 1 + fi + if ! fm_lock_try_acquire "$BUDGET_LOCK"; then + fm_lock_release "$OWNER_LOCK" + return 1 + fi + old_session=$(sed -n '1s/^session=//p' "$BUDGET_FILE" 2>/dev/null || true) + old_reblocks=$(sed -n '4s/^reblocks=//p' "$BUDGET_FILE" 2>/dev/null || true) + old_signature=$(sed -n '5s/^signature=//p' "$BUDGET_FILE" 2>/dev/null || true) + case "$old_reblocks" in + ''|*[!0-9]*) old_reblocks=0 ;; + esac + role=$(fm_lock_role "$OWNER_LOCK" 2>/dev/null || true) + if [ "$role" != terminal-escalation ] || [ "$old_session" != "$SESSION_ID" ] \ + || [ "$old_reblocks" -lt "$UNCLAIMED_BLOCK_BUDGET" ] \ + || [ "$old_signature" != "$REBLOCK_SIGNATURE" ] || failure_state_present \ + || [ -e "$ESCALATION_MARKER" ]; then + fm_lock_release "$BUDGET_LOCK" + fm_lock_release "$OWNER_LOCK" + return 1 + fi + if fm_watcher_healthy "$STATE" "$WATCH" "$GRACE" "$FM_HOME"; then + if ! fm_failure_episode_reset "$STATE" held; then + fm_lock_release "$BUDGET_LOCK" + fm_lock_release "$OWNER_LOCK" + return 1 + fi + fm_lock_release "$BUDGET_LOCK" + fm_lock_release "$OWNER_LOCK" + return 2 + fi + if ! (set -C; : > "$ESCALATION_MARKER") 2>/dev/null; then + fm_lock_release "$BUDGET_LOCK" + fm_lock_release "$OWNER_LOCK" + return 1 + fi + fm_lock_release "$BUDGET_LOCK" + fm_lock_release "$OWNER_LOCK" + return 0 +} + +failure_state_present() { + local outcome + [ -e "$FAILURE_NOTICE" ] && return 0 + outcome=$(sed -n 's/^.*outcome=\([a-z][a-z-]*\) .*$/\1/p' "$STATE/.claude-autoarm-epoch" 2>/dev/null || true) + case "$outcome" in + failed|failed-suppressed) return 0 ;; + *) return 1 ;; + esac +} + failure_episode_verified() { local outcome [ ! -e "$STATE/.afk" ] || return 1 @@ -375,7 +491,12 @@ fi # The auto-arm genuinely failed to establish: consume the bounded re-block # budget before considering the verified one-time attended fail-open. -budget_account_current_epoch || block_stop +fm_supervision_status "$STATE" "$GRACE" +if [ "$FM_SUP_NEEDED" = false ]; then + [ -e "$FAILURE_NOTICE" ] || budget_reset + exit 0 +fi +budget_account_current_epoch block || block_stop terminal_fail_open terminal_status=$? if [ "$terminal_status" -eq 0 ]; then @@ -383,6 +504,8 @@ if [ "$terminal_status" -eq 0 ]; then NEED_DESC="$FM_SUP_IN_FLIGHT task(s) in flight" elif [ "$FM_SUP_SOURCES" -gt 0 ]; then NEED_DESC="$FM_SUP_SOURCES process-event source(s) registered" + elif [ "$FM_SUP_QUEUE_PENDING" = true ]; then + NEED_DESC="queued wake delivery pending" else NEED_DESC="X-mode relay polling active" fi @@ -390,4 +513,11 @@ if [ "$terminal_status" -eq 0 ]; then exit 0 fi [ "$terminal_status" -eq 2 ] && exit 0 +terminal_unclaimed_escalation +terminal_status=$? +if [ "$terminal_status" -eq 0 ]; then + printf '%s\n' '{"systemMessage":"FIRSTMATE NEEDS YOUR DECISION: automatic supervision did not start after two identical blocked turn ends. Should I keep this session open while recovery is diagnosed, or end while work is unsupervised?"}' + exit 0 +fi +case "$terminal_status" in 2|3) exit 0 ;; esac block_stop diff --git a/bin/fm-wake-lib.sh b/bin/fm-wake-lib.sh index ff32d88196..fffa32d171 100755 --- a/bin/fm-wake-lib.sh +++ b/bin/fm-wake-lib.sh @@ -299,7 +299,7 @@ fm_lock_clean_known_files() { fm_lock_set_role() { local lockdir=$1 role=$2 current pid back case "$role" in - autoarm|terminal-check) : ;; + autoarm|terminal-check|terminal-escalation) : ;; *) return 1 ;; esac current=${BASHPID:-$$} @@ -903,6 +903,7 @@ fm_failure_episode_reset() { esac for path in \ "$state/.turnend-claude-blocks" \ + "$state/.turnend-claude-escalated" \ "$state/.claude-autoarm-failure-notified" \ "$state/.claude-autoarm-failure-alarmed" do @@ -913,6 +914,7 @@ fm_failure_episode_reset() { done if ! rm -f \ "$state/.turnend-claude-blocks" \ + "$state/.turnend-claude-escalated" \ "$state/.claude-autoarm-failure-notified" \ "$state/.claude-autoarm-failure-alarmed" \ 2>/dev/null; then @@ -1194,7 +1196,7 @@ fm_wake_latest_event() { # } # Print supplemental drain-time context only after the caller has committed the -# raw queue consumption and released the append lock. +# raw queue presentation and released the append lock. fm_wake_print_annotations() { # [] local rows=$1 snapshot=${2:-} manifest status_key mode path prefix line task endpoint local snapshot_task snapshot_endpoint _snapshot_ident offset last_event event_line diff --git a/docs/architecture.md b/docs/architecture.md index afca3208d7..a2cf552442 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -29,7 +29,7 @@ Separately from heartbeat backoff and wedge handling, the watcher poll runs `bin In each home the scan considers only that home's long-inactive direct ordinary crewmates, excludes captain-held work, and accepts only `done` or `failed` from `bin/fm-crew-state.sh`. A secondmate retains a durable receipt for its idempotent report through the established parent route, and main-home captain presentation retains a separate receipt; neither path performs a forge or PR check. Absorbed wakes advance their suppression markers, log to `state/.watch-triage.log`, and keep the watcher blocking without a queue record or LLM turn. -Each `fm-wake-drain.sh` presentation runs the same liveness guard as the supervision scripts, so a lapsed watcher chain surfaces even on a turn that only handles queued wakes. +Each `fm-wake-drain.sh` presentation runs the same liveness guard as the supervision scripts, and pending wake delivery remains a supervision need until post-handling acknowledgement, so a lapsed watcher chain surfaces even on a turn that only handles queued wakes. Routine watcher polling, supervision no-ops, elapsed waiting time, and absorbed benign wakes stay silent. A declared external wait trades that silence for one bounded recheck per pause window, so a forgotten pause cannot remain invisible indefinitely. Crew status files are append-only wake-event logs, not current-state fields. @@ -78,10 +78,10 @@ It suppresses failed-looking closes when the same identity-matched watcher is he Cursor's `bin/fm-turnend-guard-cursor.sh` hook is the same between-turns shape in one synchronous step: it parks the awaited `stop` hook on the arm wrapper and translates an actionable close into one `followup_message`, with a generation baton that makes an older park still running after the next `stop` claim stand down instead of leaking a stale duplicate wake. The existing turn-end guard remains the final backstop for every harness-engine protocol, with pi-signed sharing Pi's protocol, the `--claude` mode cooperating with the auto-arm claim, and Cursor's `--cursor` mode rendering a block as one bounded follow-up because its `stop` step cannot be blocked. Its `--restart` mode signals only the watcher recorded in the current home's `state/.watch.lock`, so restarting one home cannot kill sibling secondmate watchers. -A pull-based guard (`bin/fm-guard.sh`) warns through supervision tool output if the primary checkout is tangled, if work, process-event sources, or Relay polling has an unhealthy model-aware supervision verdict, or if queued wakes are waiting to be drained. +A pull-based guard (`bin/fm-guard.sh`) warns through supervision tool output if the primary checkout is tangled, if work, process-event sources, Relay polling, or queued delivery has an unhealthy model-aware supervision verdict, or if queued wakes still await post-handling acknowledgement. The drain script calls that guard after presenting the queue; records remain durable, and may keep the queued-wakes warning visible, until the exact generation-bound acknowledgement printed by the drain succeeds after handling. It leads with a prominent bordered tangle banner, while `bin/fm-guard.sh` owns the watcher-down banner and reminder policy so repeated guarded commands stay noisy without reprinting the full banner in the same episode. -On every verified primary harness, tracked hook integration gives the primary session a push-based backstop: when work, a process-event source, or Relay polling needs supervision and no identity-matched watcher lock with a fresh beacon is live, blocking-capable Stop hooks block and nonblocking turn-end integrations force one bounded follow-up. +On every verified primary harness, tracked hook integration gives the primary session a push-based backstop: when work, a process-event source, Relay polling, or queued delivery needs supervision and no identity-matched watcher lock with a fresh beacon is live, blocking-capable Stop hooks block and nonblocking turn-end integrations force one bounded follow-up. The guard covers the main primary and genuinely marked secondmate homes, exempts child crewmate/scout worktrees, is loop-safe per harness, and is documented in [turnend-guard.md](turnend-guard.md). A presence-gated sub-supervisor (`bin/fm-supervise-daemon.sh`) extends this for walk-away supervision: the `/afk` skill starts it through the tracked foreground helper `bin/fm-afk-start.sh`, after which the watcher reverts to daemon-managed one-shot mode and the daemon self-handles routine wakes in bash. diff --git a/docs/configuration.md b/docs/configuration.md index 78ae19bd56..3c86a77067 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -562,7 +562,7 @@ FM_GUARD_GRACE=300 # seconds before guard warnings, arm health checks, and FM_CLAUDE_AUTOARM_ATTEMPTS=2 # bounded Stop-owned arm attempts per Claude auto-arm cycle; accepted values are 1, 2, or 3 FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=800 # milliseconds the --claude turn-end guard waits for watcher health, a role-verified Stop auto-arm claim, or a fresh epoch before deciding recovery ownership or failure progression FM_CLAUDE_AUTOARM_EPOCH_FRESH=15 # seconds a recorded auto-arm outcome remains eligible for the current event epoch's recovery or failure decision -FM_CLAUDE_TURNEND_BLOCK_BUDGET=3 # consecutive --claude guard re-blocks before the verified one-time attended fail-open; safely below Claude Code's 8-block override +FM_CLAUDE_TURNEND_BLOCK_BUDGET=3 # fresh verified automatic-failure epochs before the one-time attended fail-open; ordinary no-claim blocks use the fixed two-identical-block escalation FM_ARM_CONFIRM_TIMEOUT=10 # seconds fm-watch-arm waits to confirm a fresh watcher before reporting FAILED; default 30 on Git Bash/MSYS FM_ARM_ATTACH_POLL=0.5 # seconds between checks while fm-watch-arm is attached to an existing healthy watcher cycle FM_OPENCODE_ARM_READY_TIMEOUT_MS=12000 # milliseconds the OpenCode primary watcher plugin waits for an arm attempt to report started, healthy, wake, or failure; default 35000 on Windows to stay above the MSYS confirm budget diff --git a/docs/supervision-protocols/claude.md b/docs/supervision-protocols/claude.md index 1e5033a55e..04a7494215 100644 --- a/docs/supervision-protocols/claude.md +++ b/docs/supervision-protocols/claude.md @@ -12,9 +12,11 @@ When this session owns supervision and away mode is not active: 4. On the one `Stop hook feedback` automatic-mechanism failure notice (`firstmate watcher auto-arm FAILED ...`), drain, inspect the automatic mechanism failure, and do not turn the notice into a repeating manual-arm loop. 5. If the Stop hook does not claim the home or reports an exhausted failure, inspect its registration and watcher startup path before ending blind. Keep the Stop-owned automatic mechanism as the only Claude arm owner. + On the second genuinely identical no-claim observation, the guard itself ends the continuation loop with exactly one captain-facing question; do not synthesize or repeat that question. + Every subsequent unchanged Stop passes silently, while changed supervision evidence starts a fresh count as specified in [`turnend-guard.md`](../turnend-guard.md). 6. Treat `watcher: started ...` and `watcher: attached ...` inside automatic arm output as proof that one live cycle exists. On attach, the arm follows verified identity-matched successors instead of exiting when the first cycle ends. -7. The durable wake queue preserves actionable events between a rewake and the next Stop-launched arm, while the bounded turn-end guard prevents a blind Stop when recovery did not start. +7. The durable wake queue preserves actionable events between a rewake and the next Stop-launched arm and remains a supervision need until the printed post-handling acknowledgement consumes them, while the bounded turn-end guard prevents a blind Stop when recovery did not start. No PreToolUse hook denies fleet commands based on watcher status. [`watcher-continuity.md`](../watcher-continuity.md) owns the exact session-lock recovery boundary. 8. The turn-end guard (`bin/fm-turnend-guard.sh --claude`) remains the final backstop. diff --git a/docs/turnend-guard.md b/docs/turnend-guard.md index 3620230f83..db3bd3105a 100644 --- a/docs/turnend-guard.md +++ b/docs/turnend-guard.md @@ -13,7 +13,7 @@ Do not infer this guard's scope, loop safety, or compatibility tradeoffs for tho `bin/fm-guard.sh` is a pull-based warning that runs only when another supervision command invokes it. The turn-end guard closes the remaining gap at the primary's own turn boundary. -When work, a process-event source, or Relay polling needs supervision at that boundary and no identity-matched watcher has a fresh beacon, the harness integration must either block the turn end or force one bounded follow-up that uses the recovery instruction from the emitted session-start protocol. +When work, a process-event source, Relay polling, or queued wake delivery needs supervision at that boundary and no identity-matched watcher has a fresh beacon, the harness integration must either block the turn end or force one bounded follow-up that uses the recovery instruction from the emitted session-start protocol. The mid-turn pull warning uses the model-aware supervision verdict described below, while the turn-end guard keeps the PID-strict watcher predicate. The guard remains a backstop; [`watcher-continuity.md`](watcher-continuity.md) owns normal continuity. @@ -28,6 +28,7 @@ It also requires `AGENTS.md`, `bin/`, and the effective state directory. For an in-scope primary, the guard counts in-flight work from `state/*.meta`. Registered `state/procevent/*.source` records also require supervision even though they have no task metadata. +Non-empty `state/.wake-queue` delivery remains a supervision need after its producing task or process source retires and until post-handling acknowledgement consumes the queued records. The default cross-harness mode exits silently with no supervision need. Every mode treats `state/x-watch.check.sh` as supervision need, so Relay polling remains guarded without an in-flight task. Otherwise it calls `fm_watcher_healthy [grace-seconds] [home]` from `bin/fm-wake-lib.sh`, the same PID-strict identity-matched lock and fresh-beacon check used by `bin/fm-watch-arm.sh`: a stale beacon blocks even when a watcher pid is live, and a fresh leftover beacon blocks when the lock is missing, dead, or identity-mismatched. @@ -71,13 +72,19 @@ Both payloads carry `stop_hook_active`. In the default Codex mode, a true value lets the second stop finish after one forced continuation. Claude runs the guard with `--claude`, which ignores `stop_hook_active` and cooperates with the Stop-owned auto-arm. +Before applying the supervision predicate, Claude mode applies the auto-arm's session-lock boundary: when another proven-live harness session owns `state/.lock`, this lock-refused session is read-only, so its guard exits without mutating the lock owner's block budget. +Missing, malformed, stale, and self-owned session locks remain on the ordinary guarded path. Claude Code sets `stop_hook_active=true` on every stop after any stop-hook continuation, including `asyncRewake` rewakes, which re-opened the 2026-07-21 blind window under the default one-shot behavior. The Claude mode waits up to `FM_CLAUDE_AUTOARM_SYNC_WAIT_MS` (default 800 milliseconds) and allows the stop when the watcher is healthy, `state/.claude-autoarm.lock` has a live `autoarm` role owner whose eventual failure must exit 2, or `state/.claude-autoarm-epoch` contains a fresh actionable rewake owned by this event epoch. Fresh `failed` and `failed-suppressed` outcomes enter or advance the failure progression instead of acting as unconditional recovery proof. The auto-arm itself rechecks the healthy watcher predicate and retries a bounded number of times before reporting a genuine failure. The first fresh exhausted-failure epoch preserves its handoff without consuming a blocked-stop count, while later fresh failed epochs advance the same monotonic progression instead of resetting it. -When none of those proofs appears, it re-blocks up to `FM_CLAUDE_TURNEND_BLOCK_BUDGET` times (default 3, below Claude's 8-block override). -In Claude mode, positive watcher recovery clears the block budget, failure notice, and attended alarm together under the existing budget lock before either hook reports ordinary recovery. +When none of those proofs appears and no automatic-failure state exists, the first no-claim observation blocks for that session and evidence signature. +On the second genuinely identical observation, the guard emits exactly one captain-facing `systemMessage` question and ends the continuation loop itself. +Every later unchanged Stop passes silently through `state/.turnend-claude-escalated`, while any evidence-signature change resets the count, including task or process-source identities, queued-delivery state, Relay or AFK state, and auto-arm epoch outcome. +Only a supervision need that disappears during the claim wait without leaving a queued wake clears the episode before passing. +A verified automatic failure retains the separate `FM_CLAUDE_TURNEND_BLOCK_BUDGET` progression (default 3, below Claude's 8-block override) and its stronger attended alarm. +In Claude mode, positive watcher recovery clears the block budget, one-shot escalation, failure notice, and attended alarm together under the existing budget lock before either hook reports ordinary recovery. The one loud attended fail-open is available only when the auto-arm has recorded an exhausted failure, its one notice is already consumed, the block budget is exhausted, and a final check finds neither a healthy watcher nor an automatic continuation. Each epoch identity is accounted at most once under the budget lock. Whenever both coordination locks are needed, positive auto-arm recovery and the terminal check acquire the auto-arm owner lock before the budget lock. @@ -129,7 +136,7 @@ That warning uses `bin/fm-supervision-instructions.sh --repair-line`, so it alwa ## Compatibility limits - Child crewmate and scout worktrees are outside scope. -- A valid secondmate home is in scope; an idle secondmate endpoint with no Relay poll remains healthy because it has no supervision need. +- A valid secondmate home is in scope; an idle secondmate endpoint with no task, process source, Relay poll, or queued delivery remains healthy because it has no supervision need. - The blocking and bounded-follow-up mechanisms are limited to the primary integrations listed above. - OpenCode headless mode and untrusted Grok project hooks remain fail-open at the host boundary. - Cursor's `stop` step does not fire in headless `cursor-agent -p`, the same class of limit as OpenCode headless; firstmate primaries run interactive. @@ -146,7 +153,7 @@ That warning uses `bin/fm-supervision-instructions.sh --repair-line`, so it alwa ## Regression coverage -`tests/fm-turnend-guard.test.sh` covers the predicate, main and secondmate primary scope, child-worktree exclusion, `FM_HOME` and `FM_STATE_OVERRIDE` precedence, the live-lock and fresh-beacon guard predicate, the cooperative `--claude` claim wait, monotonic failed-epoch progression, bounded attended fail-open, post-alarm continuation suppression, positive recovery reset, Pi logical-run latching, missing-`jq` behavior, all five primary registrations, Grok native and legacy selection, typed field precedence, malformed input, and exactly-one-path safety. +`tests/fm-turnend-guard.test.sh` covers the predicate, main and secondmate primary scope, child-worktree exclusion, `FM_HOME` and `FM_STATE_OVERRIDE` precedence, the live-lock and fresh-beacon guard predicate, the cooperative `--claude` claim wait, foreign-live-owner read-only exit, repeated-block captain escalation, monotonic failed-epoch progression, bounded attended fail-open, post-alarm continuation suppression, positive recovery reset, Pi logical-run latching, missing-`jq` behavior, all six primary registrations, Grok native and legacy selection, typed field precedence, malformed input, and exactly-one-path safety. `tests/fm-guard-stale-banner.test.sh` covers the pull-guard predicate, including the persistent-model fresh-leftover-beacon negative control, the auto-arm model's healthy fresh-beacon-without-a-watcher case and stale-beacon alarm, and the extension model's live-watcher path, ownership-qualified fresh hand-off, held-lock failures, independently broken ownership signals, stale-beacon alarm, queued-wake warning, and Pi and pi-signed harness routing. It also covers true-reason banner wording and reason-keyed episode dedup surviving a beacon mtime change. `tests/fm-cursor-primary.test.sh` covers the Cursor park end to end over real processes with no harness installed: each tracked Claude-shaped entrypoint standing down on a Cursor payload, both follow-up sources, the bounded repair nag and its reset, the nested loop bounds, supersession, away-mode and lock-ownership inertness, child-worktree exclusion, and that the adapter never exits 2. @@ -154,4 +161,4 @@ It also covers true-reason banner wording and reason-keyed episode dedup survivi `tests/fm-kimi-harness.test.sh` covers the separate Kimi crew hook's format preservation, idempotence, refusal cases, token guard, spawn registration, and teardown cleanup. `tests/fm-supervision-instructions.test.sh` covers recovery-line ownership and pi-signed's identity-preserving reuse of Pi's protocol. `FM_PI_LIVE_E2E=1 tests/fm-pi-primary-live-e2e.test.sh` is the opt-in isolated Pi path. -[`verification/supervision.md`](verification/supervision.md#turn-end-guard) records the active cross-harness empirical evidence, including the 2026-07-24 Claude `asyncRewake` revalidation. +[`verification/supervision.md`](verification/supervision.md#turn-end-guard) records the active cross-harness empirical evidence, including the 2026-08-14 two-session Claude ownership and `asyncRewake` revalidation. diff --git a/docs/verification/process-event-sources.md b/docs/verification/process-event-sources.md index 55da9098a6..779bea2369 100644 --- a/docs/verification/process-event-sources.md +++ b/docs/verification/process-event-sources.md @@ -78,7 +78,7 @@ Exercised by `tests/fm-procevent.test.sh` against a fake blocking source whose c | capture before publication | the captured result exists at `0600` and its event names its committed sequence only afterward | | proactive delivery of a captured result | a real capture into an isolated home queues its `check` record, and a healthy watcher with a fresh beacon then exits reporting that queued result as an actionable check, before any manual drain | | single delivery per source and sequence | after that first proactive wake, a still-unhandled result keeps being re-announced onto the durable queue but never wakes the watcher again; once existing records receive the drain's post-handling acknowledgement and the source result is acknowledged, it is neither re-announced nor reported | -| proactive-delivery crash and drain boundaries | dotted and underscored source ids at the same sequence receive distinct markers; a concurrent drain cannot consume between queue revalidation and marker commit; failed output, failed marker commit, and a crash before marker commit leave replay available, while successful output still ends the actionable cycle and a crash after marker commit suppresses a duplicate | +| proactive-delivery crash and drain boundaries | dotted and underscored source ids at the same sequence receive distinct markers; a concurrent presentation cannot remove the still-unacknowledged row between queue revalidation and marker commit; failed output, failed marker commit, and a crash before marker commit leave replay available, while successful output still ends the actionable cycle and a crash after marker commit suppresses a duplicate | | adapter-owned terminal verdict | two fixture adapters - one that ends on any result, one with no terminal knowledge - decide the outcome alone: the first has its registration and claim retired automatically after one capture and is never restarted, the second stays armed | | adapter-owned application of a captured result | a remote-secondmate reply captured through the real relay in an isolated home reaches that secondmate's local status mirror, settles its correlated pending-reply expectation, re-arms the next cursor-anchored source, and is acknowledged, with no handler step or duplicate `check` wake; its new mirrored bytes remain visible to the watcher's signal gate, while a cursor-loss whole-log recapture that adds no bytes is acknowledged quietly; for an already-escalated request, the same path closes the exact decision so the open-decision fold clears and remains clear; a capture whose adapter application fails because local storage for a referenced remote document is obstructed is left unacknowledged and receives the fallback `check` wake, and the handler's own `handle` still applies it in full after storage recovers | | terminal retirement preserves the result | the retired source's captured output, its announced event, its handled acknowledgement, and later explicit `retire` all still behave normally | diff --git a/docs/verification/supervision.md b/docs/verification/supervision.md index d0837023d3..838b4f818b 100644 --- a/docs/verification/supervision.md +++ b/docs/verification/supervision.md @@ -207,11 +207,11 @@ tests/fm-crew-state.test.sh ## Turn-end guard -The blocking and bounded-follow-up mechanisms were validated across six harnesses on 2026-07-08 through 2026-08-13, with Claude's replacement Stop-owned path revalidated on 2026-07-24 and Cursor's stop-hook park validated on 2026-08-13. +The blocking and bounded-follow-up mechanisms were validated across six harnesses on 2026-07-08 through 2026-08-14, with Claude's replacement Stop-owned path revalidated on 2026-08-14 and Cursor's stop-hook park validated on 2026-08-13. | Harness | Version verified | Mechanism | Observed result | | --- | --- | --- | --- | -| Claude | 2.1.219 | Cooperative blocking `Stop` guard plus `asyncRewake` auto-arm | A fresh unsupervised session ran session start first, reclaimed a stale dead-owner lock, completed two tokenless rewake cycles with no model arm command or guard continuation, and left a competing live owner unchanged. | +| Claude | 2.1.232 | Cooperative blocking `Stop` guard plus `asyncRewake` auto-arm | Two real sessions shared an isolated home: the read-only session traced the foreign live-owner gate and finished without a guard loop, then the lock-owning session restored supervision and delivered an actionable rewake without human intervention. | | Codex | 0.142.1 | Blocking `Stop` hook | Hook process root stayed anchored to the trusted checkout and one continuation ran. | | OpenCode | 1.17.6 | Passive `session.idle` callback | Throwing could not block, while `promptAsync` scheduled one TUI follow-up; headless remained fail-open. | | Pi | 0.80.5 | Passive `agent_settled` callback | Exactly one guard follow-up ran for an unhealthy cycle, with no recursion across tool turns. | @@ -296,7 +296,10 @@ Harness identity is read from the executable path and `argv[0]` as well as the c The same suite ingests a keyed remote-secondmate parent reply through the real adapter, establishes the incremental OPEN DECISIONS cursor, interrupts supervision, and proves re-arm replays every unacknowledged queue row plus the still-open decision through the ordinary drain path. It also covers decision-only recovery, interrupted handling, handling-window generation reuse, non-fatal moved-generation acknowledgement with sequence-bounded consumption, and a persistent successor remaining live after recovery is acknowledged. -The Claude product live path ran with Claude Code 2.1.219 on 2026-07-24: +The Claude product live path ran with Claude Code 2.1.232 on 2026-08-14. +Claude's current [hooks reference](https://code.claude.com/docs/en/hooks), read the same day, states that all matching hooks run in parallel, that Stop exit 2 prevents stopping and continues the conversation, and that `asyncRewake` wakes Claude on exit 2; it documents no sibling cancellation that would support the earlier short-circuit explanation. +The live check deliberately separated the competing session from the lock owner, which is the condition that falsified that earlier hook-order explanation: the blocked Stop produced an auto-arm entry trace naming `gate-live-session-owner`, while a lock-owning Stop delivered `asyncRewake` normally. +An absent entry trace on the blocked Stop would have falsified the identity-gate diagnosis; a claimed owner cycle without delivered `Stop hook feedback` would have supported the discarded-rewake candidate. ```sh claude --version @@ -306,10 +309,50 @@ FM_CLAUDE_LIVE_E2E=1 tests/fm-claude-stop-autoarm-live-e2e.test.sh Observed output: ```text -2.1.219 (Claude Code) -ok - Claude 2.1.219 (Claude Code) live E2E reclaimed a stale session lock through session start, completed two tokenless Stop-owned rewake cycles, and preserved the competing-live-owner boundary +2.1.232 (Claude Code) +ok - Claude 2.1.232 (Claude Code) live E2E let the read-only competing session finish, then restored supervision from the lock-owning Stop hook without human intervention ``` +The two-session regression was also required to fail against its immediate unfixed parent, `fe30ee2e2ccf678bba877659e47bae71318a5fab`, on 2026-08-14. +The portable control kept the current real-process regression and shared test helper while restoring the parent implementation. + +```sh +test "$(git -C .review-unfixed-stop-guard rev-parse --show-toplevel)" = "$PWD/.review-unfixed-stop-guard" && rm -rf "$PWD/.review-unfixed-stop-guard" +git clone -q . .review-unfixed-stop-guard +git -C .review-unfixed-stop-guard checkout -q fe30ee2e2ccf678bba877659e47bae71318a5fab +cp tests/fm-turnend-guard.test.sh tests/fm-claude-stop-autoarm-live-e2e.test.sh tests/lib.sh .review-unfixed-stop-guard/tests/ +(cd .review-unfixed-stop-guard && bash -o pipefail -c 'tests/fm-turnend-guard.test.sh 2>&1 | tail -8') +``` + +Observed output and exit status `1`: + +```text +ok - tracked .claude/settings.json entries: 5 inert under grok, the documented subagent exception still armed, all live under Claude +ok - .codex/hooks.json: Stop hook uses hook process root when payload cwd is outside +ok - .codex/hooks.json: Stop hook ignores nested git root guard scripts +ok - .opencode primary plugin: guard path is anchored to worktree, not directory +ok - .pi primary extension: no-tool and multi-tool runs each inject exactly one guard follow-up +ok - .pi primary extension: delivery failure resets the logical-run latch +ok - fm-turnend-guard --claude: re-blocks a loop-guarded stop while unhealthy and unclaimed (incident regression) +not ok - a read-only session must not be trapped by a guard whose matching auto-arm cannot own recovery: expected exit 0, got 2 +``` + +The real-Claude control used the same parent fixture and the current env-gated live guard. +The test-only gate bypass is confined to its disposable Claude processes so the live guard can execute from a no-mistakes validation worktree. + +```sh +cp tests/fm-claude-stop-autoarm-live-e2e.test.sh .review-unfixed-stop-guard/tests/ +(cd .review-unfixed-stop-guard && bash -o pipefail -c "FM_CLAUDE_LIVE_E2E=1 tests/fm-claude-stop-autoarm-live-e2e.test.sh 2>&1 | grep '^not ok -'") +``` + +Observed output and exit status `1`: + +```text +not ok - read-only Claude session was trapped by the blind-turn guard: session=fa5c402a-511c-4cf2-b323-a9a5da85b70c +``` + +The corresponding green live result is recorded immediately above, and the green portable suite result is recorded in the focused 2026-08-14 run below. + Current entry points: ```sh @@ -351,6 +394,30 @@ fm-doc-audience-check: ok surfaces=64 local_links=188 FM_TEST_SUMMARY total=4 failed=0 skipped_gate=0 duration_ms=80078 ``` +The foreign-session Stop-loop correction, bounded entry trace, and one-shot repeated-block escalation were verified on 2026-08-14 with ShellCheck 0.11.0. +The portable suite uses real operating-system processes without a vendor harness, while the credentialed live guard above supplies the separate Claude-dependent verdict. + +```sh +bin/fm-lint.sh +bin/fm-doc-audience-check.sh +bin/fm-test-run.sh tests/fm-claude-stop-autoarm.test.sh tests/fm-turnend-guard.test.sh tests/fm-supervision-instructions.test.sh | tail -8 +``` + +Observed output: + +```text +fm-lint.sh: ShellCheck 0.11.0 (pinned 0.11.0) +fm-doc-audience-check: ok surfaces=67 local_links=233 +FM_TEST_END 2026-08-14T02:34:08Z tests/fm-supervision-instructions.test.sh exit=0 duration_ms=711 gate_skip=false +FM_TEST_SUMMARY total=3 failed=0 skipped_gate=0 duration_ms=141882 +FM_TEST_SUMMARY_FAMILY family=pure-contract-unit count=1 duration_ms=711 failed=0 +FM_TEST_SUMMARY_FAMILY family=unclassified count=1 duration_ms=63319 failed=0 +FM_TEST_SUMMARY_FAMILY family=watcher-wake-lock count=1 duration_ms=76892 failed=0 +FM_TEST_SLOWEST rank=1 script=tests/fm-turnend-guard.test.sh duration_ms=76892 +FM_TEST_SLOWEST rank=2 script=tests/fm-claude-stop-autoarm.test.sh duration_ms=63319 +FM_TEST_SLOWEST rank=3 script=tests/fm-supervision-instructions.test.sh duration_ms=711 +``` + The Pi extension-model pull-guard correction (`bin/fm-guard.sh` no longer reports a false watcher-down on a Pi primary during the extension's own watcher hand-off) was verified on 2026-08-13 with the installed ShellCheck 0.11.0 and isolated behavior suites. The guard verdict itself reads only state files and process liveness, so the portable suites are the enforcing evidence; `bin/fm-harness.sh`'s Pi marker detection, which selects the model, is exercised in the same suite through `PI_CODING_AGENT`. @@ -413,11 +480,11 @@ fm-claude-stop-autoarm: ok ## Watcher continuity -The cross-harness evidence combines the 2026-07-17 live pass with Claude's replacement Stop-owned path revalidated on 2026-07-24, all against isolated project and home state. +The cross-harness evidence combines the 2026-07-17 live pass with Claude's replacement Stop-owned path revalidated on 2026-08-14, all against isolated project and home state. No credential material was copied into a fixture. ```text -Claude Code 2.1.219 +Claude Code 2.1.232 codex-cli 0.144.4 OpenCode 1.17.18 Pi 0.80.10 @@ -426,7 +493,7 @@ grok 0.2.103 (89c3d36fb6f1) [stable] | Harness | Exact opt-in command | Observed guarantee | | --- | --- | --- | -| Claude | `FM_CLAUDE_LIVE_E2E=1 tests/fm-claude-stop-autoarm-live-e2e.test.sh` | Session start reclaimed a stale owner before two Stop-owned cycles, and a competing live owner prevented arm, rewake, epoch write, or lock replacement. | +| Claude | `FM_CLAUDE_LIVE_E2E=1 tests/fm-claude-stop-autoarm-live-e2e.test.sh` | A read-only competing session defers without a guard loop, then the lock-owning session restores supervision and receives the actionable rewake. | | Codex | `FM_CODEX_LIVE_E2E=1 tests/fm-codex-continuity-live-e2e.test.sh` | The one-second foreground checkpoint returned without switching to the arm wrapper. | | OpenCode | `FM_OPENCODE_LIVE_E2E=1 tests/fm-opencode-primary-live-e2e.test.sh` | A verified successor existed before prompt handling, with no model re-arm or turn-end fallback. | | Pi | `FM_PI_LIVE_E2E=1 tests/fm-pi-primary-live-e2e.test.sh` | One initial tool call led to extension-owned successors and clean child retirement on exit. | diff --git a/docs/watcher-continuity.md b/docs/watcher-continuity.md index 1a94ec0ede..be7f12d687 100644 --- a/docs/watcher-continuity.md +++ b/docs/watcher-continuity.md @@ -13,7 +13,9 @@ Cursor's `.cursor/hooks.json` `stop` hook (`bin/fm-turnend-guard-cursor.sh`) own Claude's `.claude/settings.json` Stop `asyncRewake` hook (`bin/fm-claude-stop-autoarm.sh`) owns routine tokenless re-arm. The hook fires on every Stop, and an eligible primary with supervision need admits one home-scoped owner that foregrounds `bin/fm-watch-arm.sh` inside the hook-owned process tree. A numeric session-lock owner that fails the shared `fm_harness_pid_alive` predicate is reclaimed through `bin/fm-lock.sh` before auto-arm state changes, while a live owner, absent lock, or malformed lock keeps the competing hook inert. +The matching Claude turn-end guard applies the same proven foreign-live-owner boundary: a lock-refused read-only session may finish without consuming the lock owner's guard budget, while the lock-owning session remains the sole mutable supervision owner. The stale-owner claim occurs only after the existing AFK and supervision-need gates pass. +Every auto-arm invocation appends a best-effort entry and selected pre-claim gate to `state/.claude-autoarm-entry-trace`; the hook never waits for its trimming lock and behaves unchanged when diagnostic I/O fails, while successful trimming keeps the volatile trace at 256 lines. After each non-actionable arm close, the hook rechecks the identity-matched watcher lock and fresh beacon before retrying a bounded number of times. A cycle-end failure is benign when that live-watcher predicate is true, and the hook suppresses the arm output and continues silently. Only an exhausted failure with no verified watcher emits one last-resort notice for the continuous failure episode; later consecutive Stop cycles exit 2 to guarantee another Stop-owned retry without repeating the notice until the turn-end guard consumes the attended fail-open. @@ -30,7 +32,7 @@ After the configured retry bound is exhausted, it delivers the original wake wit This is deliberate Option B ordering: the fleet is protected before the model handles the wake whenever restoration succeeds, but the model is never left blind when it does not. Claude's Stop hook starts the successor arm at the next Stop after the handling turn, rather than before notification as Pi and OpenCode do. -The durable wake queue preserves actionable events during the residual active-turn window, and the bounded turn-end guard enforces recovery at Stop when no watcher or auto-arm claim is present. +The durable wake queue preserves actionable events during the residual active-turn window and remains a supervision need until post-handling acknowledgement consumes them, so source retirement cannot strand an undelivered result, and the bounded turn-end guard enforces recovery at Stop when no watcher or auto-arm claim is present. For every supported arm path, a successor that observes an accepted down stretch emits `check: rearm-resurface` through the ordinary durable handling path before settling into its live wait. That recovery presentation includes all unacknowledged queue rows, the cursor-folded OPEN DECISIONS set, and still-unread informational status lines, so a still-open decision or a buried `note:` answer reappears even when recovery has no queue row of its own. The model no longer re-arms after ordinary wakes. @@ -80,8 +82,8 @@ The same suite covers ordinary same-process session replacement for `/new`, `/re `tests/fm-watch-arm.test.sh` covers durable queue replay, real remote parent-replies ingestion into the authoritative status log, decision-only OPEN DECISIONS recovery, interrupted handling replay, generation-bound acknowledgement, a persistent live successor after recovery, a watcher close inside the handling window that must leave the printed acknowledgement valid, and the self-healing moved-generation acknowledgement that consumes its handled rows and names its remedy. `tests/fm-watcher-lock.test.sh` covers verified-successor attach, recovery publication before stale-lock removal, the typed self-eviction failure, bounded and successor-linked lifecycle rows, and a SIGSTOP counterfactual that distinguishes a live PID from a stale beacon before classifying termination. `tests/fm-subagent-pretool-check.test.sh` proves Claude retains only the non-status Bash seatbelts. -`tests/fm-claude-stop-autoarm.test.sh` covers the auto-arm's scope, stale and live session owners, unchanged AFK and need boundaries, single-flight, bounded failure retries, benign live-watcher cycle ends, one-notice failure episodes, and exit-2 translation. -`FM_CLAUDE_LIVE_E2E=1 tests/fm-claude-stop-autoarm-live-e2e.test.sh` starts with the reproduced stale-lock state, runs session start first, completes two tokenless cycles, and checks the competing-live-owner negative control. +`tests/fm-claude-stop-autoarm.test.sh` covers the auto-arm's scope, bounded gate trace, stale and live session owners, unchanged AFK and need boundaries, single-flight, bounded failure retries, benign live-watcher cycle ends, one-notice failure episodes, and exit-2 translation. +`FM_CLAUDE_LIVE_E2E=1 tests/fm-claude-stop-autoarm-live-e2e.test.sh` drives two real Claude sessions against one isolated home, proves the read-only session finishes after tracing the foreign-live-owner gate, and proves the lock-owning session restores supervision on its next Stop. `tests/fm-turnend-guard.test.sh` covers the cooperative `--claude` guard, including monotonic failed-epoch progression, the integrated bounded fail-open, post-alarm continuation suppression, and positive recovery reset. ## Active limits and verification @@ -91,4 +93,4 @@ No zero-latency guarantee is claimed because lock verification, watcher startup, OpenCode support targets persistent TUI sessions rather than headless `opencode run`. Claude depends on the Stop `asyncRewake` rewake, Cursor depends on its awaited stop-hook park, Grok retains native background-completion notifications, and Codex retains bounded foreground checkpoints. -[`verification/supervision.md`](verification/supervision.md#watcher-continuity) records the current five-harness live evidence, the 2026-07-24 Stop-owned Claude auto-arm results, and exact opt-in commands. +[`verification/supervision.md`](verification/supervision.md#watcher-continuity) records the current five-harness live evidence, the 2026-08-14 two-session Claude auto-arm result, and exact opt-in commands. diff --git a/tests/fm-claude-stop-autoarm-live-e2e.test.sh b/tests/fm-claude-stop-autoarm-live-e2e.test.sh index c7e2cab880..423c0c6892 100755 --- a/tests/fm-claude-stop-autoarm-live-e2e.test.sh +++ b/tests/fm-claude-stop-autoarm-live-e2e.test.sh @@ -1,15 +1,19 @@ #!/usr/bin/env bash # Opt-in credentialed Claude live regression for the Stop-owned auto-arm # (bin/fm-claude-stop-autoarm.sh + bin/fm-turnend-guard.sh --claude). -# Proves, against the real installed Claude Code and the real tracked hook -# registration: a fresh session with in-flight work, no watcher, and a stale -# session lock can run fm-session-start.sh first; session start reclaims the -# dead owner; at least two tokenless auto-arm and rewake cycles then complete -# with zero model-issued arm commands; and the cooperative guard consumes no -# forced continuation while the hook's launch is healthy. -# The project and FM_HOME are isolated; Claude keeps using its existing managed -# authentication. No live fleet home, worktree, or session is touched. -# shellcheck disable=SC2016 # the model, not this test shell, reads the prompt text +# +# Two real Claude sessions share one isolated Firstmate home. +# The lock-owning session stays active while a read-only competing session ends +# a turn with work in flight and no watcher. The competing auto-arm must trace +# its live-owner gate, and its matching guard must let that read-only session +# finish instead of trapping it in a continuation loop. When the owner ends its +# own turn, its Stop hook must claim the home and restore supervision without a +# model-issued arm command or human intervention. +# +# The project and FM_HOME are isolated under this disposable test directory. +# Claude uses its existing managed authentication; no live fleet home, worktree, +# or session is touched. +# shellcheck disable=SC2016 # the model, not this test shell, reads prompt literals set -u if [ "${FM_CLAUDE_LIVE_E2E:-0}" != 1 ]; then @@ -25,140 +29,149 @@ fail() { } command -v claude >/dev/null 2>&1 || fail "claude not found" +command -v jq >/dev/null 2>&1 || fail "jq not found" LAB="$ROOT/.claude-autoarm-live-e2e.$$" PROJECT="$LAB/project" HOME_DIR="$LAB/fmhome" -LIVE_OWNER_HOME="$LAB/live-owner-home" -TRANSCRIPT="$LAB/claude.jsonl" +OWNER_TRANSCRIPT="$LAB/owner.jsonl" +COMPETING_TRANSCRIPT="$LAB/competing.jsonl" CLAUDE_VERSION=$(claude --version) +OWNER_PID= +COMPETING_PID= cleanup() { + [ -z "$COMPETING_PID" ] || kill "$COMPETING_PID" 2>/dev/null || true + [ -z "$OWNER_PID" ] || kill "$OWNER_PID" 2>/dev/null || true rm -rf "$LAB" } trap cleanup EXIT +wait_for_path() { # + local path=$1 pid=$2 remaining=$3 + while [ ! -e "$path" ] && [ "$remaining" -gt 0 ]; do + kill -0 "$pid" 2>/dev/null || return 1 + sleep 0.1 + remaining=$((remaining - 1)) + done + [ -e "$path" ] +} + +wait_for_exit() { # + local pid=$1 remaining=$2 + while kill -0 "$pid" 2>/dev/null && [ "$remaining" -gt 0 ]; do + sleep 0.1 + remaining=$((remaining - 1)) + done + ! kill -0 "$pid" 2>/dev/null +} + mkdir -p "$LAB" -# git clone of this worktree carries only committed state, so copy the -# working-tree surfaces under test (same pattern as the continuity live E2E). git clone -q "$ROOT" "$PROJECT" +# A clone carries only committed state, so copy the working-tree surfaces under +# test, including the instrumentation and candidate fix being validated. cp -R "$ROOT/bin/." "$PROJECT/bin/" cp "$ROOT/.claude/settings.json" "$PROJECT/.claude/settings.json" -# The lab keeps the real tracked .claude/settings.json SessionStart nudge, -# Stop guard, and asyncRewake auto-arm registration. -# The only local hook records model-issued Bash calls without acquiring the -# session lock or otherwise changing lifecycle behavior. -cat > "$PROJECT/.claude/settings.local.json" <<'JSON' -{ - "hooks": { - "PreToolUse": [ - { - "matcher": "Bash", - "hooks": [ - { "type": "command", "command": "\"$CLAUDE_PROJECT_DIR\"/bin/tool-logger.sh" } - ] - } - ] - } -} -JSON - -cat > "$PROJECT/bin/tool-logger.sh" <<'SH' -#!/usr/bin/env bash -P=$(cat 2>/dev/null || true) -printf '%s\n' "$P" | jq -r '.tool_input.command // "unknown"' >> "$FM_HOME/state/tool-calls.log" 2>/dev/null -exit 0 -SH -chmod +x "$PROJECT/bin/tool-logger.sh" - mkdir -p "$HOME_DIR/state" "$HOME_DIR/config" "$HOME_DIR/data" printf 'project=fixture\nwindow=fixture\nbackend=tmux\n' > "$HOME_DIR/state/task.meta" -# A numeric pid above the supported OS pid range is a demonstrably dead prior -# harness owner under fm_harness_pid_alive, matching the reproduced incident. -printf '9999999\n' > "$HOME_DIR/state/.lock" -# Rapid-death arm fixture: started plus an immediate actionable reason, the -# exact spent-Stop edge shape. Runs 1-2 close actionable; run 3 closes clean so -# a misbehaving session can never loop forever. +cat > "$PROJECT/bin/owner-hold.sh" <<'SH' +#!/usr/bin/env bash +: > "$FM_HOME/state/owner-hold-started" +while [ ! -e "$FM_HOME/state/release-owner" ]; do + sleep 0.1 +done +SH cat > "$PROJECT/bin/fm-watch-arm.sh" <<'SH' #!/usr/bin/env bash -N=$(cat "$FM_HOME/state/arm-count" 2>/dev/null || echo 0); N=$((N+1)); echo "$N" > "$FM_HOME/state/arm-count" -echo "arm-run=$N pid=$$" >> "$FM_HOME/state/arm-ran" -if [ "$N" -ge 3 ]; then - rm -f "$FM_HOME/state/task.meta" - printf 'watcher: attached pid=%s (beacon 2s)\n' "$$" - exit 0 -fi +printf 'arm-run pid=%s\n' "$$" >> "$FM_HOME/state/arm-ran" printf 'watcher: started pid=%s (beacon fresh)\n' "$$" -printf 'stale: fixture-rapid-%s\n' "$N" -exit 0 +printf 'stale: live-owner-recovery\n' SH -# Drain fixture: session start invokes it once, then the model invokes it once -# per rewake. The third total drain ends the in-flight need after two complete -# Stop-owned cycles. -cat > "$PROJECT/bin/fm-wake-drain.sh" <<'SH' +cat > "$PROJECT/bin/finish-live.sh" <<'SH' #!/usr/bin/env bash -N=$(cat "$FM_HOME/state/drain-count" 2>/dev/null || echo 0); N=$((N+1)); echo "$N" > "$FM_HOME/state/drain-count" -echo "drain-run=$N" >> "$FM_HOME/state/drain-ran" -if [ "$N" -ge 3 ]; then - rm -f "$FM_HOME/state/task.meta" -fi -printf 'stale: fixture-rapid drained\n' +rm -f "$FM_HOME/state/task.meta" SH -chmod +x "$PROJECT/bin/fm-watch-arm.sh" "$PROJECT/bin/fm-wake-drain.sh" +chmod +x "$PROJECT/bin/owner-hold.sh" "$PROJECT/bin/fm-watch-arm.sh" "$PROJECT/bin/finish-live.sh" + +OWNER_PROMPT='Use Bash to run exactly `bin/owner-hold.sh` and wait for it. After it returns, reply exactly OWNER_RELEASED and end the turn. If Stop hook feedback then wakes you, use Bash to run exactly `bin/finish-live.sh`, reply exactly OWNER_RECOVERED, and end. Never run an arm command or any other tool.' +( + cd "$PROJECT" || exit 1 + exec env FM_HOME="$HOME_DIR" FM_GATE_REFUSE_BYPASS=1 CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false \ + claude -p "$OWNER_PROMPT" --dangerously-skip-permissions --effort low \ + --output-format stream-json --verbose --include-hook-events +) > "$OWNER_TRANSCRIPT" 2>&1 & +OWNER_PID=$! -PROMPT='Run exactly `bin/fm-session-start.sh` with Bash as your first tool call. After reading its complete digest, reply with exactly CYCLE0 and stop. Whenever a Stop hook feedback message wakes you, run exactly `bin/fm-wake-drain.sh` once with Bash, then reply with exactly ACK and stop. Never run bin/fm-watch-arm.sh or any other arm command, and never use any other tool.' +wait_for_path "$HOME_DIR/state/.lock" "$OWNER_PID" 600 \ + || fail "lock-owning Claude session did not acquire the isolated home: $(tail -20 "$OWNER_TRANSCRIPT")" +wait_for_path "$HOME_DIR/state/owner-hold-started" "$OWNER_PID" 600 \ + || fail "lock-owning Claude session did not enter the controlled active turn: $(tail -20 "$OWNER_TRANSCRIPT")" +LOCK_OWNER=$(cat "$HOME_DIR/state/.lock" 2>/dev/null || true) +kill -0 "$LOCK_OWNER" 2>/dev/null || fail "recorded session-lock owner is not alive" +COMPETING_PROMPT='Reply exactly COMPETING_READ_ONLY and end the turn without using tools.' ( cd "$PROJECT" || exit 1 - FM_HOME="$HOME_DIR" CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false \ - claude -p "$PROMPT" --dangerously-skip-permissions --effort low --output-format stream-json --verbose -) > "$TRANSCRIPT" 2>&1 || fail "Claude credentialed auto-arm session failed: $(tail -20 "$TRANSCRIPT")" - -ARM_RUNS=$(wc -l < "$HOME_DIR/state/arm-ran" 2>/dev/null | tr -d ' ') -[ "$ARM_RUNS" = 2 ] || fail "expected exactly 2 hook-owned arm cycles, got $ARM_RUNS: $(cat "$HOME_DIR/state/arm-ran" 2>/dev/null)" -DRAIN_RUNS=$(wc -l < "$HOME_DIR/state/drain-ran" 2>/dev/null | tr -d ' ') -[ "$DRAIN_RUNS" = 3 ] || fail "expected one session-start drain plus two model wake drains, got $DRAIN_RUNS drains" -REWAKES=$(grep -c 'Stop hook feedback' "$TRANSCRIPT" 2>/dev/null || true) -[ "$REWAKES" -ge 2 ] || fail "expected at least 2 exit-2 rewake deliveries, got $REWAKES" -grep -q 'stale: fixture-rapid-1' "$TRANSCRIPT" || fail "first rapid rewake reason missing from the transcript" -grep -q 'stale: fixture-rapid-2' "$TRANSCRIPT" || fail "second rapid rewake reason missing from the transcript" -[ "$(sed -n '1p' "$HOME_DIR/state/tool-calls.log" 2>/dev/null)" = 'bin/fm-session-start.sh' ] \ - || fail "fresh Claude session did not run session start first: $(cat "$HOME_DIR/state/tool-calls.log" 2>/dev/null)" -[ "$(cat "$HOME_DIR/state/.lock" 2>/dev/null)" != 9999999 ] \ - || fail "session start did not reclaim the stale dead-owner lock" -if [ -f "$HOME_DIR/state/tool-calls.log" ]; then - ! grep -q 'fm-watch-arm.sh' "$HOME_DIR/state/tool-calls.log" \ - || fail "model issued an arm command despite Stop-owned continuity: $(cat "$HOME_DIR/state/tool-calls.log")" - ! grep -q '&' "$HOME_DIR/state/tool-calls.log" \ - || fail "model used a shell ampersand: $(cat "$HOME_DIR/state/tool-calls.log")" + exec env FM_HOME="$HOME_DIR" FM_GATE_REFUSE_BYPASS=1 CLAUDE_CODE_ENABLE_PROMPT_SUGGESTION=false \ + claude -p "$COMPETING_PROMPT" --dangerously-skip-permissions --effort low \ + --output-format stream-json --verbose --include-hook-events +) > "$COMPETING_TRANSCRIPT" 2>&1 & +COMPETING_PID=$! + +remaining=600 +while kill -0 "$COMPETING_PID" 2>/dev/null \ + && [ ! -e "$HOME_DIR/state/.turnend-claude-blocks" ] \ + && [ "$remaining" -gt 0 ]; do + sleep 0.1 + remaining=$((remaining - 1)) +done +if [ -e "$HOME_DIR/state/.turnend-claude-blocks" ]; then + fail "read-only Claude session was trapped by the blind-turn guard: $(cat "$HOME_DIR/state/.turnend-claude-blocks")" fi -! grep -q 'TURN WOULD END BLIND' "$TRANSCRIPT" \ - || fail "cooperative guard consumed a forced continuation while the auto-arm launch was healthy" -[ "$(sed -n 's/^.*outcome=\([a-z][a-z]*\) .*$/\1/p' "$HOME_DIR/state/.claude-autoarm-epoch" 2>/dev/null)" = rewake ] \ - || fail "auto-arm epoch ledger must record the rewake outcome" -[ ! -e "$HOME_DIR/state/.claude-autoarm.lock" ] || fail "auto-arm owner lock was left behind" - -# Live-owner negative control: a separate supported-harness process owns a -# second isolated home while another Stop hook fires from the same primary -# project. The competing hook must not replace the session lock, arm, write an -# epoch, or rewake. -FAKE_CLAUDE="$LAB/claude" -ln -s /bin/bash "$FAKE_CLAUDE" -mkdir -p "$LIVE_OWNER_HOME/state" "$LIVE_OWNER_HOME/config" -printf 'project=fixture\n' > "$LIVE_OWNER_HOME/state/task.meta" -"$FAKE_CLAUDE" -c 'sleep 3; :' & -LIVE_OWNER_PID=$! -printf '%s\n' "$LIVE_OWNER_PID" > "$LIVE_OWNER_HOME/state/.lock" -LIVE_OWNER_RC=0 -printf '%s\n' '{"session_id":"live-owner-control"}' \ - | FM_HOME="$LIVE_OWNER_HOME" FM_ROOT_OVERRIDE="$PROJECT" "$FAKE_CLAUDE" -c '"$FM_ROOT_OVERRIDE/bin/fm-claude-stop-autoarm.sh"' \ - >"$LAB/live-owner.out" 2>"$LAB/live-owner.err" || LIVE_OWNER_RC=$? -[ "$LIVE_OWNER_RC" -eq 0 ] || fail "competing Stop hook returned $LIVE_OWNER_RC while another live session owned the home" -[ "$(cat "$LIVE_OWNER_HOME/state/.lock")" = "$LIVE_OWNER_PID" ] || fail "competing Stop hook replaced the live session owner" -[ ! -e "$LIVE_OWNER_HOME/state/arm-ran" ] || fail "competing Stop hook armed while another live session owned the home" -[ ! -e "$LIVE_OWNER_HOME/state/.claude-autoarm-epoch" ] || fail "competing Stop hook wrote an epoch while another live session owned the home" -[ ! -s "$LAB/live-owner.out" ] && [ ! -s "$LAB/live-owner.err" ] || fail "competing Stop hook produced a rewake while another live session owned the home" -wait "$LIVE_OWNER_PID" - -printf 'ok - Claude %s live E2E reclaimed a stale session lock through session start, completed two tokenless Stop-owned rewake cycles, and preserved the competing-live-owner boundary\n' "$CLAUDE_VERSION" +wait_for_exit "$COMPETING_PID" 300 \ + || fail "read-only Claude session did not finish after deferring supervision to the live lock owner" +wait "$COMPETING_PID" || fail "read-only Claude session exited unsuccessfully: $(tail -20 "$COMPETING_TRANSCRIPT")" +COMPETING_PID= +jq -e -s 'any(.[]; + .type == "assistant" + and any(.message.content[]?; .type == "text" and .text == "COMPETING_READ_ONLY") +)' "$COMPETING_TRANSCRIPT" >/dev/null \ + || fail "read-only Claude session did not produce its exact completion response" + +grep -q 'event=gate-live-session-owner' "$HOME_DIR/state/.claude-autoarm-entry-trace" \ + || fail "real competing Stop hook did not trace the live-session-owner gate" +[ "$(cat "$HOME_DIR/state/.lock")" = "$LOCK_OWNER" ] \ + || fail "read-only Stop hooks displaced the live session-lock owner" +[ ! -e "$HOME_DIR/state/arm-ran" ] \ + || fail "read-only Stop hook armed despite deferring recovery to the lock owner" + +: > "$HOME_DIR/state/release-owner" +wait_for_exit "$OWNER_PID" 900 \ + || fail "lock-owning Claude session did not finish its Stop-owned recovery" +wait "$OWNER_PID" || fail "lock-owning Claude recovery session failed: $(tail -20 "$OWNER_TRANSCRIPT")" +OWNER_PID= + +[ "$(wc -l < "$HOME_DIR/state/arm-ran" 2>/dev/null | tr -d ' ')" = 1 ] \ + || fail "expected exactly one owner-hook arm cycle: $(cat "$HOME_DIR/state/arm-ran" 2>/dev/null)" +grep -q 'event=claimed' "$HOME_DIR/state/.claude-autoarm-entry-trace" \ + || fail "lock-owning Stop hook never traced its auto-arm claim" +[ "$(sed -n 's/^.*outcome=\([a-z][a-z-]*\) .*$/\1/p' "$HOME_DIR/state/.claude-autoarm-epoch" 2>/dev/null)" = rewake ] \ + || fail "lock-owning Stop hook did not record outcome=rewake: $(cat "$HOME_DIR/state/.claude-autoarm-epoch" 2>/dev/null)" +jq -e -s 'any(.[]; + .type == "system" + and .subtype == "hook_response" + and .hook_event == "Stop" + and .exit_code == 2 + and ((.output // "") | contains("firstmate watcher wake")) +)' "$OWNER_TRANSCRIPT" >/dev/null \ + || fail "owner-hook actionable result was not delivered as a real exit-2 Stop response" +jq -e -s 'any(.[]; + .type == "assistant" + and any(.message.content[]?; .type == "text" and .text == "OWNER_RECOVERED") +)' "$OWNER_TRANSCRIPT" >/dev/null \ + || fail "real Stop feedback did not continue the owner session through recovery" +[ ! -e "$HOME_DIR/state/task.meta" ] \ + || fail "live fixture did not complete its in-flight supervision need" + +printf 'ok - Claude %s live E2E let the read-only competing session finish, then restored supervision from the lock-owning Stop hook without human intervention\n' "$CLAUDE_VERSION" diff --git a/tests/fm-claude-stop-autoarm.test.sh b/tests/fm-claude-stop-autoarm.test.sh index 7015fc4995..221514c31b 100755 --- a/tests/fm-claude-stop-autoarm.test.sh +++ b/tests/fm-claude-stop-autoarm.test.sh @@ -204,6 +204,40 @@ test_inert_without_session_lock() { pass "auto-arm: inert with no session lock" } +test_entry_trace_names_gate_and_stays_bounded() { + local dir unwritable out status lines i + dir=$(make_primary_dir "$TMP_ROOT/entry-trace") + : > "$dir/state/task.meta" + write_arm_fixture "$dir" actionable + + out=$(printf '%s\n' '{"session_id":"trace"}' \ + | FM_HOME="$dir" bash "$dir/bin/fm-claude-stop-autoarm.sh" 2>&1); status=$? + expect_code 0 "$status" "a missing session lock must keep the hook silent" + [ -z "$out" ] || fail "entry tracing changed hook output: $out" + assert_grep 'event=entry' "$dir/state/.claude-autoarm-entry-trace" "entry trace did not record hook entry" + assert_grep 'event=gate-lock-missing' "$dir/state/.claude-autoarm-entry-trace" "entry trace did not name the missing-lock gate" + assert_absent "$dir/state/.claude-autoarm-entry-trace.lock" "entry trace left its trimming lock behind" + + i=0 + while [ "$i" -lt 260 ]; do + printf 'at=0 pid=0 event=fixture-%s\n' "$i" >> "$dir/state/.claude-autoarm-entry-trace" + i=$((i + 1)) + done + printf '%s\n' '{"session_id":"trace"}' \ + | FM_HOME="$dir" bash "$dir/bin/fm-claude-stop-autoarm.sh" >/dev/null 2>&1 + lines=$(awk 'END { print NR }' "$dir/state/.claude-autoarm-entry-trace") + [ "$lines" -eq 256 ] || fail "entry trace must self-trim to 256 lines, got $lines" + + unwritable=$(make_primary_dir "$TMP_ROOT/entry-trace-unwritable") + : > "$unwritable/state/task.meta" + mkdir "$unwritable/state/.claude-autoarm-entry-trace" + out=$(printf '%s\n' '{"session_id":"trace"}' \ + | FM_HOME="$unwritable" bash "$unwritable/bin/fm-claude-stop-autoarm.sh" 2>&1); status=$? + expect_code 0 "$status" "an unavailable entry trace must not change the selected hook gate" + [ -z "$out" ] || fail "unavailable entry tracing changed hook output: $out" + pass "auto-arm: best-effort entry trace names the selected gate, self-trims, and cannot become a hook failure" +} + test_reclaims_stale_session_lock_before_arming() { local dir out status expected_owner actual_owner dir=$(make_primary_dir "$TMP_ROOT/stale-lock") @@ -328,6 +362,19 @@ test_inert_when_fleet_idle() { pass "auto-arm: inert with nothing in flight and no X-mode need" } +test_arms_for_queue_only_delivery_need() { + local dir out status + dir=$(make_primary_dir "$TMP_ROOT/queue-only") + FM_STATE_OVERRIDE="$dir/state" bash -c \ + '. "$1/bin/fm-wake-lib.sh"; fm_wake_append check pending-result "check: pending result"' _ "$dir" \ + || fail "could not seed the durable wake" + write_arm_fixture "$dir" actionable + out=$(run_autoarm "$dir" 2>/dev/null); status=$? + expect_code 2 "$status" "a queued wake must keep the auto-arm active after its source retires" + [ -e "$dir/state/arm-ran" ] || fail "hook took gate-no-supervision with a queued wake pending" + pass "auto-arm: queue-only delivery need arms the cycle" +} + # --- the armed cycle ---------------------------------------------------------- test_actionable_close_rewakes_with_reason() { @@ -579,12 +626,14 @@ test_fm_lock_status_still_works_with_shared_lib() { test_inert_in_child_worktree test_inert_without_session_lock +test_entry_trace_names_gate_and_stays_bounded test_reclaims_stale_session_lock_before_arming test_inert_when_lock_held_by_other_harness test_inert_when_afk test_stale_lock_recovery_preserves_afk_and_need_gates test_resolves_outermost_claude_pid_in_nested_bgspare_chain test_inert_when_fleet_idle +test_arms_for_queue_only_delivery_need test_actionable_close_rewakes_with_reason test_actionable_close_with_live_successor_rewakes_once test_failed_close_rewakes_with_failure_banner diff --git a/tests/fm-guard-stale-banner.test.sh b/tests/fm-guard-stale-banner.test.sh index 4171301f6c..36433065fa 100755 --- a/tests/fm-guard-stale-banner.test.sh +++ b/tests/fm-guard-stale-banner.test.sh @@ -263,6 +263,56 @@ test_queued_wake_warning_stays_independent() { pass "fm-guard stale banner: queued-wake warning remains independent" } +test_queue_reason_uses_shared_snapshot_during_concurrent_drain() { + local dir home fakebin ready release guard_pid i out status + dir=$(make_guard_case queue-snapshot) + home=$(case_home "$dir") + rm -f "$home/state/task.meta" + printf 'pending wake\n' > "$home/state/.wake-queue" + fakebin=$(fm_fakebin "$dir") + ready="$dir/verdict-ready" + release="$dir/verdict-release" + cat > "$fakebin/stat" <<'SH' +#!/usr/bin/env bash +: > "$FM_TEST_GUARD_SNAPSHOT_READY" +while [ ! -e "$FM_TEST_GUARD_SNAPSHOT_RELEASE" ]; do + sleep 0.01 +done +exit 1 +SH + chmod +x "$fakebin/stat" + + ( + PATH="$fakebin:$PATH" \ + FM_TEST_GUARD_SNAPSHOT_READY="$ready" \ + FM_TEST_GUARD_SNAPSHOT_RELEASE="$release" \ + run_guard_case "$dir" > "$dir/guard.out" 2>&1 + printf '%s\n' "$?" > "$dir/guard.status" + ) & + guard_pid=$! + i=0 + while [ ! -e "$ready" ] && [ "$i" -lt 200 ]; do + sleep 0.01 + i=$((i + 1)) + done + if [ ! -e "$ready" ]; then + : > "$release" + wait "$guard_pid" 2>/dev/null || true + fail "guard did not reach the post-status watcher verdict" + fi + : > "$home/state/.wake-queue" + : > "$release" + wait "$guard_pid" + status=$(cat "$dir/guard.status") + out=$(cat "$dir/guard.out") + expect_code 0 "$status" "guard must remain advisory during a concurrent queue drain" + assert_contains "$out" "Durable queued wake delivery pending" \ + "guard reason did not retain the shared pending-queue snapshot" + assert_not_contains "$out" "X-mode relay polling needs supervision" \ + "concurrent queue drain changed the shared snapshot into a false X-mode reason" + pass "fm-guard stale banner: queue reason uses one shared status snapshot" +} + test_read_only_before_writable_does_not_consume_full_banner() { local dir home marker lock out_ro out_rw dir=$(make_guard_case read-only-before-writable) @@ -703,6 +753,7 @@ test_healthy_recovery_rearms_next_stale_episode test_concurrent_same_episode_prints_one_full_banner test_home_isolation test_queued_wake_warning_stays_independent +test_queue_reason_uses_shared_snapshot_during_concurrent_drain test_read_only_before_writable_does_not_consume_full_banner test_read_only_during_episode_observes_without_mutating_marker test_healthy_read_only_does_not_clear_marker diff --git a/tests/fm-spawn-dispatch-profile.test.sh b/tests/fm-spawn-dispatch-profile.test.sh index d1f1effb41..e82378633a 100755 --- a/tests/fm-spawn-dispatch-profile.test.sh +++ b/tests/fm-spawn-dispatch-profile.test.sh @@ -13,23 +13,6 @@ set -u SPAWN="$ROOT/bin/fm-spawn.sh" TMP_ROOT=$(fm_test_tmproot fm-spawn-dispatch-profile) -make_spawn_pi_probe() { - local fakebin=$1 tool=$2 - cat > "$fakebin/$tool" <<'SH' -#!/usr/bin/env bash -set -u -if [ "${1:-}" = --help ]; then - if [ "${FM_FAKE_PI_VERSION:-0.84.0}" = 0.82.0 ]; then - printf '%s\n' 'Pi 0.82.0' 'Options: --help' - else - printf '%s\n' "Pi ${FM_FAKE_PI_VERSION:-0.84.0}" 'Options: --help --tui-mode ' - fi -fi -exit 0 -SH - chmod +x "$fakebin/$tool" -} - make_spawn_fakebin() { local dir=$1 fakebin fakebin=$(fm_fakebin "$dir") @@ -59,7 +42,7 @@ esac exit 0 SH chmod +x "$fakebin/tmux" - fm_fake_exit0 "$fakebin" treehouse + fm_fake_exit0 "$fakebin" treehouse pi pi-signed cat > "$fakebin/timeout" <<'SH' #!/usr/bin/env bash shift @@ -74,8 +57,6 @@ fi exit 0 SH chmod +x "$fakebin/timeout" "$fakebin/cursor-agent" - make_spawn_pi_probe "$fakebin" pi - make_spawn_pi_probe "$fakebin" pi-signed printf '%s\n' "$fakebin" } @@ -126,7 +107,7 @@ run_spawn() { FM_PROJECTS_OVERRIDE="$home/projects" FM_CONFIG_OVERRIDE="$home/config" \ FM_SPAWN_NO_GUARD=1 FM_FAKE_PANE_PATH="$wt" TMUX="fake,1,0" \ CLAUDE_CONFIG_DIR="${FM_TEST_CLAUDE_CONFIG_DIR:-}" \ - FM_FAKE_LAUNCH_LOG="$launchlog" FM_FAKE_PI_VERSION="${FM_TEST_PI_VERSION:-0.84.0}" \ + FM_FAKE_LAUNCH_LOG="$launchlog" \ FM_FAKE_CURSOR_MODELS="${FM_TEST_CURSOR_MODELS:-}" \ FM_FAKE_CURSOR_LIST_STATUS="${FM_TEST_CURSOR_LIST_STATUS:-0}" \ GROK_HOME="$home/grok-home" PATH="$fakebin:$PATH" \ @@ -623,8 +604,10 @@ test_pi_threads_model_and_max_effort() { expect_code 0 "$status" "pi spawn with max effort should succeed" assert_meta_profile "$HOME_DIR/state/$id.meta" pi openai-codex/gpt-5.6-sol max launch=$(cat "$LAUNCH_LOG") - assert_contains "$launch" "FM_PI_HARNESS=pi '$FAKEBIN_DIR/pi' --tui-mode regular --model 'openai-codex/gpt-5.6-sol' --thinking 'max' -e" \ - "pi launch did not force the regular TUI while threading the requested model and max thinking level" + assert_contains "$launch" "FM_PI_HARNESS=pi '$FAKEBIN_DIR/pi' --model 'openai-codex/gpt-5.6-sol' --thinking 'max' -e" \ + "pi launch did not thread the requested model and max thinking level" + assert_not_contains "$launch" "--tui-mode" \ + "pi launch passed the removed Pi 0.83 TUI mode option" assert_not_contains "$launch" "FM_FIRSTMATE_PI_LAUNCH_BRIEF=" \ "pi launch still exports the removed Calm input-reroute binding" assert_contains "$launch" "fm-operational-input.sh' encode launch-brief" \ @@ -645,8 +628,10 @@ test_pi_signed_threads_shared_pi_profile_and_preserves_identity() { assert_contains "$out" "spawned $id harness=pi-signed" "pi-signed spawn did not preserve its visible identity" assert_meta_profile "$HOME_DIR/state/$id.meta" pi-signed openai-codex/gpt-5.6-sol max launch=$(cat "$LAUNCH_LOG") - assert_contains "$launch" "FM_PI_HARNESS=pi-signed '$FAKEBIN_DIR/pi-signed' --tui-mode regular --model 'openai-codex/gpt-5.6-sol' --thinking 'max' -e" \ - "pi-signed launch did not force the regular TUI with Pi's model, thinking, and extension semantics" + assert_contains "$launch" "FM_PI_HARNESS=pi-signed '$FAKEBIN_DIR/pi-signed' --model 'openai-codex/gpt-5.6-sol' --thinking 'max' -e" \ + "pi-signed launch did not retain Pi's model, thinking, and extension semantics" + assert_not_contains "$launch" "--tui-mode" \ + "pi-signed launch passed the removed Pi 0.83 TUI mode option" assert_contains "$launch" "fm-operational-input.sh' encode launch-brief" \ "pi-signed launch lost the canonical typed launch-brief envelope" assert_present "$HOME_DIR/state/$id.pi-ext.ts" "pi-signed launch did not install Pi's turn-end extension" @@ -665,36 +650,6 @@ test_pi_signed_threads_shared_pi_profile_and_preserves_identity() { pass "pi-signed shares Pi launch semantics while preserving its configured and recorded identity" } -test_pi_tui_mode_probe_is_safe_for_old_and_new_pi() { - local harness version rec id out status launch - for harness in pi pi-signed; do - for version in 0.82.0 0.84.0; do - id="profile-${harness}-tui-${version//./}-z8d" - rec=$(make_spawn_case "profile-__MODELFLAG__-${harness}-tui-${version//./}" "$harness" "$id") - read_case_record "$rec" - - out=$(FM_TEST_PI_VERSION="$version" \ - run_ship_spawn "$HOME_DIR" "$WT_DIR" "$FAKEBIN_DIR" "$LAUNCH_LOG" \ - "$id" "$PROJ_DIR") - status=$? - expect_code 0 "$status" "$harness $version spawn should succeed" - launch=$(cat "$LAUNCH_LOG") - assert_contains "$launch" "'$FAKEBIN_DIR/$harness'" \ - "$harness $version launch must use the executable selected for probing" - assert_not_contains "$launch" "FM_PI_HARNESS=$harness $harness" \ - "$harness $version launch must not re-resolve a bare executable in the worker" - if [ "$version" = 0.82.0 ]; then - assert_not_contains "$launch" "--tui-mode" \ - "$harness $version launch must omit unsupported --tui-mode" - else - assert_contains "$launch" "'$FAKEBIN_DIR/$harness' --tui-mode regular" \ - "$harness $version launch must preserve the regular TUI" - fi - done - done - pass "Pi launch probing omits --tui-mode on older Pi and preserves it on supporting Pi" -} - test_pi_signed_missing_binary_refuses_before_endpoint_or_metadata() { local rec id out status id=profile-pi-signed-missing-z8c @@ -735,8 +690,10 @@ test_pi_signed_persistent_secondmate_uses_pi_extensions_and_identity() { "pi-signed secondmate spawn did not preserve its runtime identity" assert_meta_profile "$HOME_DIR/state/$id.meta" pi-signed default default launch=$(cat "$LAUNCH_LOG") - assert_contains "$launch" "FM_PI_HARNESS=pi-signed '$FAKEBIN_DIR/pi-signed' --tui-mode regular -e '$sm/.pi/extensions/fm-primary-turnend-guard.ts' -e '$sm/.pi/extensions/fm-primary-pi-watch.ts'" \ - "pi-signed secondmate did not force the regular TUI with Pi's primary extension launch shape" + assert_contains "$launch" "FM_PI_HARNESS=pi-signed '$FAKEBIN_DIR/pi-signed' -e '$sm/.pi/extensions/fm-primary-turnend-guard.ts' -e '$sm/.pi/extensions/fm-primary-pi-watch.ts'" \ + "pi-signed secondmate lost Pi's primary extension launch shape" + assert_not_contains "$launch" "--tui-mode" \ + "pi-signed secondmate passed the removed Pi 0.83 TUI mode option" pass "pi-signed is a distinct persistent secondmate runtime with shared Pi supervision semantics" } @@ -848,7 +805,6 @@ test_cursor_refuses_model_absent_from_live_catalog test_cursor_failed_catalog_probe_does_not_block_spawn test_opencode_threads_model_and_ignores_effort_axis test_pi_threads_model_and_max_effort -test_pi_tui_mode_probe_is_safe_for_old_and_new_pi test_pi_signed_threads_shared_pi_profile_and_preserves_identity test_pi_signed_missing_binary_refuses_before_endpoint_or_metadata test_pi_signed_persistent_secondmate_uses_pi_extensions_and_identity diff --git a/tests/fm-turnend-guard.test.sh b/tests/fm-turnend-guard.test.sh index ac02c7c37c..762c89e71b 100755 --- a/tests/fm-turnend-guard.test.sh +++ b/tests/fm-turnend-guard.test.sh @@ -67,14 +67,21 @@ test_predicate_healthy_fresh_beacon() { } test_predicate_queue_pending_flag() { - local state="$TMP_ROOT/pred-queue/state" + local state="$TMP_ROOT/pred-queue/state" first_queue second_queue mkdir -p "$state" fm_supervision_status "$state" 300 [ "$FM_SUP_QUEUE_PENDING" = false ] || fail "empty/absent wake queue must not read as pending" printf 'record\n' > "$state/.wake-queue" - fm_supervision_status "$state" 300 + fm_supervision_needed "$state" 300 || fail "a pending wake did not register as supervision need" [ "$FM_SUP_QUEUE_PENDING" = true ] || fail "a non-empty wake queue must read as pending" - pass "fm_supervision_status: FM_SUP_QUEUE_PENDING tracks state/.wake-queue" + [ "$FM_SUP_NEEDED" = true ] || fail "a pending wake must set FM_SUP_NEEDED" + first_queue=$FM_SUP_QUEUE_FINGERPRINT + printf 'different record\n' > "$state/.wake-queue" + fm_supervision_status "$state" 300 + second_queue=$FM_SUP_QUEUE_FINGERPRINT + [ "$first_queue" != "$second_queue" ] || fail "changed wake records left the queue fingerprint unchanged" + fm_supervision_unhealthy "$state" 300 || fail "a pending wake with no beacon must be unhealthy" + pass "fm_supervision_status: a pending wake needs supervision" } test_predicate_x_mode_needs_supervision() { @@ -98,6 +105,34 @@ test_predicate_source_needs_supervision() { pass "fm_supervision_unhealthy: source-only home needs supervision" } +test_predicate_identity_fingerprint_tracks_exact_owners() { + local state="$TMP_ROOT/pred-identities/state" task_a task_b source_a source_b + mkdir -p "$state/procevent" + : > "$state/task-a.meta" + : > "$state/procevent/source-a.source" + fm_supervision_status "$state" 300 + task_a=${FM_SUP_IDENTITY_FINGERPRINT:-} + [ -n "$task_a" ] || fail "shared supervision status did not publish an identity fingerprint" + + rm -f "$state/task-a.meta" + : > "$state/task-b.meta" + fm_supervision_status "$state" 300 + task_b=${FM_SUP_IDENTITY_FINGERPRINT:-} + [ "$task_a" != "$task_b" ] || fail "same-count task replacement left the supervision identity fingerprint unchanged" + + rm -f "$state/procevent/source-a.source" + : > "$state/procevent/source-b.source" + fm_supervision_status "$state" 300 + source_a=${FM_SUP_IDENTITY_FINGERPRINT:-} + [ "$task_b" != "$source_a" ] || fail "same-count process-source replacement left the supervision identity fingerprint unchanged" + + touch "$state/.last-watcher-beat" + fm_supervision_status "$state" 300 + source_b=${FM_SUP_IDENTITY_FINGERPRINT:-} + [ "$source_a" = "$source_b" ] || fail "volatile beacon age changed the supervision identity fingerprint" + pass "fm_supervision_status: identity fingerprint tracks exact tasks and process sources only" +} + # --- HOOK: bin/fm-turnend-guard.sh ------------------------------------------ # # Each scenario gets its own directory carrying a copy of the two guard scripts @@ -116,6 +151,7 @@ install_guard_scripts() { cp "$ROOT/bin/fm-supervision-lib.sh" "$dir/bin/fm-supervision-lib.sh" cp "$ROOT/bin/fm-wake-lib.sh" "$dir/bin/fm-wake-lib.sh" cp "$ROOT/bin/fm-hook-host-lib.sh" "$dir/bin/fm-hook-host-lib.sh" + cp "$ROOT/bin/fm-session-lock-lib.sh" "$dir/bin/fm-session-lock-lib.sh" mkdir -p "$dir/docs" cp -R "$ROOT/docs/supervision-protocols" "$dir/docs/supervision-protocols" chmod +x "$dir/bin/fm-turnend-guard.sh" "$dir/bin/fm-turnend-guard-grok.sh" "$dir/bin/fm-operational-input.sh" "$dir/bin/fm-supervision-instructions.sh" "$dir/bin/fm-harness.sh" @@ -250,6 +286,18 @@ test_hook_blocks_source_only_home() { pass "fm-turnend-guard: non-Claude path blocks a source-only home" } +test_hook_blocks_queue_only_home() { + local dir out status + dir=$(make_primary_dir "$TMP_ROOT/hook-queue-only") + FM_STATE_OVERRIDE="$dir/state" bash -c \ + '. "$1/bin/fm-wake-lib.sh"; fm_wake_append check pending-result "check: pending result"' _ "$dir" \ + || fail "could not seed the durable wake" + out=$(run_hook "$dir" false); status=$? + expect_code 2 "$status" "non-Claude hook must block when a queued wake has no watcher" + assert_contains "$out" "queued wake delivery pending" "block reason must identify the undelivered wake" + pass "fm-turnend-guard: non-Claude path blocks a queue-only home" +} + test_hook_blocks_when_dead_lock_has_fresh_beacon() { local dir dead out status dir=$(make_primary_dir "$TMP_ROOT/hook-dead-lock-fresh") @@ -1164,6 +1212,40 @@ test_hook_claude_mode_reblocks_stop_hook_active_when_unhealthy() { pass "fm-turnend-guard --claude: re-blocks a loop-guarded stop while unhealthy and unclaimed (incident regression)" } +test_hook_claude_mode_foreign_live_owner_does_not_starve_recovery() { + local dir claude owner auto_out auto_status guard_out guard_status owner_after + dir=$(make_primary_dir "$TMP_ROOT/hook-claude-foreign-owner") + : > "$dir/state/task1.meta" + install_integrated_autoarm "$dir" + write_integrated_failed_arm "$dir" + claude="$dir/claude" + ln -s /bin/bash "$claude" + + "$claude" -c 'sleep 60; :' & + owner=$! + printf '%s\n' "$owner" > "$dir/state/.lock" + # shellcheck disable=SC2016 # the fake harness expands FM_HOME in its child shell. + auto_out=$(printf '%s\n' '{"session_id":"foreign","stop_hook_active":false}' \ + | FM_HOME="$dir" "$claude" -c '"$FM_HOME/bin/fm-claude-stop-autoarm.sh"' 2>&1); auto_status=$? + # shellcheck disable=SC2016 # the fake harness expands FM_HOME in its child shell. + guard_out=$(printf '%s\n' '{"session_id":"foreign","stop_hook_active":false}' \ + | FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 FM_HOME="$dir" "$claude" -c \ + '"$FM_HOME/bin/fm-turnend-guard.sh" --claude' 2>&1); guard_status=$? + owner_after=$(cat "$dir/state/.lock") + kill "$owner" 2>/dev/null || true + wait "$owner" 2>/dev/null || true + + expect_code 0 "$auto_status" "a read-only session's auto-arm must defer to the live lock owner" + [ -z "$auto_out" ] || fail "foreign-owner auto-arm produced output: $auto_out" + expect_code 0 "$guard_status" "a read-only session must not be trapped by a guard whose matching auto-arm cannot own recovery" + [ -z "$guard_out" ] || fail "foreign-owner guard produced output: $guard_out" + assert_grep 'event=gate-live-session-owner' "$dir/state/.claude-autoarm-entry-trace" \ + "auto-arm entry trace did not identify the foreign live-owner gate" + [ "$owner_after" = "$owner" ] || fail "foreign-owner reproduction displaced the session lock owner" + assert_absent "$dir/state/.turnend-claude-blocks" "read-only guard consumed the lock owner's block budget" + pass "fm-turnend-guard --claude: a foreign live session owner cannot trap the read-only session in an unrecoverable Stop loop" +} + test_hook_claude_mode_reblocks_x_mode_without_tasks() { local dir out status dir=$(make_primary_dir "$TMP_ROOT/hook-claude-x-mode") @@ -1466,17 +1548,102 @@ test_hook_claude_mode_stale_rewake_epoch_blocks() { pass "fm-turnend-guard --claude: stale rewake epoch does not allow a blind stop" } -test_hook_claude_mode_budget_without_verified_failure_keeps_blocking() { - local dir out status i +test_hook_claude_mode_repeated_identical_block_escalates_once() { + local dir first second later status questions dir=$(make_primary_dir "$TMP_ROOT/hook-claude-budget") : > "$dir/state/task1.meta" - for i in 1 2 3 4; do - out=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? - expect_code 2 "$status" "--claude block $i must exit 2 within the budget" - done - assert_not_contains "$out" 'systemMessage' "budget exhaustion without verified auto-arm failure must not fail open" - assert_absent "$dir/state/.claude-autoarm-failure-alarmed" "unverified budget exhaustion recorded an attended alarm" - pass "fm-turnend-guard --claude: budget exhaustion alone cannot permit a blind stop" + printf 'epoch=3 owner_pid=999 outcome=rewake updated_at=1\n' > "$dir/state/.claude-autoarm-epoch" + touch -t 202001010000 "$dir/state/.claude-autoarm-epoch" + first=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? + expect_code 2 "$status" "the first no-claim observation must block" + assert_contains "$first" 'TURN WOULD END BLIND' "the first block lost the guard banner" + + second=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? + expect_code 0 "$status" "the second unchanged stop must terminate with the captain escalation" + assert_contains "$second" 'FIRSTMATE NEEDS YOUR DECISION' "the second unchanged stop did not escalate the supervision choice" + assert_contains "$second" 'after two identical blocked turn ends' "terminal escalation did not name the bounded trigger" + questions=$(printf '%s' "$first$second" | tr -cd '?' | wc -c | tr -d ' ') + [ "$questions" -eq 1 ] || fail "the two-block exchange must contain exactly one captain-facing question, got $questions: $first$second" + [ "$(sed -n '2s/^count=//p' "$dir/state/.turnend-claude-blocks")" = 1 ] \ + || fail "frozen epoch unexpectedly advanced the failure-epoch budget" + [ "$(sed -n '4s/^reblocks=//p' "$dir/state/.turnend-claude-blocks")" = 2 ] \ + || fail "frozen epoch did not advance the separate identical-block count" + assert_present "$dir/state/.turnend-claude-escalated" "terminal escalation did not record its one-shot marker" + assert_absent "$dir/state/.claude-autoarm-failure-alarmed" "unverified escalation consumed the verified-failure alarm" + + later=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? + expect_code 0 "$status" "an already escalated unchanged episode must stay terminal" + [ -z "$later" ] || fail "terminal captain escalation repeated in one unchanged episode: $later" + rm -f "$dir/state/task1.meta" + : > "$dir/state/task2.meta" + later=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? + expect_code 2 "$status" "changed evidence after escalation must start a fresh block sequence" + assert_absent "$dir/state/.turnend-claude-escalated" "changed evidence inherited the prior episode's escalation marker" + [ "$(sed -n '4s/^reblocks=//p' "$dir/state/.turnend-claude-blocks")" = 1 ] \ + || fail "changed evidence after escalation did not reset the identical-block count" + rm -f "$dir/state/task2.meta" + later=$(run_hook_claude "$dir" false); status=$? + expect_code 0 "$status" "an ended supervision need must stay silent" + assert_absent "$dir/state/.turnend-claude-escalated" "ended supervision need left the volatile escalation marker" + assert_absent "$dir/state/.turnend-claude-blocks" "ended supervision need left the volatile block budget" + pass "fm-turnend-guard --claude: two identical blocks terminate in one captain escalation instead of an unbounded loop" +} + +test_hook_claude_mode_changed_task_identity_resets_escalation_count() { + local dir out status + dir=$(make_primary_dir "$TMP_ROOT/hook-claude-budget-task-change") + : > "$dir/state/task1.meta" + out=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? + expect_code 2 "$status" "first no-claim observation must block" + rm -f "$dir/state/task1.meta" + : > "$dir/state/task2.meta" + out=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? + expect_code 2 "$status" "same-count task replacement must start a fresh block sequence" + [ "$(sed -n '4s/^reblocks=//p' "$dir/state/.turnend-claude-blocks")" = 1 ] \ + || fail "same-count task replacement did not reset the identical-block count" + pass "fm-turnend-guard --claude: changed task identity resets the identical-block escalation count" +} + +test_hook_claude_mode_changed_source_identity_resets_escalation_count() { + local dir out status + dir=$(make_primary_dir "$TMP_ROOT/hook-claude-budget-source-change") + mkdir -p "$dir/state/procevent" + : > "$dir/state/procevent/source1.source" + out=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? + expect_code 2 "$status" "first source-only no-claim observation must block" + rm -f "$dir/state/procevent/source1.source" + : > "$dir/state/procevent/source2.source" + out=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? + expect_code 2 "$status" "same-count process-source replacement must start a fresh block sequence" + [ "$(sed -n '4s/^reblocks=//p' "$dir/state/.turnend-claude-blocks")" = 1 ] \ + || fail "same-count process-source replacement did not reset the identical-block count" + pass "fm-turnend-guard --claude: changed process-source identity resets the identical-block escalation count" +} + +test_hook_claude_mode_source_retirement_during_wait_keeps_wake_supervised() { + local dir out status retire_pid + dir=$(make_primary_dir "$TMP_ROOT/hook-claude-budget-source-retires") + mkdir -p "$dir/state/procevent" + : > "$dir/state/procevent/source1.source" + out=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? + expect_code 2 "$status" "first source-only no-claim observation must block" + + ( + sleep 0.1 + FM_STATE_OVERRIDE="$dir/state" bash -c \ + '. "$1/bin/fm-wake-lib.sh"; fm_wake_append check procevent:source1:1 "check: procevent test source1 1"' _ "$dir" + rm -f "$dir/state/procevent/source1.source" + ) & + retire_pid=$! + out=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=400 run_hook_claude "$dir" false); status=$? + wait "$retire_pid" + assert_present "$dir/state/.wake-queue" "source retirement did not leave its durable wake" + expect_code 2 "$status" "a retired source with an undelivered wake must remain guarded" + assert_contains "$out" "queued wake delivery pending" "retired source block did not identify the undelivered wake" + [ "$(sed -n '4s/^reblocks=//p' "$dir/state/.turnend-claude-blocks")" = 1 ] \ + || fail "source retirement with a durable wake inherited the prior evidence count" + assert_absent "$dir/state/.turnend-claude-escalated" "changed source evidence emitted a stale captain escalation" + pass "fm-turnend-guard --claude: terminal source wake remains supervised after retirement" } test_hook_claude_mode_verified_failure_alarm_is_loud_and_once() { @@ -1538,6 +1705,7 @@ test_hook_claude_mode_allow_resets_budget() { out=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? expect_code 2 "$status" "first --claude block must exit 2" [ -f "$dir/state/.turnend-claude-blocks" ] || fail "--claude block must record the consecutive-block budget" + : > "$dir/state/.turnend-claude-escalated" : > "$dir/state/.claude-autoarm-failure-notified" : > "$dir/state/.claude-autoarm-failure-alarmed" sleep 60 & @@ -1555,6 +1723,7 @@ test_hook_claude_mode_allow_resets_budget() { rm -rf "$dir/state/.watch.lock" expect_code 0 "$status" "--claude must allow once the watcher is healthy again" [ ! -f "$dir/state/.turnend-claude-blocks" ] || fail "--claude allow must reset the consecutive-block budget" + [ ! -f "$dir/state/.turnend-claude-escalated" ] || fail "positive watcher recovery must reset the one-shot escalation" [ ! -f "$dir/state/.claude-autoarm-failure-notified" ] || fail "positive watcher recovery must reset the failure notice" [ ! -f "$dir/state/.claude-autoarm-failure-alarmed" ] || fail "positive watcher recovery must reset the attended alarm" out=$(FM_CLAUDE_AUTOARM_SYNC_WAIT_MS=100 run_hook_claude "$dir" false); status=$? @@ -1608,9 +1777,11 @@ test_predicate_healthy_fresh_beacon test_predicate_queue_pending_flag test_predicate_x_mode_needs_supervision test_predicate_source_needs_supervision +test_predicate_identity_fingerprint_tracks_exact_owners test_hook_silent_when_no_work_in_flight test_hook_blocks_when_fresh_beacon_has_no_live_lock test_hook_blocks_source_only_home +test_hook_blocks_queue_only_home test_hook_blocks_when_dead_lock_has_fresh_beacon test_hook_silent_with_live_lock_and_fresh_beacon test_hook_non_claude_health_ignores_claude_budget_contention @@ -1648,6 +1819,7 @@ test_opencode_plugin_anchors_guard_to_worktree test_pi_extension_injects_once_per_logical_agent_run test_pi_extension_retries_after_followup_delivery_failure test_hook_claude_mode_reblocks_stop_hook_active_when_unhealthy +test_hook_claude_mode_foreign_live_owner_does_not_starve_recovery test_hook_claude_mode_reblocks_x_mode_without_tasks test_hook_claude_mode_allows_when_autoarm_owner_alive test_hook_claude_mode_repeated_failed_to_arming_interleavings_reach_fail_open @@ -1658,7 +1830,10 @@ test_hook_claude_mode_integrated_monotonic_fail_open test_hook_claude_mode_recovery_contention_is_not_ordinary_allow test_hook_claude_mode_concurrent_recovery_resets_are_idempotent test_hook_claude_mode_stale_rewake_epoch_blocks -test_hook_claude_mode_budget_without_verified_failure_keeps_blocking +test_hook_claude_mode_repeated_identical_block_escalates_once +test_hook_claude_mode_changed_task_identity_resets_escalation_count +test_hook_claude_mode_changed_source_identity_resets_escalation_count +test_hook_claude_mode_source_retirement_during_wait_keeps_wake_supervised test_hook_claude_mode_verified_failure_alarm_is_loud_and_once test_hook_claude_mode_fail_open_requires_notice_and_failure_epoch test_hook_claude_mode_away_mode_never_uses_stop_autoarm_fail_open diff --git a/tests/lib.sh b/tests/lib.sh index 915741ba0d..0e7f99017c 100644 --- a/tests/lib.sh +++ b/tests/lib.sh @@ -189,23 +189,24 @@ SH # --- deterministic git identity and fixtures -------------------------------- -# fm_git_identity [name] [email]: export a fixed author/committer identity so -# fixture commits never depend on the host git config. +# fm_git_identity [name] [email]: export a fixed author/committer identity and +# disable inherited signing so fixture commits never depend on host Git config. fm_git_identity() { export GIT_AUTHOR_NAME=${1:-fmtest} GIT_AUTHOR_EMAIL=${2:-fmtest@example.invalid} export GIT_COMMITTER_NAME=$GIT_AUTHOR_NAME GIT_COMMITTER_EMAIL=$GIT_AUTHOR_EMAIL + export GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=commit.gpgsign GIT_CONFIG_VALUE_0=false } # fm_git_init_commit : create a git repo at with a README and one -# commit. Uses an inline identity so it works whether or not fm_git_identity was -# called. +# commit. Uses an inline identity and disables inherited signing so it works +# whether or not fm_git_identity was called and regardless of global Git config. fm_git_init_commit() { local dir=$1 mkdir -p "$dir" git -C "$dir" init -q printf '# %s\n' "$(basename "$dir")" > "$dir/README.md" git -C "$dir" add README.md - git -C "$dir" -c user.name='Firstmate Tests' -c user.email='tests@example.invalid' commit -qm initial + git -C "$dir" -c user.name='Firstmate Tests' -c user.email='tests@example.invalid' -c commit.gpgsign=false commit -qm initial } # fm_git_add_origin : clone bare into and register it