Pi-on-Herdr: fm-control relaunch permanently refuses after the Pi process dies, even with an fm-recovery idle event applied

[atibus]

Reproduced three times in a production home (2026-08-21 twice, 2026-08-23 once). When a Pi agent on the herdr backend exits without its extension closing the turn (fm-control-delivered exit, or an ENOSPC crash printing 'pi exiting due to uncaughtException'), the pane shows a bare shell and pgrep finds no pi process, yet fm-control <id> relaunch fails every attempt with: 'failed while stopping the old agent, which is still running; its original instructions were restored'.

Today's new datapoint: applying the documented recovery reset (fm-busy-event.sh apply <state> <id> idle --current-gen --source fm-recovery --event ...) succeeds and the busy record reads idle, but relaunch STILL refuses identically - so the stop-phase liveness check is not (only) reading the busy record. The same shell-in-pane situation relaunches fine for harness=claude, so the defect appears specific to the pi agent-state classification on herdr. The stop phase also types '/quit' into the bash shell ('bash: quit: command not found'), suggesting it believes a Pi composer is present.

Operator impact: the only recovery is captain-authorized manual WIP commit/push plus forced teardown and a fresh spawn - heavy for a routine crash. Happy to provide the exact state files from the incident on request.

[kunchenguid]

Speaking as Kun's firstmate: inspected on current main 7b88520c0554 (feat: add persistent Pi supervision branch (#2858)).

VISION.md (each rule):

• One captain, one interface: aligns. A dead Pi that cannot relaunch forces a heavy teardown; recovery is hidden behind a false "still running".
• Authority is explicit: aligns. Forced teardown is destructive; relaunch should work when the process is gone.
• Scripts own the mechanics: aligns. Stop/liveness is scripted (bin/fm-control.sh / herdr agent-state), not judgment.
• A restart is a non-event: aligns. This is the opposite: a crash becomes a captain-authorized reconstruct.
• Delegation with a spine: aligns.
• The fleet outlives any vendor: aligns (Pi-on-Herdr contract, not Pi pixels).
• Scope: aligns. Control/relaunch is firstmate's command layer.

Evidence on this SHA: do_relaunch still journals stopping then do_exit; rollback still prints exactly failed while stopping the old agent, which is still running; its original instructions were restored. Pi's exit command is still /quit (fm-control-lib.sh). Herdr fm_backend_herdr_agent_state maps a registered agent to alive even when the pane is a bare shell if agent get still reports a status — and do_exit will type /quit into that shell. Busy-record idle is not this path. Claude working in the same situation matches a Pi-specific classify miss. No covering open PR found.

Labeling ready-for-pr.

[kunchenguid]

Speaking as Kun's firstmate:

VISION.md read in full from current main 7b88520c055408a18f1476ecce08be60b2885fc9 (#2858). Per-rule, inspected bin/fm-control.sh stop/relaunch, bin/backends/herdr.sh fm_backend_herdr_pane_agent_state / fm_backend_herdr_agent_state, bin/backends/tmux.sh fm_backend_tmux_agent_state, bin/fm-backend.sh fm_backend_agent_state, tests/fm-control-herdr-smoke.test.sh, bin/fm-control-lib.sh fm_control_exit_command:

• One captain, one interface: aligns. A dead Pi that still reports "still running" hides a failure and forces a captain-heavy teardown instead of an honest already-stopped relaunch.
• Authority is explicit: aligns. Relaunch is an existing captain-invoked control verb; the fix is proving the agent actually stopped, not widening autonomy.
• Scripts own the mechanics: aligns. Agent liveness is an exact classifier. Herdr currently treats agent get registry success as recovery-grade alive; tmux corroborates with the foreground process group. The busy record is a different plane.
• A restart is a non-event: aligns. A Pi crash or /quit that leaves a bare shell must be recoverable from disk + live endpoint state without forced teardown.
• Delegation with a spine: aligns. This is the existing relaunch primitive, not a new task shape.
• The fleet outlives any vendor: aligns. Bind to process/endpoint semantics Herdr actually exposes (process_info, pane capture), not a stale agent get registration.
• Scope: aligns. Command-layer recovery. Validation stays with no-mistakes.

Contributor report (atibus): Pi-on-Herdr, reproduced 2026-08-21 twice and 2026-08-23 once. After Pi exits without its extension closing the turn (fm-control-delivered exit, or ENOSPC pi exiting due to uncaughtException), the pane is a bare shell and pgrep finds no pi, yet fm-control <id> relaunch fails every time with failed while stopping the old agent, which is still running; its original instructions were restored. Applying fm-busy-event.sh apply … idle --source fm-recovery succeeds and the busy record reads idle, but relaunch still refuses identically. Same shell-in-pane relaunches for harness=claude. Stop phase types /quit into bash (bash: quit: command not found). Only recovery today is captain-authorized WIP commit/push plus forced teardown.

Inspected current main 7b88520c055408a18f1476ecce08be60b2885fc9 (this is SHA evidence, not the report):

• bin/fm-control.sh:313-315,442-456,474-486,551-574: relaunch stop-phase liveness is fm_backend_agent_state, not the busy record. do_exit treats alive as "send the harness exit command and wait for dead". Rollback at phase stopping + alive prints the exact field error. An fm-recovery idle event cannot unblock this path.
• bin/fm-control-lib.sh:160-163: Pi's exit command is /quit. That is why a misclassified-alive pane gets /quit typed into bash.
• bin/backends/herdr.sh:1889-1910,1929-1937: Herdr recovery classifier is registry-only. agent get success with agent_status working|idle|done|blocked is live → alive, including done. agent_not_found is no-agent → dead. It never inspects whether a pi process is present. bin/backends/herdr.sh:1164-1217 already has process_info / bare-idle-shell proof, but fm_backend_herdr_agent_state does not use it.
• bin/backends/tmux.sh:259-333: tmux's recovery classifier does inspect the foreground process group; a pane that is only shells is dead. That is why the same shell-in-pane situation can relaunch for Claude-on-tmux, and why a Claude-on-Herdr death that actually unregisters (agent_not_found) already-stops.
• tests/fm-control-herdr-smoke.test.sh:12-14,98-147: pins today's contract against a real Herdr: classification comes from Herdr's agent registry, not process liveness. Registering a fake idle agent on a plain shell makes exit type the harness command into that shell and fail closed (did not stop). That is the reporter's hang, treated as correct fail-closed when the registry is stale.
• Busy vs liveness are separate: bin/backends/herdr.sh:2953-2966 maps done/idle to watcher idle, while pane-agent-state still maps those same statuses to live. The recovery idle event matching the report cannot change stop-phase classification.

Still present on this SHA. No open covering PR in the current open list (no relaunch/Pi-Herdr control PR). #2858 (persistent Pi supervision branch on this HEAD) does not change fm_backend_herdr_agent_state or the relaunch stop-phase; it is not this fix.

Constraints for a patch (queue, not a design): keep fail-closed when Herdr and process evidence still show a live harness; do not treat registry success alone as alive when the pane is a proven bare shell / no pi process; if /quit lands in a shell, that is death evidence, not liveness; do not require fm-recovery idle as the operator lever; do not make forced teardown the routine path. Cover Pi-on-Herdr: process gone, pane is shell, agent get still idle/done, relaunch proceeds as already-stopped then replacement launch. Claude-on-Herdr already-stopped (agent_not_found) must stay idempotent.

ready-for-pr is a queue label, not a merge vote.