diff --git a/AGENTS.md b/AGENTS.md index 9427247d769..cfd3369ea01 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -79,8 +79,9 @@ tools/ independently versioned provider-neutral components, commit config/crew-harness crewmate harness override; LOCAL, gitignored; absent or "default" = same as firstmate. Inherited as the literal file: a concrete primary adapter value also controls a secondmate home's own crewmates (section 4) config/crew-dispatch.json optional crewmate dispatch profiles; LOCAL, gitignored; firstmate-maintained but human-editable natural-language rules that choose a per-task harness/model/effort profile (section 4). Inherited by secondmate homes config/secondmate-harness harness the PRIMARY uses to launch SECONDMATE agents, optionally followed by a model and effort token on the same line (" [] []"; section 4); LOCAL, gitignored; absent or "default" harness falls back to config/crew-harness then firstmate's own. The primary's own setting; NOT inherited into secondmate homes (secondmates do not spawn secondmates) -config/account-routing-mode optional Agent Fleet routing policy (`off`, `observe`, or `enforce`); LOCAL, gitignored; default off; inherited by secondmate homes (docs/configuration.md "Agent Fleet account routing") +config/account-routing-mode optional account routing policy (`off`, `observe`, or `enforce`); new ship/scout observe/enforce launches use direct account directories, secondmate launches retain legacy Agent Fleet routing, and existing managed metadata retains legacy Agent Fleet recovery; LOCAL, gitignored; default off; inherited by secondmate homes (docs/configuration.md "Agent Fleet account routing") config/secondmate-account-pool optional Agent Fleet pool the PRIMARY uses for SECONDMATE launches when routing is enabled; LOCAL, gitignored; selection-only and NOT inherited +Direct account-directory launch currently covers ship/scout crewmates only; secondmate integration is deferred to a dedicated follow-up. config/backlog-backend backlog backend override; LOCAL, gitignored; absent or "tasks-axi" = default tasks-axi backend, "manual" = force routine backlog updates to hand-editing; inherited by secondmate homes (section 10) config/backend runtime session-provider backend override for new tasks; LOCAL, gitignored; absent = falls through to runtime auto-detection (the runtime firstmate itself is executing inside), then tmux; tmux is the verified reference backend, herdr/zellij/cmux are experimental new-task spawn backends, and Orca is legacy-recovery-only (docs/tmux-backend.md, docs/herdr-backend.md, docs/zellij-backend.md, docs/orca-backend.md, docs/cmux-backend.md) - herdr and cmux can also be selected by runtime auto-detection, zellij and Orca never are (always explicit), and codex-app is not accepted; see docs/codex-app-backend.md; not inherited into secondmate homes config/cmux-socket-password optional cmux control-socket password; LOCAL, gitignored; read fresh on every cmux CLI call and passed through without ever overriding an operator's own ambient CMUX_SOCKET_PASSWORD when absent (docs/cmux-backend.md "Setup") @@ -102,7 +103,12 @@ state/ volatile runtime signals; gitignored .status appended by crewmates: ": " wake-event lines, not current-state truth .turn-ended touched by turn-end hooks .grok-turnend-token firstmate-owned grok hook registry token for the task; removed by teardown - .meta written by fm-spawn: window=, worktree=, project=, harness=, model=, effort=, kind=, mode=, yolo=, tasktmp=, generation_id=, report_required=; managed tasks add account_pool=, account_profile=, account_task=, account_attempt=, and provider_session_id= (docs/configuration.md "Agent Fleet account routing"); kind=secondmate also records home= and projects=; a non-default runtime backend records further backend-specific fields (docs/configuration.md "Runtime backend"; bin/fm-backend.sh, section 8); fm-pr-check, including through fm-pr-merge, appends pr= and GitHub's pr_head= when available; fm-x-link appends x_request=, x_request_ts=, x_followups=, and optional x_platform=/x_reply_max_chars= for an X-mode-originated task (section 14) + .meta written by fm-spawn: window=, worktree=, project=, harness=, model=, effort=, kind=, mode=, yolo=, tasktmp=, generation_id=, report_required= + Direct ship and scout launches also own account_home=, worktree_git_dir=, worktree_git_dir_identity=, and exactly one authoritative final-state field: worktree_git_ref= for an attached branch or worktree_git_head= for an intentional detached HEAD. Their metadata may temporarily carry worktree_git_setup_ref= and worktree_git_setup_head= while the brief's required `fm/` branch transition is pending; recovery accepts only that exact setup state or the authoritative `fm/` ref and removes the setup fields after adoption. + Direct recovery validates the canonical worktree path, exact physical Git-dir identity, and authoritative final state before account preparation and again immediately before endpoint creation; any drift fails closed without launching. + If endpoint removal after a failed new direct spawn cannot be confirmed, direct_spawn_cleanup=pending and rollback_pending=1 retain the endpoint and worktree identity for explicit teardown. + Secondmate Agent Fleet routing and legacy managed recovery own account_pool=, account_profile=, account_task=, account_attempt=, and provider_session_id= (docs/configuration.md "Agent Fleet account routing"). + kind=secondmate also records home= and projects=; a non-default runtime backend records further backend-specific fields (docs/configuration.md "Runtime backend"; bin/fm-backend.sh, section 8); fm-pr-check, including through fm-pr-merge, appends pr= and GitHub's pr_head= when available; fm-x-link appends x_request=, x_request_ts=, x_followups=, and optional x_platform=/x_reply_max_chars= for an X-mode-originated task (section 14) .check.sh optional slow poll you write per task (e.g. merged-PR check) x-watch.check.sh generated X-mode relay poll shim; present only when opted in (section 14) x-inbox/ generated X-mode pending mention payloads; fmx-respond drains it (section 14) @@ -207,8 +213,9 @@ That refusal is the consultation backstop, so the rules are never silently skipp The requirement is gated only on the file's presence; when the file is absent, `fm-spawn.sh` keeps resolving the crewmate harness from `config/crew-harness` as before. Secondmate launches are exempt because they resolve through `fm-harness.sh secondmate`, not the crewmate dispatch-profile rules. -`quota-balanced` selection is deterministic and owned by `bin/fm-dispatch-select.sh`; its header documents the general-window rules, Agent Fleet pool-summary path, freshness margin, and every fallback, and it degrades to the first array element whenever quota data is unusable. -When candidates carry account pools, the selector uses only Agent Fleet summaries and never falls through to quota-axi's default-account data; pass its selected pool to spawn for the atomic concrete-profile lease. +`quota-balanced` selection is deterministic and owned by `bin/fm-dispatch-select.sh`; every real new ship/scout launch uses direct account-directory selection. +When candidates carry account pools, the selector chooses the ordered first profile and passes its pool to spawn only as a compatibility activation input, never as an account choice or lease request. +The now-unreachable Agent Fleet pool-summary implementation and inactive new-lease fixtures are legacy code deferred to follow-up task `remove-fleet-routing-deadcode`. Quota trouble must never block dispatch. Precedence, highest first: @@ -255,6 +262,8 @@ Reconcile reality with your records before doing anything else, working from the Do not sweep every `fm-*` tmux window, herdr tab, zellij tab, Orca terminal, or cmux workspace across all sessions during recovery; another firstmate home's child endpoints may share that namespace and are not this home's orphans. 5. If the digest reports a recorded direct-report's endpoint as `dead` (or a meta has no `window=`), reconcile it through its meta as described below. 6. For meta with no window, or an endpoint the digest reported dead, reconcile by kind. + If ship/scout meta records `account_home=`, relaunch with `bin/fm-spawn.sh --recover-direct-account`. + That mode reloads the recorded task context and performs fresh direct account-directory selection. If meta records `account_profile=`, first try the exact sticky session with `bin/fm-spawn.sh --resume-account`; when that session/profile is unavailable, use `--continue-account` for the task-owned provider-neutral handoff after re-verifying live and repository state, with `bin/fm-account-continuation.sh` owning the fail-closed packet contract. For ordinary crewmates, check the recorded backend metadata first; use `treehouse status` for treehouse-backed tasks, and the recorded `orca_worktree_id=`/`terminal=` for Orca tasks. For an unmanaged `kind=secondmate`, load `secondmate-provisioning`; its "Recovery" section owns the explicit routing decision and respawn procedure. @@ -429,13 +438,14 @@ Load `harness-adapters` before spawning or recovering any direct report so trust ```sh bin/fm-spawn.sh projects/ # uses the active crewmate harness only when no crew-dispatch.json is active bin/fm-spawn.sh projects/ --harness codex --model gpt-5.5 --effort high # explicit profile axes -bin/fm-spawn.sh projects/ --harness codex --account-pool codex-crew # dynamic Agent Fleet account pool -bin/fm-spawn.sh projects/ --harness claude --account-profile claude-2 # explicit Agent Fleet profile -bin/fm-spawn.sh --resume-account # sticky managed recovery; never a fresh prompt -bin/fm-spawn.sh --continue-account # fresh managed session from verified task-owned continuation state +bin/fm-spawn.sh projects/ --harness codex --account-pool codex-crew # compatibility flag activating direct account-directory selection +bin/fm-spawn.sh projects/ --harness claude --account-profile claude-2 # compatibility flag activating direct account-directory selection +bin/fm-spawn.sh --recover-direct-account # metadata-preserving ship/scout direct account recovery with fresh selection +bin/fm-spawn.sh --resume-account # sticky legacy managed recovery; never a fresh prompt +bin/fm-spawn.sh --continue-account # fresh legacy managed session from verified task-owned continuation state bin/fm-spawn.sh projects/ --backend # explicit new-task runtime backend (docs/configuration.md "Runtime backend") bin/fm-spawn.sh projects/ --scout # scout task; records kind=scout in meta -bin/fm-spawn.sh [] --secondmate # launch or recover a persistent secondmate in its home +bin/fm-spawn.sh [] --secondmate # launch a persistent secondmate in its home bin/fm-spawn.sh =projects/ =projects/ [--scout] # batch: one call, several tasks ``` diff --git a/bin/fm-account-directory.sh b/bin/fm-account-directory.sh new file mode 100755 index 00000000000..3ba66b5462c --- /dev/null +++ b/bin/fm-account-directory.sh @@ -0,0 +1,398 @@ +#!/usr/bin/env bash +# Select and prepare direct Claude or Codex account-directory launches. +# Usage: +# fm-account-directory.sh select +# fm-account-directory.sh install-herdr-hook +# fm-account-directory.sh prepare +# +# This header is the single owner of the direct account-directory contract. +# FM_ACCOUNT_DIRECTORY_CUTOVER: direct-observe-passwd-home-v2 +# Account homes are discovered under the current passwd user's +# .local/share/agent-fleet/accounts// tree without fixed counts. +# Codex selection removes that account's quota-axi window cache immediately +# before every read, sets CODEX_HOME plus the account-isolated XDG_CACHE_HOME, +# accepts only a fresh result with at least one numeric five_hour or weekly +# window, and picks the account with the highest minimum remaining percentage. +# A Codex account with no such freshly readable window is skipped as unhealthy. +# Claude quota is not currently distinguishable per config directory because +# quota-axi cannot non-interactively resolve Claude's config-dir-specific macOS +# Keychain credential. +# Claude therefore never treats a missing usage window as account failure and +# selects the first real account directory in stable bytewise sort order. +# Selection prints only the chosen absolute account home on stdout and logs +# health, fallback, and choice diagnostics on stderr. +# prepare selects the account and idempotently runs Herdr's own integration +# installer with CODEX_HOME or CLAUDE_CONFIG_DIR set to the chosen home. +# It verifies the installed per-profile hook before printing the chosen home. +# +# Credential state is read-only. +# This script never logs in, imports credentials, or invokes a provider model. +# Test-only command, root, passwd-home, Perl, and timeout overrides require +# FM_ACCOUNT_DIRECTORY_TEST_LAB=firstmate-account-directory-test-lab-v1. +set -u + +TEST_LAB_TOKEN=firstmate-account-directory-test-lab-v1 + +usage() { + sed -n '2,32p' "$0" | sed 's/^# \{0,1\}//' >&2 +} + +log() { + printf 'fm-account-directory: %s\n' "$*" >&2 +} + +test_lab_enabled() { + [ "${FM_ACCOUNT_DIRECTORY_TEST_LAB:-}" = "$TEST_LAB_TOKEN" ] +} + +system_perl() { + if test_lab_enabled && [ -n "${FM_ACCOUNT_DIRECTORY_PERL_BIN:-}" ]; then + printf '%s\n' "$FM_ACCOUNT_DIRECTORY_PERL_BIN" + else + printf '%s\n' /usr/bin/perl + fi +} + +passwd_home() { + local home perl_bin + if test_lab_enabled && [ -n "${FM_ACCOUNT_DIRECTORY_PASSWD_HOME:-}" ]; then + home=$FM_ACCOUNT_DIRECTORY_PASSWD_HOME + else + perl_bin=$(system_perl) || return 1 + [ -x /usr/bin/env ] && [ -x "$perl_bin" ] || { + echo "error: /usr/bin/env and /usr/bin/perl are required to resolve the current passwd home" >&2 + return 1 + } + # shellcheck disable=SC2016 # Perl source is intentionally single-quoted. + home=$(/usr/bin/env -i PATH=/usr/bin:/bin:/usr/sbin:/sbin "$perl_bin" -e ' + my @p = getpwuid($<); + exit 1 unless @p && defined $p[7] && $p[7] =~ m{^/}; + exit 1 if $p[7] =~ /[\x00-\x1f\x7f]/; + print $p[7]; + ' 2>/dev/null) || { + echo "error: cannot resolve the current passwd home" >&2 + return 1 + } + fi + case "$home" in + *$'\n'*|*$'\r'*) + echo "error: passwd home contains a line break" >&2 + return 1 + ;; + /*) ;; + *) + echo "error: passwd home must be absolute: $home" >&2 + return 1 + ;; + esac + [ -d "$home" ] && [ ! -L "$home" ] || { + echo "error: passwd home is not a real directory: $home" >&2 + return 1 + } + printf '%s\n' "$home" +} + +account_root() { + local root home + if test_lab_enabled && [ -n "${FM_ACCOUNT_DIRECTORY_ROOT:-}" ]; then + root=$FM_ACCOUNT_DIRECTORY_ROOT + else + home=$(passwd_home) || return 1 + root=$home/.local/share/agent-fleet/accounts + fi + case "$root" in + *$'\n'*|*$'\r'*) + echo "error: account-directory root contains a line break" >&2 + return 1 + ;; + /*) ;; + *) + echo "error: account-directory root must be absolute: $root" >&2 + return 1 + ;; + esac + printf '%s\n' "$root" +} + +quota_command() { + if test_lab_enabled && [ -n "${FM_ACCOUNT_DIRECTORY_QUOTA_AXI:-}" ]; then + printf '%s\n' "$FM_ACCOUNT_DIRECTORY_QUOTA_AXI" + return 0 + fi + command -v quota-axi 2>/dev/null || { + echo "error: quota-axi is required for fresh Codex account selection" >&2 + return 1 + } +} + +herdr_command() { + if test_lab_enabled && [ -n "${FM_ACCOUNT_DIRECTORY_HERDR:-}" ]; then + printf '%s\n' "$FM_ACCOUNT_DIRECTORY_HERDR" + return 0 + fi + command -v herdr 2>/dev/null || { + echo "error: herdr is required to install the selected account's integration hook" >&2 + return 1 + } +} + +quota_timeout_seconds() { + local timeout=15 + if test_lab_enabled && [ -n "${FM_ACCOUNT_DIRECTORY_QUOTA_TIMEOUT_SECONDS:-}" ]; then + timeout=$FM_ACCOUNT_DIRECTORY_QUOTA_TIMEOUT_SECONDS + fi + case "$timeout" in + ''|*[!0-9]*|0) + echo "error: Codex quota timeout must be a positive integer" >&2 + return 1 + ;; + esac + printf '%s\n' "$timeout" +} + +run_bounded() { + local timeout=$1 perl_bin + shift + perl_bin=$(system_perl) || return 1 + [ -x "$perl_bin" ] || return 127 + # shellcheck disable=SC2016 # Perl source is intentionally single-quoted. + PERL5LIB='' PERL5OPT='' "$perl_bin" -e ' + use POSIX qw(setpgid WNOHANG); + my ($timeout, @command) = @ARGV; + my $pid = fork(); + exit 125 unless defined $pid; + if ($pid == 0) { + setpgid(0, 0); + exec {$command[0]} @command; + exit 127; + } + setpgid($pid, $pid); + my $deadline = time() + $timeout; + while (1) { + my $waited = waitpid($pid, WNOHANG); + if ($waited == $pid) { + my $status = $?; + exit(($status & 127) ? 128 + ($status & 127) : ($status >> 8)); + } + exit 125 if $waited == -1; + if (time() >= $deadline) { + kill "TERM", -$pid; + for (1 .. 4) { + select undef, undef, undef, 0.05; + exit 124 if waitpid($pid, WNOHANG) == $pid; + } + kill "KILL", -$pid; + waitpid($pid, 0); + exit 124; + } + select undef, undef, undef, 0.05; + } + ' "$timeout" "$@" +} + +valid_account_home() { # + local vendor_dir=$1 candidate=$2 name + [ -d "$candidate" ] && [ ! -L "$candidate" ] || return 1 + case "$candidate" in + "$vendor_dir"/*) ;; + *) return 1 ;; + esac + name=${candidate##*/} + case "$name" in + ''|.*|*[!A-Za-z0-9._-]*) return 1 ;; + esac +} + +first_account_home() { # + local vendor=$1 root vendor_dir candidate + root=$(account_root) || return 1 + vendor_dir=$root/$vendor + [ -d "$vendor_dir" ] && [ ! -L "$vendor_dir" ] || { + echo "error: no account-directory root for $vendor at $vendor_dir" >&2 + return 1 + } + LC_ALL=C + export LC_ALL + for candidate in "$vendor_dir"/*; do + valid_account_home "$vendor_dir" "$candidate" || continue + printf '%s\n' "$candidate" + return 0 + done + echo "error: no account directories found for $vendor under $vendor_dir" >&2 + return 1 +} + +fresh_codex_usage_json() { # + local account_home=$1 quota_bin=$2 cache_home cache_file environment_name timeout status + timeout=$(quota_timeout_seconds) || return 1 + cache_home=$account_home/.agent-fleet-quota-cache + cache_file=$cache_home/quota-axi/quotas.json + if { [ -e "$cache_home" ] || [ -L "$cache_home" ]; } \ + && { [ ! -d "$cache_home" ] || [ -L "$cache_home" ]; }; then + log "codex account $account_home skipped: its quota cache root is not a real directory" + return 1 + fi + if { [ -e "$cache_home/quota-axi" ] || [ -L "$cache_home/quota-axi" ]; } \ + && { [ ! -d "$cache_home/quota-axi" ] || [ -L "$cache_home/quota-axi" ]; }; then + log "codex account $account_home skipped: its quota-axi cache directory is not a real directory" + return 1 + fi + if [ -e "$cache_file" ] || [ -L "$cache_file" ]; then + rm -f "$cache_file" || { + log "codex account $account_home skipped: could not clear its quota cache for a fresh health read" + return 1 + } + fi + ( + while IFS='=' read -r environment_name _; do + case "$environment_name" in + XDG_*|QUOTA_AXI_*|AGENT_FLEET_*) unset "$environment_name" ;; + esac + done < <(/usr/bin/env) + CODEX_HOME=$account_home + XDG_CACHE_HOME=$cache_home + export CODEX_HOME XDG_CACHE_HOME + if run_bounded "$timeout" "$quota_bin" --provider codex --json 2>/dev/null; then + return 0 + else + status=$? + fi + if [ "$status" -eq 124 ]; then + log "codex account $account_home skipped: quota read timed out after ${timeout}s" + fi + return "$status" + ) +} + +codex_score() { # + jq -er ' + [.providers[]? + | select(.provider == "codex" and .state.status == "fresh") + | (.windows // [])[]? + | select((.id == "five_hour" or .id == "weekly") + and (.kind // "") != "model" + and (.percentRemaining | type) == "number") + | .percentRemaining] + | if length == 0 then empty else min end + ' 2>/dev/null <&2 + return 1 + } + command -v jq >/dev/null 2>&1 || { + echo "error: jq is required for Codex account usage selection" >&2 + return 1 + } + quota_bin=$(quota_command) || return 1 + LC_ALL=C + export LC_ALL + for candidate in "$vendor_dir"/*; do + valid_account_home "$vendor_dir" "$candidate" || continue + usage=$(fresh_codex_usage_json "$candidate" "$quota_bin") || usage= + score=$(codex_score "$usage") || score= + if [ -z "$score" ]; then + log "codex account $candidate skipped: no freshly readable usage window" + continue + fi + log "codex account $candidate fresh remaining score=$score" + if [ -z "$best_home" ] || awk -v candidate_score="$score" -v current_score="$best_score" \ + 'BEGIN { exit !(candidate_score > current_score) }'; then + best_home=$candidate + best_score=$score + fi + done + [ -n "$best_home" ] || { + echo "error: no healthy Codex account has a freshly readable usage window" >&2 + return 1 + } + log "selected codex account $best_home with fresh remaining score=$best_score" + printf '%s\n' "$best_home" +} + +select_claude() { + local selected + selected=$(first_account_home claude) || return 1 + log "CLAUDE USAGE UNREADABLE: quota-axi cannot non-interactively resolve Claude's config-dir-specific macOS Keychain credential today; selecting the first account directory by stable sort: $selected" + printf '%s\n' "$selected" +} + +select_account() { # + case "$1" in + codex) select_codex ;; + claude) select_claude ;; + *) + echo "error: direct account-directory selection supports only claude or codex, not '$1'" >&2 + return 1 + ;; + esac +} + +install_herdr_hook() { # + local vendor=$1 account_home=$2 root vendor_dir herdr_bin expected_hook + root=$(account_root) || return 1 + vendor_dir=$root/$vendor + valid_account_home "$vendor_dir" "$account_home" || { + echo "error: unsafe $vendor account home for Herdr hook installation: $account_home" >&2 + return 1 + } + herdr_bin=$(herdr_command) || return 1 + case "$vendor" in + codex) + CODEX_HOME=$account_home "$herdr_bin" integration install codex >/dev/null || { + echo "error: Herdr Codex integration install failed for $account_home" >&2 + return 1 + } + expected_hook=$account_home/herdr-agent-state.sh + ;; + claude) + CLAUDE_CONFIG_DIR=$account_home "$herdr_bin" integration install claude >/dev/null || { + echo "error: Herdr Claude integration install failed for $account_home" >&2 + return 1 + } + expected_hook=$account_home/hooks/herdr-agent-state.sh + ;; + *) + echo "error: Herdr account hook installation supports only claude or codex, not '$vendor'" >&2 + return 1 + ;; + esac + [ -f "$expected_hook" ] && [ ! -L "$expected_hook" ] || { + echo "error: Herdr installer did not create the expected $vendor hook at $expected_hook" >&2 + return 1 + } + log "Herdr $vendor hook ready at $expected_hook" +} + +case "${1:-}" in + -h|--help) + usage + exit 0 + ;; + select) + [ "$#" -eq 2 ] || { usage; exit 2; } + select_account "$2" + ;; + install-herdr-hook) + [ "$#" -eq 3 ] || { usage; exit 2; } + install_herdr_hook "$2" "$3" + ;; + prepare) + [ "$#" -eq 2 ] || { usage; exit 2; } + selected_home=$(select_account "$2") || exit 1 + install_herdr_hook "$2" "$selected_home" || exit 1 + printf '%s\n' "$selected_home" + ;; + *) + usage + exit 2 + ;; +esac diff --git a/bin/fm-account-routing-lib.sh b/bin/fm-account-routing-lib.sh index b046a3c15cb..3fb31539b61 100644 --- a/bin/fm-account-routing-lib.sh +++ b/bin/fm-account-routing-lib.sh @@ -15,12 +15,10 @@ # FM_ACCOUNT_ROUTING and executable overrides are accepted only when the # unmistakable FM_ACCOUNT_ROUTING_TEST_LAB opt-in is active. # Valid modes are off, observe, and enforce. -# Off does not invoke Agent Fleet. -# Observe performs only `choose --dry-run`, never creates a lease, never wraps -# the provider launch, and never writes managed account metadata. -# Enforce atomically reserves one profile after endpoint and worktree setup, -# immediately before provider launch, and fails closed on every Agent Fleet or -# validation error. +# New observe and enforce launches use bin/fm-account-directory.sh and do not +# invoke Agent Fleet. +# Agent Fleet selection and lease helpers remain as legacy recovery support and +# isolated dead-code fixtures pending remove-fleet-routing-deadcode. # # FM_AGENT_FLEET_BIN may name a deterministic fake only in tests/labs. # Production always opens the current passwd user's fixed regular front door at @@ -1012,7 +1010,7 @@ fm_account_safe_lineage_value() { fm_account_meta_key_owned() { # case "$1" in - window|worktree|project|harness|kind|mode|yolo|tasktmp|model|effort|report_required|generation_id|backend|tmux_window_id|tmux_session_target|account_pool|account_profile|account_task|account_attempt|account_predecessor_task|account_predecessor_attempt|account_predecessor_provider|account_predecessor_profile|account_predecessor_pool|account_predecessor_session|account_predecessor_cleanup|account_rollback_cleanup|account_rollback_backup|account_rollback_artifacts|account_rollback_preserve_session|continuation_packet|provider_session_id|herdr_session|herdr_workspace_id|herdr_tab_id|herdr_pane_id|zellij_session|zellij_tab_id|zellij_pane_id|orca_worktree_id|terminal|cmux_workspace_id|cmux_surface_id|home|projects|rollback_pending) return 0 ;; + window|worktree|worktree_git_dir|worktree_git_dir_identity|worktree_git_ref|worktree_git_head|worktree_git_setup_ref|worktree_git_setup_head|project|harness|kind|mode|yolo|tasktmp|model|effort|report_required|generation_id|backend|tmux_window_id|tmux_session_target|account_home|direct_spawn_cleanup|direct_spawn_backup|direct_spawn_artifacts|direct_recovery_cleanup|direct_recovery_backup|direct_recovery_artifacts|account_pool|account_profile|account_task|account_attempt|account_predecessor_task|account_predecessor_attempt|account_predecessor_provider|account_predecessor_profile|account_predecessor_pool|account_predecessor_session|account_predecessor_cleanup|account_rollback_cleanup|account_rollback_backup|account_rollback_artifacts|account_rollback_preserve_session|continuation_packet|provider_session_id|herdr_session|herdr_workspace_id|herdr_tab_id|herdr_pane_id|zellij_session|zellij_tab_id|zellij_pane_id|orca_worktree_id|terminal|cmux_workspace_id|cmux_surface_id|home|projects|rollback_pending) return 0 ;; *) return 1 ;; esac } diff --git a/bin/fm-bootstrap.sh b/bin/fm-bootstrap.sh index 17388bf7ff5..a849158afe5 100755 --- a/bin/fm-bootstrap.sh +++ b/bin/fm-bootstrap.sh @@ -35,8 +35,8 @@ # distinct from the endpoint pane-presence check): outcomes distinguish # no-op, successful respawn, explicit-routing deferral, skipped # recovery, and failed recovery. A confirmed-dead unmanaged generation -# is deferred until an operator chooses whether to preserve unmanaged -# routing or convert it to managed routing. Session-start scope only; +# is deferred until an operator chooses whether to preserve +# unmanaged routing or convert it to managed routing. Session-start scope only; # see AGENTS.md "Session start" and docs/tmux-backend.md / # docs/herdr-backend.md "Agent liveness probe" for the empirical basis. # A TANGLE line means the firstmate primary checkout (FM_ROOT) is stranded @@ -603,6 +603,7 @@ EOF BOOTSTRAP_JQ_REPORTED=0 ACCOUNT_ROUTING_MODE=off +ACCOUNT_ROUTING_NEEDS_DIRECT_TOOLS=0 ACCOUNT_ROUTING_NEEDS_AGENT_FLEET=0 CREW_DISPATCH_ROUTING_VALID=unknown @@ -614,26 +615,48 @@ account_routing_preflight() { mode_error=${mode#error: } echo "ACCOUNT_ROUTING: invalid routing policy - $mode_error" fi + [ "$ACCOUNT_ROUTING_MODE" = off ] || ACCOUNT_ROUTING_NEEDS_DIRECT_TOOLS=1 [ "$ACCOUNT_ROUTING_MODE" != enforce ] || ACCOUNT_ROUTING_NEEDS_AGENT_FLEET=1 } account_routing_dependency_preflight() { - local needs_agent_fleet=$ACCOUNT_ROUTING_NEEDS_AGENT_FLEET dispatch + local needs_direct=$ACCOUNT_ROUTING_NEEDS_DIRECT_TOOLS needs_agent_fleet=$ACCOUNT_ROUTING_NEEDS_AGENT_FLEET dispatch meta direct_perl dispatch="$CONFIG/crew-dispatch.json" if [ -f "$dispatch" ]; then if [ "$CREW_DISPATCH_ROUTING_VALID" = 1 ]; then - jq -e '.. | objects | select(has("account_pool") or has("account_profile"))' "$dispatch" >/dev/null 2>&1 && needs_agent_fleet=1 + jq -e '.. | objects | select(has("account_pool") or has("account_profile"))' "$dispatch" >/dev/null 2>&1 && needs_direct=1 elif [ "$CREW_DISPATCH_ROUTING_VALID" = unknown ] \ && grep -Eq '"account_(pool|profile)"[[:space:]]*:' "$dispatch" 2>/dev/null; then - needs_agent_fleet=1 + needs_direct=1 fi fi - [ "$needs_agent_fleet" = 1 ] || return 0 - fm_account_fleet_bin >/dev/null 2>&1 || missing_tool_diagnostic agent-fleet - if ! command -v jq >/dev/null 2>&1; then + for meta in "$STATE"/*.meta; do + [ -f "$meta" ] && [ ! -L "$meta" ] || continue + if grep -q '^account_home=.' "$meta" 2>/dev/null \ + && ! grep -qx 'kind=secondmate' "$meta" 2>/dev/null; then + needs_direct=1 + fi + if grep -q '^account_profile=' "$meta" 2>/dev/null \ + || grep -qx 'account_rollback_cleanup=pending' "$meta" 2>/dev/null; then + needs_agent_fleet=1 + fi + done + if [ "$needs_direct" = 1 ] && ! command -v jq >/dev/null 2>&1; then echo "MISSING: jq (install: $(install_cmd jq))" BOOTSTRAP_JQ_REPORTED=1 fi + if [ "$needs_direct" = 1 ] && ! command -v herdr >/dev/null 2>&1; then + missing_tool_diagnostic herdr + fi + direct_perl=/usr/bin/perl + if [ "${FM_ACCOUNT_DIRECTORY_TEST_LAB:-}" = firstmate-account-directory-test-lab-v1 ] \ + && [ -n "${FM_ACCOUNT_DIRECTORY_PERL_BIN:-}" ]; then + direct_perl=$FM_ACCOUNT_DIRECTORY_PERL_BIN + fi + if [ "$needs_direct" = 1 ] && [ ! -x "$direct_perl" ]; then + missing_tool_diagnostic perl + fi + [ "$needs_agent_fleet" = 0 ] || fm_account_fleet_bin >/dev/null 2>&1 || missing_tool_diagnostic agent-fleet } crew_dispatch_validate() { diff --git a/bin/fm-dispatch-select.sh b/bin/fm-dispatch-select.sh index f2dc5278ebb..a95fe07d336 100755 --- a/bin/fm-dispatch-select.sh +++ b/bin/fm-dispatch-select.sh @@ -11,15 +11,11 @@ # contract: # - Any candidate carrying account_profile is invalid because pinned profiles # are direct per-spawn overrides, never inputs to quota-balanced selection. -# - A candidate set carrying account_pool uses only Agent Fleet's no-secret -# `pool status` summaries. Every candidate must then carry account_pool and -# use claude/codex. Only a non-degraded, quota-fresh provider summary backed -# by at least one freshly proven eligible profile is available. The best -# adjusted headroom wins; exact ties use the first array element. Stale or -# otherwise degraded summaries are diagnostics only. Agent Fleet trouble -# degrades to the first element and never falls through to default-account -# quota-axi data; enforced spawn still obtains the real fresh lease before -# any provider launch. +# - A candidate set carrying account_pool uses the first array element and +# passes the pool only as a compatibility activation input. Spawn performs +# the real direct account-directory selection. New dispatch never calls +# Agent Fleet. The unreachable pool-summary implementation remains only for +# deferred removal under remove-fleet-routing-deadcode. # - Enforced account routing rejects quota-balanced candidates without pools. # Off and observe retain the legacy no-pool quota-axi path. # - Per candidate vendor it takes the minimum percentRemaining across that @@ -40,8 +36,8 @@ # # quota-balanced uses quota-axi --json unless --quota-json supplies a fixture. # FM_DISPATCH_QUOTA_AXI overrides the quota command. -# FM_DISPATCH_AGENT_FLEET and FM_AGENT_FLEET_BIN are test/lab-only. Production -# uses the fixed passwd-home ~/.local/bin/agent-fleet front door. +# FM_DISPATCH_AGENT_FLEET and FM_AGENT_FLEET_BIN belong only to the unreachable +# legacy pool-summary implementation pending remove-fleet-routing-deadcode. # FM_DISPATCH_STALE_CLEAR_MARGIN overrides the default 20 point stale margin. set -u @@ -87,18 +83,6 @@ log() { printf 'fm-dispatch-select: %s\n' "$*" >&2 } -if [ -n "${FM_DISPATCH_AGENT_FLEET_TIMEOUT:-}" ]; then - AGENT_FLEET_TIMEOUT=$FM_DISPATCH_AGENT_FLEET_TIMEOUT - case "$AGENT_FLEET_TIMEOUT" in - ''|*[!0-9]*|0) - echo "error: FM_DISPATCH_AGENT_FLEET_TIMEOUT must be a positive integer" >&2 - exit 2 - ;; - esac -else - AGENT_FLEET_TIMEOUT=$(fm_account_selection_timeout) || exit 2 -fi - while [ "$#" -gt 0 ]; do case "$1" in --select) @@ -220,6 +204,30 @@ if [ "$routing_mode" = enforce ]; then fi fi if [ "$pooled_count" -gt 0 ]; then + if [ "$pooled_count" -ne "$profile_count" ] || ! printf '%s\n' "$profiles_json" | "$FM_DISPATCH_JQ_BIN" -e 'all(.[]; (.account_pool | length) > 0 and (.harness == "claude" or .harness == "codex"))' >/dev/null 2>&1; then + log "account_pool quota-balanced candidates must all name claude/codex pools; using first profile" + first_profile + exit 0 + fi + log "account_pool is a direct-routing compatibility input; using the first profile and deferring account selection to spawn" + first_profile + exit 0 +fi + +# Legacy Agent Fleet pool summaries are unreachable and retained only until +# remove-fleet-routing-deadcode removes the wider selection and lease machinery. +if [ "$pooled_count" -gt 0 ]; then + if [ -n "${FM_DISPATCH_AGENT_FLEET_TIMEOUT:-}" ]; then + AGENT_FLEET_TIMEOUT=$FM_DISPATCH_AGENT_FLEET_TIMEOUT + case "$AGENT_FLEET_TIMEOUT" in + ''|*[!0-9]*|0) + echo "error: FM_DISPATCH_AGENT_FLEET_TIMEOUT must be a positive integer" >&2 + exit 2 + ;; + esac + else + AGENT_FLEET_TIMEOUT=$(fm_account_selection_timeout) || exit 2 + fi if [ "$pooled_count" -ne "$profile_count" ] || ! printf '%s\n' "$profiles_json" | "$FM_DISPATCH_JQ_BIN" -e 'all(.[]; (.account_pool | length) > 0 and (.harness == "claude" or .harness == "codex"))' >/dev/null 2>&1; then log "account_pool quota-balanced candidates must all name claude/codex pools; using first profile" first_profile diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index d78eb85e70a..30d2e5ac098 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -1,8 +1,10 @@ #!/usr/bin/env bash +# FM_ACCOUNT_DIRECTORY_CUTOVER: direct-observe-passwd-home-v2 # Spawn a direct report: a new crewmate in a treehouse worktree, an eligible # pre-cutover Orca respawn, or a secondmate in its isolated firstmate home. # Usage: fm-spawn.sh [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] [--account-pool ] [--account-profile ] [--no-account-routing] [--scout] # fm-spawn.sh [] [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] [--account-pool ] [--account-profile ] [--no-account-routing] --secondmate +# fm-spawn.sh --recover-direct-account # fm-spawn.sh (--resume-account|--continue-account) [--harness ] [--account-pool ] [--account-profile ] # --harness is the explicit per-spawn harness/profile adapter. The old # positional harness arg still works for back-compat. @@ -49,27 +51,30 @@ # respawn exactly like the harness axis, and explicit --model/--effort flags # still win over the file's tokens. # Account routing is independently default-off. Its precedence and off/observe/ -# enforce behavior are owned by fm-account-routing-lib.sh. --account-pool asks -# Agent Fleet to atomically select one concrete profile; --account-profile pins -# a concrete profile (and optionally validates it belongs to --account-pool). -# Either explicit account flag enforces routing for this spawn even when the -# global mode is off/observe. --no-account-routing is the emergency per-spawn -# opt-out and cannot be combined with either account flag. Enforced routing is -# supported only for claude/codex and fails closed; it never silently launches -# the default account. The resolved profile wraps the provider command before -# it is submitted to the selected backend, so every backend receives the same -# launch string. -# config/secondmate-account-pool is the primary's durable, non-inherited pool -# for secondmate AGENTS. An explicit account flag overrides it. A secondmate's -# own crewmates use inherited crew dispatch/routing policy, not this pool. -# --resume-account is an internal recovery path. It requires existing sticky -# account/profile/session metadata plus Agent Fleet's matching SessionStart -# mapping, reuses the recorded worktree/home, and executes `agent-fleet resume -# --task`; any missing or mismatched recovery truth blocks before pane creation. -# --continue-account is the provider-neutral recovery path. It verifies a dead -# endpoint and current repository state, builds a task-owned continuation packet, -# launches a fresh provider session through a new namespaced Agent Fleet attempt, -# and releases the predecessor only after the new SessionStart mapping is bound. +# enforce resolution is owned by fm-account-routing-lib.sh. Direct account- +# directory launch currently covers ship/scout crewmates only; secondmate +# integration is deferred and retains legacy Agent Fleet routing. +# For a NEW routed Claude or Codex ship/scout, fm-account-directory.sh discovers +# the current user's account homes, chooses one through its direct per-vendor +# usage contract, installs that profile's Herdr hook, and prefixes the provider +# command with CLAUDE_CONFIG_DIR or CODEX_HOME. +# Existing --account-pool and --account-profile inputs remain compatibility +# activation signals for new direct launches; their aliases do not constrain +# the direct usage choice. --no-account-routing remains the emergency per-spawn +# opt-out and cannot be combined with either account flag. Off launches retain +# their existing default-identity behavior. +# config/secondmate-account-pool remains the primary's durable, non-inherited +# Agent Fleet selection input for secondmate agents when routing is enabled. A +# secondmate's own crewmates use inherited crew dispatch/routing policy, not +# this setting. +# --resume-account and --continue-account are legacy recovery paths only for +# existing account_profile metadata. They retain the sealed Agent Fleet +# session/lease behavior needed to recover those already-managed generations; +# ship/scout launches never create that metadata. +# --recover-direct-account is the ship/scout account_home recovery path. It reloads kind, +# project, worktree, harness, backend, model, effort, mode, yolo, and report +# requirements from metadata, selects a fresh account directory, and creates +# only a replacement endpoint in the recorded worktree. # A --secondmate spawn also propagates the primary's declared inheritable config # into the secondmate home's config/, so the secondmate's OWN crewmates, # dispatch profiles, and backlog backend inherit the primary's settings @@ -158,10 +163,164 @@ spawn_managed_endpoint_state() { #