Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ Require the matching `resolved` event, forbid `--yes`, and require the worker to
Resume fleet supervision immediately after the decision lands.

Judge validation by the current-code-matched run step through `bin/fm-crew-state.sh`, not by shell liveness or the last status event.
Running, fixing, or CI states remain working; parked approval or fix-review states require the worker to follow the active gate help; passed or checks-passed is done; failed or cancelled is failed.
Running, fixing, or CI states remain working; parked approval or fix-review states require the worker to follow the active gate help; passed is done; a checks-passed claim is done only when `bin/fm-crew-state.sh` scores at least one successful check and no pending, failed, cancelled, skipped, or never-run conclusions; a run that only reports completion, with no outcome to score, is unknown rather than done; failed or cancelled is failed.
A worker hand-editing, committing, aborting, or restarting during an active validation run duplicates pipeline ownership outside the supersession sequence above; steer it back to the gate response flow.
The worker reports the PR when CI first becomes green rather than waiting for merge monitoring to finish.

Expand Down
60 changes: 60 additions & 0 deletions bin/fm-ci-verdict-lib.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#!/usr/bin/env bash
# Shared CI-ready scoring for a set of check conclusions.
#
# ONE owner of the rule that a green verdict is valid only when at least one
# check completed with success AND none are pending, failed, cancelled,
# skipped, or never-run. Zero checks is a distinct non-success state, never
# passed. Used by current-state reporting so a cancelled-only or empty check
# set cannot be read as checks-passed.
#
# Public interface: fm_ci_ready_verdict [conclusion...]
# Reads conclusions from the arguments, or from stdin (one per line) when
# none are given. Blank lines are ignored.
# Prints exactly one of: passed | empty | not-passed
# Sourced by callers; also runnable as a command so tests drive the same
# public function through an executable interface.

fm_ci_verdict_trim() {
local s=${1:-}
s="${s#"${s%%[![:space:]]*}"}"
s="${s%"${s##*[![:space:]]}"}"
printf '%s' "$s"
}

# 0 if $1 is a completed success conclusion.
fm_ci_verdict_is_success() {
case "$1" in
SUCCESS|success) return 0 ;;
*) return 1 ;;
esac
}

fm_ci_ready_verdict() {
local raw conclusion saw_success=0
if [ "$#" -eq 0 ]; then
while IFS= read -r raw || [ -n "$raw" ]; do
set -- "$@" "$raw"
done
fi
for raw in "$@"; do
conclusion=$(fm_ci_verdict_trim "$raw")
[ -n "$conclusion" ] || continue
if fm_ci_verdict_is_success "$conclusion"; then
saw_success=1
continue
fi
printf 'not-passed'
return 0
done
if [ "$saw_success" = 1 ]; then
printf 'passed'
else
printf 'empty'
fi
}

if [ "${BASH_SOURCE[0]}" = "$0" ]; then
set -u
fm_ci_ready_verdict "$@"
printf '\n'
fi
92 changes: 65 additions & 27 deletions bin/fm-crew-state.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,18 @@
# diverged from it, invalidates attribution.
# The run-step is AUTHORITATIVE: running/fixing -> working, ci -> working,
# awaiting_approval/fix_review -> parked (with gate findings), terminal
# passed/checks-passed -> done, failed/cancelled -> failed. EXCEPT: while
# the active step is ci, `axi status` alone cannot tell "still waiting on
# checks" from "checks green, waiting on merge" (see nm_ci_checks_state) -
# a ci-step log-tail check overrides working -> done once checks read
# green, so a green PR is never silently read as still-validating.
# passed -> done, failed/cancelled -> failed. A terminal run that carries
# no outcome at all (a bare `completed` status, whether from `axi status`
# or the coarse runs list) reads unknown, never done: there is no verdict
# to score. A checks-passed outcome is
# done only when the CI-ready scorer in bin/fm-ci-verdict-lib.sh confirms
# at least one successful check and no pending, failed, cancelled,
# skipped, or never-run conclusions; zero checks is a distinct non-success
# state, never done. EXCEPT: while the active step is ci, `axi status`
# alone cannot tell "still waiting on checks" from "checks green, waiting
# on merge" (see nm_ci_checks_state) - a ci-step log-tail check overrides
# working -> done only when that same scorer reads green, so a cancelled
# or empty check set is never silently read as still-validating-and-green.
# 3. Reconcile the status log: if its last line says needs-decision/blocked but
# the run-step shows the run moved on, the log is deterministically stale and
# is flagged superseded. A genuinely parked run plus a needs-decision log
Expand Down Expand Up @@ -66,6 +73,8 @@ STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}"
. "$SCRIPT_DIR/fm-busy-lib.sh"
# shellcheck source=bin/fm-nm-run-lib.sh
. "$SCRIPT_DIR/fm-nm-run-lib.sh"
# shellcheck source=bin/fm-ci-verdict-lib.sh
. "$SCRIPT_DIR/fm-ci-verdict-lib.sh"

ID=${1:-}
[ -n "$ID" ] || { echo "usage: fm-crew-state.sh <id>" >&2; exit 2; }
Expand Down Expand Up @@ -281,26 +290,37 @@ nm_effective_ci_step_status() {
# actually merged (or failed/cancelled if closed). `axi status`'s steps[] table
# never distinguishes "still waiting on checks" from "checks green, waiting on
# merge": both read as plain `ci,running,...`. The only place that transition is
# recorded is the ci step's own log text, e.g. "all CI checks passed - still
# monitoring until merged or closed" or "no CI checks reported - still
# monitoring until merged or closed" (verified against 360+ real run logs under
# ~/.no-mistakes/logs/*/ci.log on the installed v1.32.2 binary, including the
# actual PR #252 run). Reads the ci step's log tail via `axi logs` and scans it
# for the MOST RECENT recognized marker (the log is append-only/chronological,
# so the last match is current): green with nothing red after it means CI is
# green right now, still only waiting on merge/close.
# recorded is the ci step's own log text. Reads the ci step's log tail via
# `axi logs` and scores the MOST RECENT recognized marker (the log is
# append-only/chronological, so the last match is current). Green is valid
# only when bin/fm-ci-verdict-lib.sh sees at least one success and no
# pending, failed, cancelled, skipped, or never-run conclusions. Zero checks
# is a distinct non-success state, never green.
nm_ci_marker_conclusion() {
case "$1" in
*"all CI checks passed"*) printf 'SUCCESS' ;;
*"CI check cancelled"*|*"CI checks were cancelled"*|*"cancelled without"*) printf 'CANCELLED' ;;
*"checks failed"*|*"CI failures"*|*"issues detected"*) printf 'FAILURE' ;;
*"CI checks running"*|*"waiting for checks"*|*"base branch advanced"*"re-arming CI monitor timeout"*) printf 'PENDING' ;;
*"no CI checks reported"*|*"repository declares no CI"*) ;;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve trusted no_ci checks-passed verdicts

Captain, for repos using no-mistakes' trusted no_ci: true mode, the upstream skill says checks-passed is valid when that declaration covers a zero-check repo (source), and this branch even selects repository declares no CI; mapping it to an empty conclusion makes fm_ci_ready_verdict return empty/not-ready, so a legitimate checks-passed run is reported blocked (or stays working in the CI monitor) and the PR is never surfaced as ready. Please distinguish the trusted declaration from a generic no CI checks reported line and score it as the accepted success case.

Useful? React with 👍 / 👎.

esac
}

nm_ci_checks_state() {
local run_id log_tail marker
local run_id log_tail marker conclusion verdict
run_id=$(strip_quotes "$(nm_field id)")
[ -n "$run_id" ] || { printf 'unknown'; return; }
log_tail=$(nm_run axi logs --step ci --run "$run_id") || true
[ -n "$log_tail" ] || { printf 'unknown'; return; }
marker=$(printf '%s\n' "$log_tail" \
| grep -E 'CI checks passed|no CI checks reported - still monitoring|no CI checks reported yet|checks failed|issues detected|CI checks running|base branch advanced.*re-arming CI monitor timeout' \
| grep -E 'CI checks passed|no CI checks reported|repository declares no CI|checks failed|CI failures|issues detected|CI checks running|waiting for checks|base branch advanced.*re-arming CI monitor timeout|CI check cancelled|CI checks were cancelled|cancelled without' \
| tail -1)
case "$marker" in
*"checks passed"*|*"no CI checks reported - still monitoring"*) printf 'green' ;;
*"no CI checks reported yet"*|*"checks failed"*|*"issues detected"*|*"CI checks running"*|*"base branch advanced"*"re-arming CI monitor timeout"*) printf 'not-ready' ;;
[ -n "$marker" ] || { printf 'unknown'; return; }
conclusion=$(nm_ci_marker_conclusion "$marker")
verdict=$(fm_ci_ready_verdict "$conclusion")
case "$verdict" in
passed) printf 'green' ;;
empty|not-passed) printf 'not-ready' ;;
*) printf 'unknown' ;;
esac
}
Expand Down Expand Up @@ -423,15 +443,20 @@ if [ "$HAVE_RUN" = 1 ]; then
RUN_STATUS=""
if [ "$RUN_SOURCE" = coarse ]; then
# No step/gate detail is available from the plain runs list - only ever
# true/working, done, or failed. A crew genuinely parked at a gate still
# working, failed, or unknown. A crew genuinely parked at a gate still
# gets full detail once `axi status` reports its own branch again (e.g.
# once its own step is the most-recently-touched one), and its own
# needs-decision/blocked status-log append (a captain-relevant VERB) is
# surfaced through signal_reason_is_actionable regardless of this
# coarse-vs-full distinction, so a real gate is never silently missed.
# A `completed` row carries no outcome and no check evidence, and the
# ci-log scorer cannot run here ($RUN_OUT holds another run's id), so it
# cannot distinguish a merged run from one that claimed checks-passed over
# cancelled or zero checks. Reporting unknown keeps the same invariant the
# scorer enforces on the full path: done requires scored green evidence.
case "$COARSE_STATUS" in
running) RUN_STATE=working; RUN_DETAIL="validating (background run)" ;;
completed) RUN_STATE="done"; RUN_DETAIL="run completed" ;;
completed) RUN_STATE=unknown; RUN_DETAIL="run completed without check evidence" ;;
failed) RUN_STATE=failed; RUN_DETAIL="run failed" ;;
cancelled) RUN_STATE=failed; RUN_DETAIL="run cancelled" ;;
*) RUN_STATE=unknown; RUN_DETAIL="runs list status: $COARSE_STATUS" ;;
Expand All @@ -448,7 +473,23 @@ if [ "$HAVE_RUN" = 1 ]; then
if [ -n "$outcome" ]; then
case "$outcome" in
passed) RUN_STATE="done"; RUN_DETAIL="run passed: PR merged/closed" ;;
checks-passed) RUN_STATE="done"; RUN_DETAIL="checks green: PR ready for review" ;;
checks-passed)
CI_LOG_STATE=$(nm_ci_checks_state)
case "$CI_LOG_STATE" in
green)
RUN_STATE="done"
RUN_DETAIL="checks green: PR ready for review"
;;
not-ready)
RUN_STATE=blocked
RUN_DETAIL="run reported checks-passed without green check evidence"
;;
*)
RUN_STATE=unknown
RUN_DETAIL="run reported checks-passed but CI evidence is unavailable"
;;
esac
;;
failed) RUN_STATE=failed; RUN_DETAIL="run failed" ;;
cancelled) RUN_STATE=failed; RUN_DETAIL="run cancelled" ;;
*) RUN_STATE=unknown; RUN_DETAIL="outcome: $outcome" ;;
Expand All @@ -472,7 +513,7 @@ if [ "$HAVE_RUN" = 1 ]; then
case "$status" in
ci) RUN_STATE=working; RUN_DETAIL="ci running" ;;
running|fixing) RUN_STATE=working; RUN_DETAIL="validating ($status)" ;;
completed) RUN_STATE="done"; RUN_DETAIL="run completed" ;;
completed) RUN_STATE=unknown; RUN_DETAIL="run completed without an outcome" ;;
failed) RUN_STATE=failed; RUN_DETAIL="run failed" ;;
cancelled) RUN_STATE=failed; RUN_DETAIL="run cancelled" ;;
"") RUN_STATE=working; RUN_DETAIL="run active" ;;
Expand All @@ -496,10 +537,7 @@ if [ "$HAVE_RUN" = 1 ]; then
fi
fi

if [ "$RUN_STATE" = working ] && log_reports_ci_ready; then
if [ "$RUN_SOURCE" = coarse ]; then
emit "done" status-log "$(status_line_note "$LOG_LINE")${SEP}run still monitoring PR"
fi
if [ "$RUN_STATE" = working ] && log_reports_ci_ready && [ "$RUN_SOURCE" != coarse ]; then
[ -n "$CI_STEP_STATUS" ] || CI_STEP_STATUS=$(nm_effective_ci_step_status)
if [ "$RUN_STATUS" = fixing ]; then
CI_LOG_STATE=not-ready
Expand All @@ -508,7 +546,7 @@ if [ "$HAVE_RUN" = 1 ]; then
elif [ "$CI_STEP_STATUS" = fixing ]; then
CI_LOG_STATE=not-ready
fi
if [ "$CI_LOG_STATE" != not-ready ]; then
if [ "$CI_LOG_STATE" = green ]; then
emit "done" status-log "$(status_line_note "$LOG_LINE")${SEP}run still monitoring PR"
fi
fi
Expand Down
23 changes: 22 additions & 1 deletion bin/fm-remote-job-worker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,21 @@ worker_recover_quarantine() { # <account-home>
rm -f -- "$WORKER_LOCK/quarantine"
}

# An owner that died between mktemp and mv leaves one of its own private temp
# entries inside the lock directory, and rmdir can never reclaim a lock that
# still holds one. Only this worker's own temp shapes are removable, and only as
# plain files, so a proven-stale lock stays reclaimable after any unclean death
# without widening what reclaim may delete.
worker_clear_lock_residue() {
local entry
for entry in "$WORKER_LOCK"/.pid.* "$WORKER_LOCK"/.start.* \
"$WORKER_LOCK"/.command.* "$WORKER_LOCK"/.quarantine.*; do
[ -e "$entry" ] || [ -L "$entry" ] || continue
[ ! -L "$entry" ] && [ -f "$entry" ] || return 1
rm -f -- "$entry" || return 1
done
}

worker_acquire_lock() {
local account_home=$1 attempt=0
while [ "$attempt" -lt 150 ]; do
Expand All @@ -164,6 +179,7 @@ worker_acquire_lock() {
fi
[ ! -L "$WORKER_LOCK/pid" ] && [ ! -L "$WORKER_LOCK/start" ] && [ ! -L "$WORKER_LOCK/command" ] || return 1
rm -f -- "$WORKER_LOCK/pid" "$WORKER_LOCK/start" "$WORKER_LOCK/command" || return 1
worker_clear_lock_residue || return 1
rmdir "$WORKER_LOCK" || return 1
done
return 1
Expand Down Expand Up @@ -291,8 +307,13 @@ worker_stop_active_execution() {
WORKER_ACTIVE_JOB=
}

# A replacement stop signals the whole worker tree and the supervisor signals
# this child again, so a second delivery always lands. Ignoring it for the
# shutdown sequence - rather than restoring the default disposition, which lets
# that second signal kill this process between publishing and clearing guarded
# ownership - is what keeps the sequence atomic and the lock free of residue.
worker_shutdown() {
trap - HUP INT TERM
trap '' HUP INT TERM
worker_publish_quarantine || {
worker_error "cannot guard worker ownership for shutdown"
trap worker_shutdown HUP INT TERM
Expand Down
5 changes: 4 additions & 1 deletion bin/fm-test-run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ family_for_basename() {
fm-arm-pretool-check.test.sh|fm-ask-user-authority.test.sh|\
fm-brief.test.sh|fm-vendor-auth-probe.test.sh|\
fm-calm-pi-extension.test.sh|fm-cd-pretool-check.test.sh|\
fm-classify-decision-key.test.sh|\
fm-ci-verdict-lib.test.sh|fm-classify-decision-key.test.sh|\
fm-composer-ghost.test.sh|fm-composer-lib.test.sh|\
fm-crew-state.test.sh|fm-decision-hold-lifecycle.test.sh|\
fm-documentation-audiences.test.sh|fm-ensure-agents-md.test.sh|fm-grok-harness.test.sh|\
Expand Down Expand Up @@ -938,6 +938,9 @@ families_for_changed_path() {
printf '%s\n' pure-contract-unit
printf '%s\n' pr-forge
;;
bin/fm-ci-verdict-lib.sh)
printf '%s\n' pure-contract-unit
;;
bin/fm-composer-lib.sh)
# The shared shape catalogue is vendor-rendered signal; a change to it
# re-selects the live guard (fm-composer-matrix-live-e2e) alongside the
Expand Down
4 changes: 3 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,9 @@ Any direct or remaining historical annotation prints every status line unread at
`bin/fm-crew-state.sh <id>` is the cheap current-state read for an actionable heartbeat review: it attributes a no-mistakes run, active or terminal, only when it matches the crew's branch and current code identity, then keeps that run-step authoritative even if the pane has closed.
The script header owns the exact run-head ancestry rules.
During no-mistakes' `ci` monitor phase, it also reads the ci step log tail because `axi status` reports both "still waiting on checks" and "checks green, waiting on merge" as `ci,running`.
The most recent recognized ci log marker wins, so checks-green monitoring reports done while a later re-arm, failed-check, or issue marker returns the crew to working.
The most recent recognized ci log marker wins, and `bin/fm-ci-verdict-lib.sh` scores it: only a completed success with no pending, failed, cancelled, skipped, or never-run conclusions reports done, while a cancelled, empty, or later re-arm, failed-check, or issue marker returns the crew to working.
A terminal `checks-passed` outcome is scored through that same owner, so a claim the scorer does not read as green reports blocked, and a missing or unreadable ci log reports unknown, instead of done.
A terminal run that carries no outcome at all, including the coarse runs-list `completed` row that has no step, gate, or check evidence to score, reads unknown rather than done.
Only when no matching run exists does it consult semantic busy state; exact busy reports working, exact idle permits fallback to a status-log event whose verb maps to a recognized run-state, and unknown or a dead pane stays unknown instead of trusting a stale log.
Decision-only events such as `resolved` never become current state or leak their prose into the current-state detail.
In that status-log fallback, a declared external wait reports the distinct `paused` state with its reason.
Expand Down
1 change: 1 addition & 0 deletions docs/scripts.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ The shared no-mistakes gate refusal for fleet lifecycle entrypoints is summarize
| `fm-supervise-daemon.sh` | Presence-gated away-mode sub-supervisor: self-handle routine wakes, guard injection by the detected primary harness, escalate batched digests, alert on failed delivery |
| `fm-crew-state.sh` | Print one deterministic current-state line for a crew |
| `fm-nm-run-lib.sh` | Shared branch-and-code-identity attribution for no-mistakes runs |
| `fm-ci-verdict-lib.sh` | Score a CI check-conclusion set so empty or cancelled results cannot read as passed |
| `fm-tangle-lib.sh` | Shared default-branch resolution and primary-checkout tangle classification |
| `fm-timeout-lib.sh` | Single owner of hard-bounded command execution and its fallback watchdog |
| `fm-timing-lib.sh` | Single owner of the deferred network stage's per-step elapsed-time records, inert unless a run asks for them |
Expand Down
Loading
Loading