Skip to content

fix: automatically start Crosscheck reviews on PR registration - #373

Merged
ruby-dlee merged 6 commits into
mainfrom
fm/crosscheck-autostart-g4
Aug 27, 2026
Merged

fix: automatically start Crosscheck reviews on PR registration#373
ruby-dlee merged 6 commits into
mainfrom
fm/crosscheck-autostart-g4

Conversation

@ruby-dlee

Copy link
Copy Markdown
Owner

Intent

Fix the review-ready merge flow so the normal PR registration command automatically starts the required exact-head independent Crosscheck review rather than only arming a merge poll and leaving the gate blocked until a separate manual command. Registration must return promptly while the review runs asynchronously. Repeated starts for the same task, PR URL, and head must reuse a live matching review, skip a matching CLEAR result, and allow retry after a dead or failed launcher; a changed PR head must start a fresh review. Reviews must be isolated per task/head with no global serialization, preserving Azure lane capacity and cost admission. Missing, unsafe, or incomplete private Azure configuration must not fail PR registration or block unrelated review starts: record an actionable per-task failure, keep the merge gate uncleared, and allow the same command to retry after repair. The launcher must load the authoritative operator-private fleet configuration through an owned safe mechanism without copying secrets into tracked files, prompts, logs, or durable records. Crosscheck must never merge, approve, or weaken exact-current-head CLEAR gating. Add executable behavior regressions for prompt return, matching-run deduplication, launcher-failure retry, new-head restart, and concurrent unrelated starts, avoiding source-substring assertions. Update only the owning docs and script headers, not AGENTS.md. Preserve the concrete single-operator design constraint: non-security launcher failures are visible and retryable rather than globally blocking, while credential custody and spending admission remain fail-closed.

What Changed

  • Automatically launch asynchronous exact-head Crosscheck reviews during PR registration, reusing active reviews and verified CLEAR results while queuing changed heads independently per task.
  • Safely load operator-private fleet configuration and surface retryable launcher failures without failing registration; reject moved heads before review spending and preserve exact-head merge gating.
  • Document the registration flow and add behavior regressions for prompt return, deduplication, failure recovery, head changes, concurrent starts, and coordinator handoff races.

Risk Assessment

🚨 High: Captain, the source repairs appear consistent with the requested invariants, but the required semantic-repair proof remains explicitly unverified and contradictory.

Testing

The supplied baseline and focused tests passed. Executable checks covered asynchronous registration, deduplication, configuration and dead-worker retries, changed heads, unrelated concurrent starts, all three race regressions, generated poll consumption and merged detection, plus moved-head refusal before spending. Captured CLI and state evidence uses controlled GitHub/reviewer fixtures; no live Azure review was performed. No UI changed.

Evidence: Registration CLI output, review calls, and persisted coordinator states from controlled fixtures

--- fm-pr-crosscheck-autostart.qrUZfr/capture-order/first.out ---
armed: state/ordered.check.sh polls https://github.com/example/repo/pull/8
crosscheck autostart: started ordered at 1111111111111111111111111111111111111111 (pid 60349)

--- fm-pr-crosscheck-autostart.qrUZfr/capture-order/second.out ---
armed: state/ordered.check.sh polls https://github.com/example/repo/pull/8
crosscheck autostart: queued new head 2222222222222222222222222222222222222222 for active task ordered

--- fm-pr-crosscheck-autostart.qrUZfr/status-race/control/calls ---
verify	statusrace	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/9
run	statusrace	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/9
verify	statusrace	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/9

--- fm-pr-crosscheck-autostart.qrUZfr/status-race/home/state/statusrace.crosscheck-autostart.json ---
{"attempt":1,"generation_id":"generation-statusrace","head_sha":"1111111111111111111111111111111111111111","log":"statusrace.crosscheck-autostart.log","message":"exact-head Crosscheck completed CLEAR","pid":60506,"pull_request":"https://github.com/example/repo/pull/9","schema":"firstmate.crosscheck-autostart.v1","state":"clear","task_id":"statusrace","updated_at":"2026-08-27T04:41:34Z"}

--- fm-pr-crosscheck-autostart.qrUZfr/capture-order/control/calls ---
verify	ordered	2222222222222222222222222222222222222222	https://github.com/example/repo/pull/8
verify	ordered	2222222222222222222222222222222222222222	https://github.com/example/repo/pull/8
run	ordered	2222222222222222222222222222222222222222	https://github.com/example/repo/pull/8
verify	ordered	2222222222222222222222222222222222222222	https://github.com/example/repo/pull/8

--- fm-pr-crosscheck-autostart.qrUZfr/capture-order/home/state/ordered.crosscheck-autostart.json ---
{"attempt":2,"generation_id":"generation-ordered","head_sha":"2222222222222222222222222222222222222222","log":"ordered.crosscheck-autostart.log","message":"exact-head Crosscheck completed CLEAR","pid":60349,"pull_request":"https://github.com/example/repo/pull/8","schema":"firstmate.crosscheck-autostart.v1","state":"clear","task_id":"ordered","updated_at":"2026-08-27T04:41:33Z"}

--- fm-pr-crosscheck-autostart.qrUZfr/retirement/control/calls ---
verify	retire	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/7
run	retire	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/7
verify	retire	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/7
verify	retire	2222222222222222222222222222222222222222	https://github.com/example/repo/pull/7
run	retire	2222222222222222222222222222222222222222	https://github.com/example/repo/pull/7
verify	retire	2222222222222222222222222222222222222222	https://github.com/example/repo/pull/7

--- fm-pr-crosscheck-autostart.qrUZfr/retirement/home/state/retire.crosscheck-autostart.json ---
{"attempt":2,"generation_id":"generation-retire","head_sha":"2222222222222222222222222222222222222222","log":"retire.crosscheck-autostart.log","message":"exact-head Crosscheck completed CLEAR","pid":60134,"pull_request":"https://github.com/example/repo/pull/7","schema":"firstmate.crosscheck-autostart.v1","state":"clear","task_id":"retire","updated_at":"2026-08-27T04:41:32Z"}

--- fm-pr-crosscheck-autostart.qrUZfr/prompt-return/control/calls ---
verify	preturn	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/1
run	preturn	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/1
verify	preturn	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/1
verify	preturn	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/1
verify	preturn	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/1

--- fm-pr-crosscheck-autostart.qrUZfr/prompt-return/home/state/preturn.crosscheck-autostart.json ---
{"attempt":3,"generation_id":"generation-preturn","head_sha":"1111111111111111111111111111111111111111","log":"preturn.crosscheck-autostart.log","message":"matching exact-head CLEAR review already exists","pid":61399,"pull_request":"https://github.com/example/repo/pull/1","schema":"firstmate.crosscheck-autostart.v1","state":"clear","task_id":"preturn","updated_at":"2026-08-27T04:41:36Z"}

--- fm-pr-crosscheck-autostart.qrUZfr/retry/control/calls ---
verify	retry	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/2
verify	retry	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/2
verify	retry	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/2
verify	retry	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/2
run	retry	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/2
verify	retry	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/2

--- fm-pr-crosscheck-autostart.qrUZfr/retry/home/state/retry.crosscheck-autostart.json ---
{"attempt":4,"generation_id":"generation-retry","head_sha":"1111111111111111111111111111111111111111","log":"retry.crosscheck-autostart.log","message":"exact-head Crosscheck completed CLEAR","pid":62160,"pull_request":"https://github.com/example/repo/pull/2","schema":"firstmate.crosscheck-autostart.v1","state":"clear","task_id":"retry","updated_at":"2026-08-27T04:41:38Z"}

--- fm-pr-crosscheck-autostart.qrUZfr/dead-coordinator/control/calls ---
verify	dead	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/3
run	dead	1111111111111111111111111111111111111111	https://github.com/example/repo/pull/3

--- fm-pr-crosscheck-autostart.qrUZfr/dead-coordinator/home/state/dead.crosscheck-autostart.json ---
{"attempt":2,"generation_id":"generation-dead","head_sha":"2222222222222222222222222222222222222222","log":"dead.crosscheck-autostart.log","message":"Crosscheck coordinator stopped before the requested head started; rerun fm-pr-check.sh","pid":0,"pull_request":"https://github.com/example/repo/pull/3","schema":"firstmate.crosscheck-autostart.v1","state":"failed","task_id":"dead","updated_at":"2026-08-27T04:41:40Z"}

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

🔧 **Rebase** - 1 issue found → auto-fixed ✅
  • ⚠️ AGENTS.md - merge conflict rebasing onto origin/main

🔧 Fix applied.
✅ Re-checked - no issues remain.

⚠️ **Review** - 2 issues (1 error, 1 warning)
  • 🚨 bin/fm-crosscheck-autostart.py:574 - A queued head can lose its coordinator: worker A reads the unchanged request at line 574 and decides to exit; registration B then fails to acquire A's still-held lock, writes a new head, and returns success; A closes its lock without reading B's request. No review starts until another registration. This contradicts “a changed PR head must start a fresh review.” Make request publication and coordinator retirement an atomic handoff, and cover this interleaving through the executable interface.
  • ⚠️ bin/fm-pr-check.sh:112 - A persisted launcher failure now prevents the check from ever querying GitHub. If the operator subsequently completes Crosscheck through the supported manual command and merges the PR, the autostart record remains failed, so every poll reports the old failure instead of 'merged'. This suppresses the watcher's merged-PR cleanup path. Preserve live merge detection independently of launcher diagnostics; it does not grant merge authorization.

🔧 Fix: Fix Crosscheck retirement handoff and live merge detection
4 issues (2 errors, 2 warnings) still open:

  • 🚨 bin/fm-crosscheck-autostart.py:642 - Concurrent registrations can replace a newer head with an older one. Registration A resolves head H1 and pauses before start; B resolves H2 and starts its coordinator; A resumes and unconditionally publishes H1 here. The coordinator then attempts H1, refuses because GitHub is at H2, and retires without reviewing H2. This contradicts “a changed PR head must start a fresh review.” Preserve registration ordering across head capture and request publication, and add an executable regression for this interleaving.
  • ⚠️ bin/fm-crosscheck-autostart.py:762 - Status can overwrite successful completion with a permanent false failure: it reads a running record, the worker writes clear and releases its lock, then status observes the unlocked coordinator and writes failed using its stale record. Subsequent polls keep reporting failure until registration is repeated. Serialize status reconciliation with coordinator retirement and re-read the record before changing it.
  • 🚨 tests/fm-pr-crosscheck-autostart.test.sh:440 - The supplied repair evidence does not establish the required fail-before/pass-after result: both executions exited zero, and repair_complete is false. The retirement regression exists, but its execution against the pre-fix implementation has not been demonstrated. Have the outer executor run the new regression against both implementations and resolve the unverified proof before accepting the repair.
  • ⚠️ bin/fm-crosscheck-autostart.py - The fixer could not prove the semantic repair with a public/executable fail-before/pass-after regression and relevant integration or consumer compatibility evidence; primary-agent handoff is required.

🔧 Fix: Preserve Crosscheck registration order and completion state
2 issues (1 error, 1 warning) still open:

  • 🚨 tests/fm-pr-crosscheck-autostart.test.sh:440 - The repair proof remains contradictory: executor evidence reports repair_complete=false, proof_status=unverified, and exit=0 for both before and after, while state/review-proof logs show targeted failures and passes without recording their implementation revisions or exit codes. Have the outer executor reconcile and verify the targeted fail-before/pass-after evidence before accepting the semantic repair; the generated polls are correctly confined to test-local state.
  • ⚠️ bin/fm-crosscheck-autostart.py - The fixer could not prove the semantic repair with a public/executable fail-before/pass-after regression and relevant integration or consumer compatibility evidence; primary-agent handoff is required.
✅ **Test** - passed

✅ No issues found.

  • if [ "${FM_AZURE_VALIDATION_CELL:-0}" = 1 ]; then exec "$FM_AZURE_VALIDATION_SHARD_BRIDGE" behavior --count "${FM_AZURE_VALIDATION_SHARD_COUNT:-8}"; else exec bin/fm-no-mistakes-test-command.sh; fi
  • Configured baseline command previously passed, as supplied by the outer executor.
  • TMPDIR="$PWD/state/local-test-tmp" tests/run.sh tests/fm-pr-crosscheck-autostart.test.sh
  • TMPDIR="$PWD/state/local-test-tmp" FM_TEST_CASE=test_registered_expected_head_refuses_a_moved_head_before_spend tests/run.sh tests/fm-crosscheck.test.sh
  • Captured generated registration output, review invocation records, and persisted coordinator state during execution.
  • git status --short confirmed a clean worktree after cleanup.
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@ruby-dlee
ruby-dlee force-pushed the fm/crosscheck-autostart-g4 branch from 27c88c0 to f1165f8 Compare August 27, 2026 04:53
@ruby-dlee
ruby-dlee merged commit 2a749d3 into main Aug 27, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant