Skip to content

fix(crosscheck): enforce reliable two-pass GLM reviews - #330

Merged
ruby-dlee merged 8 commits into
mainfrom
fm/azure-c1-accept-y4
Aug 25, 2026
Merged

fix(crosscheck): enforce reliable two-pass GLM reviews#330
ruby-dlee merged 8 commits into
mainfrom
fm/azure-c1-accept-y4

Conversation

@ruby-dlee

@ruby-dlee ruby-dlee commented Aug 25, 2026

Copy link
Copy Markdown
Owner

Intent

Complete C1 with honest post-merge evidence inside its required 20-to-30-minute band. Preserve regular Fireworks GLM 5.2 at xhigh, the full exact-base/exact-head diff, and the fixed two-pass independent challenge plus authoritative synthesis protocol; never use Fast GLM, sleep to manufacture latency, narrow review scope, lower reasoning, or weaken evidence and refusal gates. Require terminal_provider, terminal_model, review_depth_passes, and review_depth_mode for successful current-contract regular cross-family records while preserving explicit legacy compatibility and keeping failed attempts reloadable. Durable finding cc-6f8469a6098b identified the failed-record lockout and must be independently verified fixed through the unchanged Crosscheck mutation-proof contract. The prior shell regression cannot certify execution because Bash lacks a measured non-execution signal, so add the smallest tracked pytest-compatible regression that constructs real tool-failure, unreviewed, and cannot-certify current-contract ledgers, proves each reloads without successful-review evidence, and fails when the clear/blocking state gate is removed. Keep the shell end-to-end retry regression too. Validate and update PR #330 to green CI, obtain a CLEAR exact-head Crosscheck, but do not merge and do not run the post-merge C1 measurement before landing.

What Changed

  • Run regular Fireworks GLM Crosschecks as an independent challenge pass followed by an authoritative full-diff synthesis pass.
  • Record and validate terminal routing and review-depth evidence for successful current-contract reviews while keeping failed attempts reloadable.
  • Add mutation-sensitive pytest and shell regressions for failed-ledger retries, evidence enforcement, telemetry, and exact-head reuse, with corresponding documentation updates.

Risk Assessment

✅ Low: The change preserves failed-ledger reloadability while enforcing successful current-contract evidence and keeps the fixed two-pass regular GLM protocol bounded to the exact full-diff review path.

Testing

Completed 1 recorded test check.

  • Outcome: ⚠️ 1 error across 1 run (2h27m58s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

⚠️ **Test** - 1 error
  • 🚨 tests failed with exit code 1
  • if [ "${FM_AZURE_VALIDATION_CELL:-0}" = 1 ]; then exec "$FM_AZURE_VALIDATION_SHARD_BRIDGE" behavior --count "${FM_AZURE_VALIDATION_SHARD_COUNT:-8}"; else exec bin/fm-no-mistakes-test-command.sh; fi
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Exact-head Crosscheck proof route

Durable finding cc-6f8469a6098b, Failed regular GLM review permanently corrupts the ledger via ungated current_regular_contract validation, is addressed by gating the four completion-only terminal/depth fields to successful clear and blocking records while retaining them as mandatory for successful and reusable current-contract records.

The supported mutation proof is tracked pytest path tests/test_fm_crosscheck_ledger.py with zero runner arguments. Its test_failed_current_regular_reviews_remain_reloadable case constructs real tool-failure, unreviewed, and cannot-certify records and reloads each. A mutation that changes if current_regular_contract and run["state"] in {"clear", "blocking"}: back to if current_regular_contract: must make this pytest path fail. Store that unapplied complete unified Git diff only under .crosscheck/mutations/; leave tracked files unchanged and create no .bak file. Independently execute and verify the ordinary mutation-proof contract before marking cc-6f8469a6098b verified-fixed.

A minimal valid mutation artifact is the following literal unified Git diff; write these bytes directly to .crosscheck/mutations/failed-regular-state-gate.patch without first editing bin/fm-crosscheck.py:

diff --git a/bin/fm-crosscheck.py b/bin/fm-crosscheck.py
--- a/bin/fm-crosscheck.py
+++ b/bin/fm-crosscheck.py
@@ -3464,7 +3464,7 @@ def validate_ledger(value: Any, task_id: str, url: str) -> dict[str, Any]:
             and reviewer.get("review_contract_sha256")
             == review_contract_sha256(False, "pi")
         )
-        if current_regular_contract and run["state"] in {"clear", "blocking"}:
+        if current_regular_contract:
             require(
                 reviewer.get("terminal_provider") is not None
                 and reviewer.get("terminal_model") is not None

Keep the two proof artifacts separate in the final verdict. The verdict-level executed_reproduction.test_path and its receipt must point under .crosscheck/reproductions/ and prove the full exact-base/exact-head diff. Only finding_updates[0].mutation_proof.mutation_patch_path points under .crosscheck/mutations/ to the unified diff above; its test_path is tracked tests/test_fm_crosscheck_ledger.py. Never use the mutation patch as the verdict-level reproduction helper.

For finding_updates[0] on cc-6f8469a6098b, omit the optional reproduction and equivalent_to fields from Pi's generated object so host normalization makes both null. Supply only the verified-fixed status, note, and the pytest mutation_proof. Do not put the .crosscheck/mutations/ patch path into a finding-update reproduction; no separate finding-update reproduction is needed. The top-level verdict executed_reproduction remains the ordinary .crosscheck/reproductions/ helper and receipt.

@ruby-dlee
ruby-dlee force-pushed the fm/azure-c1-accept-y4 branch from fbdcfc9 to 265e93f Compare August 25, 2026 12:34
@ruby-dlee ruby-dlee changed the title fix(crosscheck): add two-pass regular GLM review fix(crosscheck): add two-pass regular review evidence Aug 25, 2026
@ruby-dlee ruby-dlee changed the title fix(crosscheck): add two-pass regular review evidence fix(crosscheck): enforce reliable two-pass GLM reviews Aug 25, 2026
@ruby-dlee
ruby-dlee force-pushed the fm/azure-c1-accept-y4 branch from 7281698 to b4bfaf1 Compare August 25, 2026 17:38
@ruby-dlee
ruby-dlee force-pushed the fm/azure-c1-accept-y4 branch from e50a9a4 to 01707a7 Compare August 25, 2026 18:34
@ruby-dlee
ruby-dlee merged commit 8ed6112 into main Aug 25, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant