Skip to content

feat(bin): add base-head test suite comparison #2405

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
coreldh wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat(bin): add base-head test suite comparison #2405

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
85d2ee7
feat(test): add bounded base-head partition runner
coreldh Aug 7, 2026
1236825
fix(test): make commit suite partition sound
coreldh Aug 7, 2026
8bd7cce
no-mistakes(review): Preserve running confirmation state on interruption
coreldh Aug 14, 2026
26d0eac
no-mistakes(review): Refuse inconclusive candidate retries
coreldh Aug 15, 2026
ec9af58
no-mistakes(document): Document Suite R2 comparison mode
coreldh Aug 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ bin/fm-test-run.sh --lane portable-serial # portable serial remainder (watcher
bin/fm-test-run.sh --list-lanes # discover exact lane names, including the current CI serial shards
bin/fm-test-run.sh --check-coverage # prove portable shards + serial + serial shards + Herdr equal the full inventory
bin/fm-test-run.sh --all # deliberate complete regression (optional local full walk; not no-mistakes Test)
bin/fm-test-run.sh --compare-commits <base> <head> --output-dir /tmp/fm-test-partition # bounded detached base/head inventories and failure partition
bin/fm-test-isolation-proof.sh --list # proven parallel candidate set (Phase 2 owner)
bin/fm-test-isolation-proof.sh --jobs 4 --json /tmp/fm-isolation-proof.json # re-run concurrent isolation proof only
[ "$(readlink CLAUDE.md)" = "AGENTS.md" ]
Expand All @@ -90,6 +91,7 @@ tmp=$(mktemp -d) && printf 'done: smoke\n' > "$tmp/smoke.status" && FM_STATE_OVE

`bin/fm-test-run.sh` is the single owner of behavior-suite selection, portable CI lane composition, optional local `--jobs` for the proven-isolated set only, per-script timing markers, family totals, the coverage guard, and the optional JSON timing artifact.
Its header and `--help` own the flags, family labels, lanes, and changed-file map; this section only documents the entry points.
Its commit-comparison mode gives independent gates per-script hard bounds, detached local copies, independently reconciled inventories, honest measured-outcome coverage, and retry-confirmed transition buckets for inherited failures, timeouts, regressions, genuine fixes, coverage erosion, and flakes.
`bin/fm-test-isolation-proof.sh` remains the single owner of the Phase 2 concurrent isolation proof and the exact proven candidate set; see `docs/fm-test-isolation-proof.md`.
Portable shard balance evidence lives in `docs/fm-test-portable-shards.md`.
Local no-mistakes Test stays intent-targeted and must not wire `commands.test` to `--all` or a `tests/*.test.sh` walk.
Expand Down
Loading
Loading