ci: sweep every preflight gate instead of stopping at the first failure - #819
Merged
jason931225 merged 1 commit intoAug 19, 2026
Merged
Conversation
`check:ci-preflight` was an 11-link `&&` chain, so the shell stopped at the first failing gate and the remaining ten never ran. A tip with three broken gates needed three full CI cycles to discover three problems -- at this repo's measured ~20 minute wall clock, an hour to learn what one sweep already knew. #813 and #802 each burned a cycle this way today. The workflow LAYER was already fail-slow and stayed that way: 80 steps in ci.yml carry `!cancelled()` and scripts/ci-collect-failures.mjs re-asserts the job red with every failing step id. That sweep stopped at the step boundary; inside one step `&&` still short-circuited. This closes the gap one level down, in the same shape, with the same precedent tools/ci/cargo-test-runner.sh already states: "one red binary never hides the rest". Red is preserved exactly. The sweep's exit is `every(ok)`, so one red gate keeps the whole run red -- proven by mutation below, because fail-slow turning into fail-open would be far worse than the problem it solves. Gates move from an inline shell string into tools/ci/gate-sweep.json so the set is diffable and a gate cannot be dropped inside a chain where no reviewer would see it. A sweep over zero gates is refused rather than passing. REQUIRED COMPANION FIX: tools/ci/check-mjs-dark-suites.mjs discovers reachability by scanning package.json script TEXT. Moving the test commands into a manifest made 20 live suites report as dark, and the natural response to that is to baseline them -- retiring real coverage on the strength of a scanner blind spot. The scanner now reads the sweep manifest as the execution path it is. Verified: npm run check:ci-preflight -> 12 passed, 0 failed, exit 0; node --test tools/ci/gate-sweep.test.mjs -> 8 pass, 0 fail; check-mjs-dark-suites --strict -> dark_count 0, exit 0. Mutation-proven end to end: two synthetic red gates injected at positions 2 and 6 -> ALL 14 gates ran, both failures reported with their output, exit 1; removed -> exit 0. Under the old chain the gate at position 2 ended the run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
jason931225
merged commit Aug 19, 2026
5eeda21
into
fix/lane-fanout-sccache-and-slice
19 of 21 checks passed
jason931225
added a commit
that referenced
this pull request
Aug 19, 2026
…re (#819) Stacked on #818. **Review the top commit only.** `check:ci-preflight` was an **11-link `&&` chain** — the shell stops at the first failing gate and the remaining ten never run. Three broken gates meant **three full CI cycles**, and at this repo's ~20 minute wall clock that's an hour to learn what one sweep already knew. The workflow **layer** was already fail-slow and stays untouched: 80 steps in `ci.yml` carry `!cancelled()`, and `scripts/ci-collect-failures.mjs` re-asserts the job red naming every failing step. That sweep stopped at the *step boundary* — inside one step, `&&` still short-circuited. This closes the gap one level down, in the same shape, with the precedent `tools/ci/cargo-test-runner.sh` already states: *"one red binary never hides the rest"*. The sweep's exit is `every(ok)`, so one red gate keeps the whole run red. Fail-slow becoming fail-**open** would be worse than the problem it solves, so that's mutation-proven rather than asserted. `check-mjs-dark-suites.mjs` discovers reachability by scanning **package.json script text**. Moving the test commands into a manifest made **20 live suites report as dark** — and the natural response to that is to baseline them, retiring real coverage on the strength of a scanner blind spot. The scanner now reads the sweep manifest as the execution path it is. Two synthetic red gates injected at positions 2 and 6: ``` FAIL synthetic-red-A <- the old chain ended the run here ...ten more gates still ran... FAIL synthetic-red-B gate-sweep: 12 passed, 2 failed gate-sweep: FAILED synthetic-red-A, synthetic-red-B exit=1 ``` Removed → exit 0. `npm run check:ci-preflight` → **12 passed, 0 failed**, exit 0 · `gate-sweep.test.mjs` → **8 pass, 0 fail** · `check-mjs-dark-suites --strict` → dark_count 0, exit 0 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #818. Review the top commit only.
The problem
check:ci-preflightwas an 11-link&&chain — the shell stops at the first failing gate and the remaining ten never run. Three broken gates meant three full CI cycles, and at this repo's ~20 minute wall clock that's an hour to learn what one sweep already knew.#813 and #802 each burned a cycle this way today.
What was already right
The workflow layer was already fail-slow and stays untouched: 80 steps in
ci.ymlcarry!cancelled(), andscripts/ci-collect-failures.mjsre-asserts the job red naming every failing step. That sweep stopped at the step boundary — inside one step,&&still short-circuited. This closes the gap one level down, in the same shape, with the precedenttools/ci/cargo-test-runner.shalready states: "one red binary never hides the rest".Red is preserved
The sweep's exit is
every(ok), so one red gate keeps the whole run red. Fail-slow becoming fail-open would be worse than the problem it solves, so that's mutation-proven rather than asserted.A companion fix that was not optional
check-mjs-dark-suites.mjsdiscovers reachability by scanning package.json script text. Moving the test commands into a manifest made 20 live suites report as dark — and the natural response to that is to baseline them, retiring real coverage on the strength of a scanner blind spot. The scanner now reads the sweep manifest as the execution path it is.Mutation proof
Two synthetic red gates injected at positions 2 and 6:
Removed → exit 0.
Verification
npm run check:ci-preflight→ 12 passed, 0 failed, exit 0 ·gate-sweep.test.mjs→ 8 pass, 0 fail ·check-mjs-dark-suites --strict→ dark_count 0, exit 0🤖 Generated with Claude Code