Repository navigation
Merge: a PR that removes or weakens a test, lint or guard waits for the owner (#783) [PROTECTED: owner merges] - #820
Conversation
…s for the owner (#783) The owner's rule (decision guide, principles 7 and 9) is that no agent removes or weakens a check, and that an added exception, allowlist entry or skip is a loosening. Until now only the evaluator's test_correct answer stood between such a PR and auto-merge. mergeDecision now gets a `design: may weaken a check:` reason from the change itself (src/merge-policy-weakening.ts), read through the one changed-files helper (changeFiles, plus a new filesAt there for a changed file's text before and after): - tests: with code changed too, an assertion gone or changed (read with its enclosing blocks, what leads into it and the exits before it, so one made conditional, unreachable, unawaited or swallowed counts) or a test's title gone; in a change of tests only, fewer assertions or tests. Always: an existing test that now runs differently (.skip/.todo/.only, its options), an existing file whose assertions call something else (an assert import changed, a shadow, a patch) or that now exits the process, and a deleted test file unless the change adds the same text as a test file in the same folder; - lints and guards: a registry of check tables (CHECK_TABLES, with spreads and same-file names followed): an entry taken out of a rule list, bad fixtures or a guard's list, or added to an allowlist or good fixtures; package.json scripts; a code line removed from a guard, lint, runner or coverage config file (GUARD_FILES), or added there with an early exit or an off switch; a line removed from a lint test outside its tables; a line added to an allowlist file; an inline suppression added anywhere. A test fails on a check-shaped table in test/ or scripts/ that isn't registered (it found test/slot-path.test.ts MAKER_ONLY from #806); - fail closed: a file it can't read or parse, a table it can't find, or a git failure holds. The module's name falls under src/merge-policy*, so changing it is self-modification. Engine source paths stand aside in a managed app's repo, as the locks do; review.ts decides the scope, so this module never names it. Measured on the last 150 merged PRs (to 178c44e): 4 would have waited that didn't otherwise (#797 added an OUTSIDE_THE_HELPER entry; #627 and #620 changed web code and its assertions; #503 added eslint-disable). Test (fails without): test/merge-policy.test.ts "acceptance (#783)" cases run merge-flow's mergeDecisionFor on scratch repos; with merge-flow.ts and merge-policy.ts as on main they get auto: true for a deleted assertion, an added .skip, an added allowlist entry and a removed lint rule, and no weakening reason for a weakened guard. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…783) CI's fix-forward test (test/fix-forward-coordinator.test.ts) merges a PR that renames totalCents to totalInCents in src/price.js and in test/price.test.js; the weakening hold read the test's assertion as changed and held it, so the PR never merged. A routine rename doesn't loosen anything: the assertion's expectation, literals and shape are the same. An assertion now counts as unchanged when it differs only by identifiers, each renamed the same way in the change's code (its old name gone from the code, its new one new there), and never an assertion's own name (equal, ok, assert*, expect*, a keyword, or a .toBe/.isTrue matcher after a dot). Test (fails without): test/merge-policy.test.ts "a rename the code makes too ... is no change" (a rename of add/sub to total/isSum held with "removes or changes a test assertion" before); CI's fix-forward-coordinator tests (held at "merged one after the other" before). Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…n from an older head Review round 1 on #816 (issuecomment-6272706512): - A merge.decided or merge.failed counts only at the PR's latest known head (its latest pr.opened, pr.status, pr.ready or pr.unready). A held decision at head A followed by a push of B no longer shows A's hold; an auto decision at A followed by failing checks on B no longer says "will merge it". The stage waits: "a new head: it waits for its checks and a fresh decision". - The card carries no free text. The block line is fixed words pointing to the Board (never issue.blocked.why); the issue's title is gone; a run's end reason and a verdict's confidence pass only as enum words. - src/hold-words.ts words every kind merge.decided carries today, not only mergeDecision's: GitHub can't merge cleanly, merging failed, GitHub still computing, the combined check (conflict, no branch point, fast tests failing), the docs-only ruleset hold, and conflict fix. An unknown prefix reads "Another reason (see the Board)", never its text. The design sentence now also names a possible weakened check (#820), worded as examples so it holds whether or not #820 lands first. - A new hold prefix fails a test by design: a source scan over every source of merge.decided reasons (merge-policy.ts, merge-flow.ts mergeOrWait, coordinator.ts combinedGate, pr-rules.ts rulesetHold, conflicts.ts conflictWaitReasons) checks each literal reason's prefix has words, and that every reason pushed from a variable is a traced one. Fixture text proves it catches a new prefix and a new untraced source. - prLink fails closed: no repo given, no link. Correction to 11c9f66's message: its "still warns ... as before" test does not pass before; it fails on main, which counts 2 steps (Your first task too) where the test expects 1. Test (fails without): test/first-task-progress.test.ts, "#741 a decision or a failed merge from an older head", "#741 the card shows no evaluator text, path or run detail", and the source-scan test (its fixture lines). Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Evaluator verdict: CHANGES NEEDED (round 1)Reviewed head Acceptance criteria
Attacks (measured: fixtures fed to
|
| Attack | Code | Tests only |
|---|---|---|
deepEqual to ok; exact regex to loose; number to range; console.assert; equal(x, x) |
HELD | pass (the issue's scope, see the owner trade-off) |
Inline timeout raised |
HELD | HELD |
timeout raised in an options const (const OPTS = { timeout }) |
pass | pass |
Test wrapped in if (process.env.FULL); test function made function* (never runs) |
HELD | pass |
t.skip() / t.todo() added in an existing test's body (test already takes t) |
pass | pass |
| try/catch around an assertion | HELD | pass |
| Assertion made a string or a comment; moved to a local helper that doesn't assert | HELD | HELD |
Rename abuse: assert.equal(got, wanted) to assert.equal(got, got), with // wanted to // got (or const wanted to const got) in src |
pass | n/a |
assertStrict renamed to checkLoose in code and test |
HELD | n/a |
Test renamed in the same folder, same text, to .spec.ts / .test.mts / .test.js / .test.tsx |
n/a | pass |
Renamed to .tset.ts or into another folder |
n/a | HELD |
| Unregistered lint test (test/pipefail.test.ts shape): rule regex loosened | pass | pass |
| Registered lint test: regex beside its table widened | n/a | HELD |
LOCK_SCOPE_BAD = [...].slice(1) (registered lint test); GUARDRAIL_PATHS = [...].filter(...) (scripts/lib/changelog.mjs) |
n/a | pass |
The same on NEVER_LOOSEN in src/delegation.ts (a guard file) |
n/a | HELD |
CI: continue-on-error, || true, a changed run: line |
n/a | HELD (both scopes) |
CI: pure-inserted if: github.actor == 'nobody' |
n/a | pass (L3 by ci-config anyway) |
.worklane/config.yaml required_checks entry removed; tests.yaml tier removed |
n/a | pass (both scopes) |
review.yaml wait_categories shrunk |
n/a | HELD |
package.json test script narrowed |
n/a | HELD |
tsconfig strict: false, noImplicitAny: false added |
n/a | HELD |
tsconfig "exclude": [...] added (tsconfig.json is L1 here: measured with computeLevel) |
n/a | pass |
eslint.config.js ignores, .eslintrc ignorePatterns, vitest exclude, jest testPathIgnorePatterns/passWithNoTests, .gitleaks.toml allowlist path added |
n/a | pass |
Inline eslint-disable, @ts-ignore |
n/a | HELD |
node --test semantics behind the t.skip() row (measured, scratch file): a test that calls t.skip() or t.todo() and then fails an assertion is reported # SKIP/# TODO. A function* test body never runs. The run reported # fail 0 and exited 0.
Fail closed (measured unless marked)
- A reader that throws holds. A test file that doesn't parse after the change holds. A bad revision (git failure) holds with "could not run". Binary bytes in a
.test.tsafter the change hold as unreadable. A blob over 256 MB makes git'smaxBufferthrow, which holds (inferred). - A huge file never passes, but it stalls the coordinator.
testFactsis quadratic in assertions per test: 1.03 MB took 2.9 s, and 4.26 MB took 70 s per side. It runs synchronously inmergeDecisionForon every poll (RC8).
Self-protection (measured: computeLevel, this repo's review.yaml and a bare review.yaml with no L3 categories)
src/merge-policy-weakening.ts, test/merge-policy.test.ts, src/merge-policy.ts, merge-flow.ts, change-files.ts, review.ts, guardrails/glob.ts, path-text.ts and brand.ts are all L3 by self-modification (locked in code), even with the bare review.yaml. The module's own lists are registered never-shrink, and NOT_A_CHECK is never-grow.
False positives (measured independently: changeFiles + weakeningOf over the last 150 merged PRs on origin/main 128453a, read-only git)
- 87 of 150 get a weakening reason, the same as the builder's figure.
- Newly held (the PR waited on nothing else): Test: list scripts/bench/hidden.ts on the changed-files reader list (main red after #688 + #757) #797, web: Needs you is one queue at /needs-you, blocking first then oldest; its badge counts Waiting on you (#585 part 2) #627, web: the sidebar's pages become tabs in six groups; every old route redirects (#585 part 1) #620 and No late answer moves a control on Settings, Inbox or Decisions (#455) #503, the builder's four. test: temporary directories live inside the test's own machine, so a run leaves nothing in /tmp (#604) #652 also shows up in my run because I used the review.yaml at each PR's merge time; the builder used today's, where test/isolate.ts is engine-guards. Consistent.
- Tests-only PRs: held are Test: list scripts/bench/hidden.ts on the changed-files reader list (main red after #688 + #757) #797 (true positive), merge-policy test: slot-dir-writes.test.ts must exist like the other engine-guards entries [PROTECTED: owner merges] #692 and test: temporary directories live inside the test's own machine, so a run leaves nothing in /tmp (#604) #652 (each waits anyway, per the PR body). The only tests-only PR that only added tests, slot dir: a test fails if engine code writes a slot-directory file in place (#616) #654, is not held.
- Rename rule:
assertStrict/equal/okcan't be "renamed" (held). But the rule accepts any identifier the src diff drops and adds, comment words included (RC2). - Docs (measured): a
docs/*.mdor README line that mentionseslint-disableornoqanext to;or#holds as an added suppression (RC9). - Runtime: at most 2.4 s per PR. The 150 PRs took 113 s in total.
No regression (measured)
- On the last 60 merged PRs, main's
mergeDecision(fail-without worktree) and the head's gave the same answer for all 22 with no weakening reason. For the 38 with one, the head isauto: false, with a superset of main's reasons. git merge-tree --write-tree: clean against origin/main (128453a). Against every open PR, the conflict set with this branch equals that PR's set with main. Approvals: one role rule on every surface, decide only as root or the coordinator user, money paths read from main (#715) [PROTECTED: owner merges] #805, Required checks count only from the App or workflow pinned for their name [PROTECTED: owner merges] #788, Engine updater counts a required check only from its pinned workflow or App [PROTECTED: owner merges] #789, Project commands run as the agent user only in a sandbox that hides its credentials; their output is redacted (#713) [PROTECTED: owner merges] #787, docs/design: amend #698 decision 1: turning auto_merge on is root-only; off stays on every surface [PROTECTED: owner merges] #775, Dashboard policy changes go through root's helper (set-policy), never a loosening (#726 part 2) [PROTECTED: owner merges] #770, Single machine: policy.yaml is root's, and the engine refuses one that isn't (#726 part 1) [PROTECTED: owner merges] #769, First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816, Tests: stateDir() refuses the real state dir in a test run; a lint that tests guard machine paths with onRealMachine (#815) [PROTECTED: owner merges] #818 and Wizard: the Claude step matches the login to agent_runtime and offers an API key (#738) [PROTECTED: owner merges] #819 are clean. doctor prints a paste-ready required_check_sources block; a new project's config starts pinned (after #788) [PROTECTED: owner merges] #803, Pin this repo's required checks to the workflow files that post them (after #788 and the engine upgrade) [PROTECTED: owner merges] #794 and Test runs: each run in a user namespace of its own; slirp4netns as the test user; a run can't join Docker's namespaces (#545) [PROTECTED: owner merges] #780 conflict in CHANGELOG.md only, and Remove the desktop app, its source, Tauri lockfile and docs; keep the desktop check one more release as a no-op (#744) [PROTECTED: owner merges] #746 in its four files, all already with main.- CHANGELOG (measured in memory): the bullet and its 4 sub-bullets survive
withSection(changelogSection('0.5.3', ...)).
Fail-without (measured)
- Scratch worktree at a3fc766 with
src/coordinator/merge-flow.tsandsrc/merge-policy.tsfrom origin/main, thennpm run build(2 TS2353 type errors onweakening, but it emits) andnode --import ./dist/test/isolate.js --test dist/test/merge-policy.test.js:# tests 41 # pass 36 # fail 5. - The five that fail are "the merge decision waits on it whatever the level" and acceptance "deletes an assertion", "adds .skip", "adds a lint allowlist entry" and "weakens a guard file", each with
+ auto: true, reasons: []. - At head:
# tests 41 # pass 41. The builder reported 35/5; 41 is the count at a3fc766 (inferred: the rename test came in that commit). - At head: merge-policy, change-files-renames, review and slot-path tests:
# pass 153 # fail 0.npm run typecheckis clean.check:denylist: "1306 files clean".
Protection, hygiene, CI
- Protected and marked correctly: the
[PROTECTED: owner merges]title,**Risk class:** protected, theneeds-ownerlabel, and real "What this can never do" and "Compromised component" sections.- "Make anything merge that waited before" is true: it only appends reasons (measured, above).
- "Write anything" is true:
filesAtruns onlyls-treeandcat-file(inferred from the code).
- Both commits are signed off. No attribution trailers or lines. No new dependencies, no
.claude/. No hardcoded product name in the new module (measured with grep). "Closes Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783" is present. - The Adversarial pass section is present.
- Scope is fine: 7 files, +1330/-0.
- CI: the only failing test in run 39287373023 is
test/package-files.test.ts"the package stays within its size budget" (tarball 1,908,377 > 1,900,000; unpacked 5,965,009 > 5,930,000): measured from the node-22 shard 1 log. gh returned no log lines for the node-24 shard 1 job. The aggregatetest (ubuntu-latest, 24)job only reports a shard failure; inferred: the same test. Every other check is green. Pending the owner, not a required change.
Required changes
t.skip()/t.todo()added inside an existing test's body must hold, in every change. Principle 9 calls it an added skip, and Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783 lists ".skip/.todo added". The PR body says skip and todo "always hold". Today a new loose skip mark is ignored (missing(before.looseSkips, after.looseSkips)only holds marks that went away). Add a fixture where the test already takest.- Rename rule abuse.
renamedOnlyacceptsassert.equal(got, wanted)toassert.equal(got, got)when the src diff dropswantedand addsgot, even in a comment.- Fix: compute
gone/camefrom code tokens only (not comments). - Fix: refuse a pair whose new name already appears in the old assertion, or whose old name still appears in the test file after the change.
- Add both fixtures.
- Fix: compute
- Renaming a test file out of the runner. A same-folder, same-text rename to
.spec.ts,.test.mts,.test.jsor.test.tsxdoesn't hold. This repo runs onlydist/test/**/*.test.jsbuilt fromtest/**/*.ts(inferred from package.json and tsconfig.json). Require the same test suffix and extension (only the stem may change), or hold. - Lint tests without a registered table. test/pipefail.test.ts's
/shell:\s*true|\bexecSync\(/, loosened, passes with or without a code change. Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783's lint item covers "a rule ... removed from a lint's ... matcher" in any PR. Either:- register lint test files (a
LINT_TESTSlist, any removed code line holds, as for registered-table files) with a registry-test heuristic that fails on an unlisted file that scans the repo's sources. Measured cost of the broadest heuristic (repoRootplusreadFileSync/readdirSync, 163 files): 3 more of 150 PRs held (escape fixture: refuses to run outside the MCP broker's sandbox, and never as root (#608) #653, Fix: siri-ask cost-leak check uses a value that can't occur by chance (#546) #547, no-late-shift: send every pending answer before turning Fetch interception off #517). A curated list costs less; or - if you judge that too big for this PR, name it as a known gap in the PR body and the CHANGELOG, and open the follow-up issue.
- register lint test files (a
- A registered table shrunk by a call on its literal (
[...].slice(1),.filter(...)) passes outside guard files (measured on a lint test and on scripts/lib/changelog.mjsGUARDRAIL_PATHS). Fix:tableSpanshould take the whole statement when anything butas const/satisfiesfollows the literal, so it reads as one changed entry. <configDir>/config.yaml(required_checks) and<configDir>/tests.yaml(test-run tiers). An entry removed from either passes in both scopes. Add them toGUARD_FILES(they're already L3 harness-config here, so it costs nothing in this repo; inferred).- Pure insertions in config guard files. Bypass words miss
exclude,ignores,ignorePatterns,testPathIgnorePatterns,passWithNoTests, a gitleaks allowlist path and a CIif:. By principle 1, prefer that any added line holds in non-source guard files (configs, workflows, hooks), keeping the bypass-word rule for JS/TS guard sources. Otherwise, at least match the stems case-insensitively (ignor,exclud,skip,allow,passWithNoTests,if:). Add a fixture for each. - Size cap. A JS/TS test or table file over a fixed size (for example 1 MB) should hold as unreadable before
testFactsruns. It stays fail closed and keepsmergeDecisionForfrom blocking for minutes on every poll (70 s per side at 4.26 MB, measured). Add a test. - Docs false positive. Leave docs paths (
isDoc) out of the suppression rule. A docs or README line that mentionseslint-disable/noqanow holds a docs-only PR (measured). - The test wrapped in a condition, or made a generator, in a tests-only change (
if (process.env.X) test(...),function*): hold it as "changes how an existing test runs". For example, put the test call's enclosing non-test headers and its function's kind intoruns. Or name it as a known gap in the PR body, with the raisedtimeoutin an options const beside it. - Correct the PR body's claims for items 1, 3 and 10 (principle 5), and update the fail-without numbers to 36/5 of 41.
For the owner (a genuine trade-off, not decided here)
- A loosened assertion in a tests-only PR doesn't wait, as Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783 scoped it. That leaves a two-PR route: PR 1 rewords or swaps assertions with test and assertion counts kept, touches only tests, and is L0. PR 2 then changes the code without touching the test.
- Measured above: every reword attack passes as tests-only, and so does swapping a real test for a trivial one with the counts kept.
- Holding any changed assertion in tests-only PRs would have held 4 of 17 tests-only PRs: Test: budget health reads the limited credential's reset time once #790, add-project-view test: the stand-in helper holds until released, so a slow runner can't see the run already failed #777, main red: the #566 switch test stamps its pause on the test's clock, not the wall clock #684 and dashboard-switch-speed: a false Board check says which part failed and what the page had #656 (the builder's measurement).
- The owner should decide whether that route is acceptable. Principle 7 says existing tests are never weakened without the owner's OK.
- Already named by the builder, and left with the owner:
land_mode: directskipsmergeDecision, so this hold doesn't apply there. A removed call to a helper imported from another file isn't counted.
The round-1 review of #820 found cheap weakenings the hold missed; each is closed by its class: - a test's run options now include t.skip()/t.todo()/this.skip() in its body, a generator body, a condition, loop, try or `&&` around it (with the literal value of a case table it loops over), and an options const it names, read as its value; any change to them holds, tests only too; - an assertion carries the literal value of each const it names (scoped: the test's own, else a top-level one); - renames come from code words only (never comments or strings), never to a name the old assertion had, and never with the old name still used in the test; - a test file moved within its folder must keep its runner suffix (`.test.ts` stays `.test.ts`); - LINT_TESTS registers lint tests by file (a removed line, or an added exit or off switch, holds); any other test that walks the sources holds on a regex line removed or changed; - a registered table narrowed, mutated or declared again where it is used holds, and a literal that goes on into a call (`[...].slice(1)`) is read as the whole statement; - config.yaml and tests.yaml are guard files; any line added to a guard that isn't engine source (configs, workflows, hooks) holds; husky, pre-commit, lefthook, pylint, mypy, stylelint and markdownlint configs join; - a file over SIZE_CAP (500,000 characters) holds unread, and the parse is linear (measured 0.2 s at the cap, 70 s per side at 4.26 MB before); - docs are no longer read for inline suppressions; more markers (pylint, rubocop, shellcheck, `#[allow(`); - whether a tests-only reworded assertion holds stays the owner's call, kept to one line (changedHolds). Measured on the last 150 merged PRs (to 128453a): 5 newly held (#797, #627, #620, #503 and #547, a changed regex in a lint-shaped test); one add-only test change gets a test reason (#499, waits anyway). Test (fails without): test/merge-policy.test.ts "weakening (round 2, ...)" and "(adversarial pass, round 2)" cases fail with src/merge-policy-weakening.ts as at a3fc766. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Evaluator verdict: CHANGES NEEDED (round 2)Reviewed head
The package size budget is pending the owner and is not a required change here. Round 1's gaps (measured: the round-1 fixtures, rerun at a8d68ed)
Round-2 attacks (measured: fixtures fed to
|
| Family | Attack | Code | Tests only / app |
|---|---|---|---|
| Assertions | Actual value overwritten before the assertion (got = 3; inserted) |
pass | pass |
| Assertions | Actual value recomputed (let got = (add(1, 2), 3)) |
pass | pass |
| Assertions | An imported test helper that throws (not assert*) emptied (test/helpers.ts, L0) |
pass | pass |
| Assertions | Expected value moved into an imported JS fixture (test/fixtures/expected.ts) and made tautological |
pass | pass |
| Assertions | The unit under test mocked to return the expected value | HELD | pass (the open owner decision) |
| Skips | { skip: SKIP_SLOW } with const SKIP_SLOW = false flipped to true, or made !process.env.X |
pass | pass |
| Skips | import { skip as test } from 'node:test' (node:test: every test # SKIP, exit 0) |
pass | pass |
| Skips | from 'node:test' swapped to a local no-op module |
pass | pass |
| Skips | beforeEach((t) => t.skip()) added (node:test: the failing test is # SKIP, exit 0) |
HELD | HELD |
| Runner | An existing test wrapped in a function or arrow that nothing calls (node:test never registers it) | HELD | pass |
| Runner | vite.config.ts test.exclude added (vitest reads it); cypress.config.ts specPattern narrowed |
pass, L1 | app pass, L1 |
| Lint tables | Registered lint test: the scanned set narrowed by a pure insertion with no bypass word (files.splice(...)), or the table copied into a new Set that grows |
pass | pass |
| Lint tables | Registered never-grow table: an allowance count raised | engine HELD | app pass (stands aside, as designed) |
| Lint regexes | test/brand.test.ts, the product-name lint CLAUDE.md names, made to skip a path (&& !p.includes('legacy')) |
n/a | pass, L0 |
| Lint regexes | test/testrun-gpu.test.ts, the nvidia-smi lint over git ls-files src: regex narrowed, or src dropped from the listing |
n/a | pass, L0 |
| Lint regexes | A lint-shaped test whose rule is a string (.includes('shell: true')), changed |
pass | pass |
| Lint regexes | A lint walking new URL('../src/', import.meta.url) (no repoRoot), regex loosened |
pass | pass |
| Guard files | .github/actions/test/action.yml: run: npm test to run: true |
engine pass (L3) | app pass, L1 |
| Guard files | .worklane/checks/lint.mjs (the project lints of the PR-gate tier): a rule removed |
pass (L3) | pass (L3) |
| Guard files | .lintstagedrc: the eslint step dropped; .github/CODEOWNERS: a reviewer line removed |
pass | app pass, L1 |
| Guard files | scripts/ci/affected.mjs and scripts/ci/shard.mjs choose which test files a PR's CI runs (ci.yml:89-91), but they aren't in GUARD_FILES (inferred from the list, no fixture run) |
L3 here (measured) | n/a |
| Configs | .semgrep.yml rule removed; .rubocop.yml cop off; deno.json lint exclude |
pass, L1 | app pass, L1 |
| Suppressions | A multi-line /* / eslint-disable / */ comment; /* eslint no-eval: "off" */ |
n/a | pass |
| Suppressions | #![allow(clippy::all)], @SuppressWarnings, // nosemgrep, // NOSONAR, stylelint-disable, tslint:disable, oxlint-disable, # pyright: ignore, # yamllint disable, # hadolint ignore= |
n/a | pass |
The node:test semantics behind the skip and runner rows were measured with Node v22.23.2, on a scratch file in the scratchpad.
Speed (measured: testFacts on one side, files of 499,800 to 500,000 characters, scratch processes)
| Shape at the cap | Time | Heap |
|---|---|---|
| The builder's shapes (many tests, 10k assertions, callbacks) | 0.1–0.3 s, as reported | small |
~19,000 nested describes |
106.6 s | 47 MB |
| 100 nested calls, each with a ~400-character first argument, then many assertions | 47.5 s | 1.7 GB |
| The same with 600 nested calls | FATAL JavaScript heap out of memory after 223 s (default heap; it also crashed with --max-old-space-size=4096) |
over 4 GB |
~11,000 tests naming one 200,000-character options const (test('t', OPTS, ...)) |
over 120 s (stopped) | n/a |
2,000 nested ifs around ~22,000 tests |
14.0 s | 393 MB |
~33,000 const a = (1); then one { at the end |
3.0 s | small |
- A PR touching 50 test files: 50 plain files near the cap took 14.8 s for
weakeningReasons(measured). 50 files of only ~50,000 characters each (10% of the cap), with nesteddescribes, took 103 s (measured). - Is the synchronous path safe? No.
weakeningOfruns insidemergeDecisionFor, which the PR batcher'swaitsForOwnerand the merge pass call (merge-flow.ts:164,:574).- A minutes-long parse blocks the coordinator's event loop. An out-of-memory error can't be caught: the coordinator dies, and on restart it reads the same head again (inferred).
filesAtalso reads every blob in full (up to git's 256 MB buffer) before the size check (inferred from the code).
- The worst shapes come from:
closeOfrun per test call (quadratic in nesting);whereandaroundrebuilt per assertion and per test from the whole bracket stack, with each first argument up to 400 characters;tableSpanresolving an options const again on every call.
False positives (measured independently: changeFiles plus weakeningOf over the 150 merged PRs to 128453a, the PR's base; read-only git)
- 91 of 150 get a reason. Newly held: Test: list scripts/bench/hidden.ts on the changed-files reader list (main red after #688 + #757) #797, web: Needs you is one queue at /needs-you, blocking first then oldest; its badge counts Waiting on you (#585 part 2) #627, web: the sidebar's pages become tabs in six groups; every old route redirects (#585 part 1) #620, Fix: siri-ask cost-leak check uses a value that can't occur by chance (#546) #547 and No late answer moves a control on Settings, Inbox or Decisions (#455) #503, the same as the builder. With each PR's merge-time review.yaml, test: temporary directories live inside the test's own machine, so a run leaves nothing in /tmp (#604) #652 joins, as in round 1.
- Tests-only PRs: 16, of which 4 are held: Test: list scripts/bench/hidden.ts on the changed-files reader list (main red after #688 + #757) #797, merge-policy test: slot-dir-writes.test.ts must exist like the other engine-guards entries [PROTECTED: owner merges] #692, test: temporary directories live inside the test's own machine, so a run leaves nothing in /tmp (#604) #652 and Fix: siri-ask cost-leak check uses a value that can't occur by chance (#546) #547. The only tests-only PR that only added tests, slot dir: a test fails if engine code writes a slot-directory file in place (#616) #654, is not held. Test: list scripts/bench/hidden.ts on the changed-files reader list (main red after #688 + #757) #797 added an allowlist entry, not a test.
- Is Fix: siri-ask cost-leak check uses a value that can't occur by chance (#546) #547 a fair hold? As fail closed, yes (inferred).
- It is a tests-only change of the exact value a privacy assertion checks (
/PRIVATE|3\.21|opus/to/PRIVATE|987654|opus/, and the seeded value too). Loosening that matcher would let a leak through. - But it is held for the wrong reason: "removes or changes code in a guard or lint". test/siri-ask.test.ts is no lint. It reads as
lintShapedbecause it callsreaddirSyncon a temp dir, importsrepoRoot, and imports from'../src/. - 80 of 421 test files read as lint-shaped (measured). Not a required change, but the reason would read better as "changes a regex in a test that reads the repo's files".
- It is a tests-only change of the exact value a privacy assertion checks (
- But three ways of only adding tests now wait (measured, synthetic fixtures; none occurs in the 150, where test-skip fired in only 2 PRs and never alone):
- a new
it()with a platformt.skip()inside an existingdescribe(that describe's body-skip count includes its new child); - a case added to a loop's case table (
CASESgains a row); - a new test appended to a
LINT_TESTSfile, on a line with a bypass word (assert.equal(x, false)).
- a new
No regression (measured)
mergeDecisionat the PR's base (128453a's merge-policy.ts and merge-flow.ts, built in a second worktree) against the head, over the last 60 merged PRs:- all 21 with no weakening reason are identical;
- the 39 with one are
auto: false, with a superset of the base's reasons; - there are no mismatches.
- Self-protection: the new module, test/merge-policy.test.ts, merge-policy.ts, merge-flow.ts, change-files.ts, review.ts, glob.ts, path-text.ts and brand.ts are L3 by
self-modification, even with a bare review.yaml (measured withcomputeLevel).LINT_TESTSandSIZE_CAPare registered never-shrink.
Fail-without (measured)
- The round-2 tests on a3fc766's module. I used a scratch worktree at a8d68ed with
src/merge-policy-weakening.tsfrom a3fc766, then rannpm run build(it emits, with 5 TS errors for the new exports) andnode --import ./dist/test/isolate.js --test dist/test/merge-policy.test.js.- Result:
# tests 50 # pass 41 # fail 9, the builder's 41/9. - The 9 are the round-2 tests (Run project commands under bash with pipefail #1/Durable slot locations; repo-scoped GitHub credentials only #10 through Sandbox agents per lane with Claude Code's native sandbox #9, and the round-2 adversarial one). Each fails with an
AssertionError, not an import error.
- Result:
- The base's wiring. With the head's module and the base's
merge-policy.tsandmerge-flow.ts:# tests 50 # pass 45 # fail 5. The five are the same five as round 1, each withauto: true.- With today's main (d4b4503) wiring, a sixth fails on
this.moneyPaths is not a function. That is main moving, not this PR.
- With today's main (d4b4503) wiring, a sixth fails on
- At head: merge-policy, change-files-renames, review and slot-path tests give
# pass 162 # fail 0.npm run typecheckis clean.check:denylistreports "1306 files clean". All runs used a scratch HOME, TMPDIR and state dirs.
Protection, hygiene, CI, overlaps
- Protected and marked correctly: the
[PROTECTED: owner merges]title,**Risk class:** protected, theneeds-ownerlabel, and real "What this can never do" and "Compromised component" sections.- "Make anything merge that waited before" is true (measured above).
- "Write anything" is true (inferred:
ls-treeandcat-fileonly). - "Compromised component" should add one line: a worker can now stall or crash the coordinator with one crafted test file (Speed).
- Hygiene (measured):
- all 3 commits are signed off;
- there are no attribution trailers or lines in the commits or the body;
- no new dependencies, and no
.claude/; - no product name in the new code;
- "Closes Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783" is present;
- the Adversarial pass section is present, with a round-2 table.
- The new tests touch no real state (measured with grep: no
homedir,HOMEor state paths).
- CI (run 39297757296): the only failure in both shard-1 jobs (node 22 and node 24: 895 pass, 1 fail) is
test/package-files.test.ts"the package stays within its size budget". The tarball is 1,912,956 bytes against 1,900,000, and the unpacked size 5,977,655 against 5,930,000. Every other check is green. Pending the owner, not a required change. - Overlaps. origin/main has moved to d4b4503 (Approvals: one role rule on every surface, decide only as root or the coordinator user, money paths read from main (#715) [PROTECTED: owner merges] #805 and Project commands run as the agent user only in a sandbox that hides its credentials; their output is redacted (#713) [PROTECTED: owner merges] #787). I ran
git merge-tree --write-treeagainst it and against every open PR:- main is clean, and so are Wizard: the Claude step matches the login to agent_runtime and offers an API key (#738) [PROTECTED: owner merges] #819, Tests: stateDir() refuses the real state dir in a test run; a lint that tests guard machine paths with onRealMachine (#815) [PROTECTED: owner merges] #818, First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816, doctor prints a paste-ready required_check_sources block; a new project's config starts pinned (after #788) [PROTECTED: owner merges] #803, Pin this repo's required checks to the workflow files that post them (after #788 and the engine upgrade) [PROTECTED: owner merges] #794, Engine updater counts a required check only from its pinned workflow or App [PROTECTED: owner merges] #789, Required checks count only from the App or workflow pinned for their name [PROTECTED: owner merges] #788, docs/design: amend #698 decision 1: turning auto_merge on is root-only; off stays on every surface [PROTECTED: owner merges] #775, Dashboard policy changes go through root's helper (set-policy), never a loosening (#726 part 2) [PROTECTED: owner merges] #770 and Single machine: policy.yaml is root's, and the engine refuses one that isn't (#726 part 1) [PROTECTED: owner merges] #769;
- Test runs: each run in a user namespace of its own; slirp4netns as the test user; a run can't join Docker's namespaces (#545) [PROTECTED: owner merges] #780 conflicts in CHANGELOG.md only (with main it already conflicts in CHANGELOG.md and src/project-env.ts);
- Remove the desktop app, its source, Tauri lockfile and docs; keep the desktop check one more release as a no-op (#744) [PROTECTED: owner merges] #746 conflicts in the same four files it already conflicts in with main.
- First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816 (inferred from its test, not run on a merged tree, per rule 12). Its hold-word scan (test/first-task-progress.test.ts
TRACED) has no entry for this PR'sreasons.push(...(i.weakening ?? []))inmergeDecision. Whichever of First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816 and Merge: a PR that removes or weakens a test, lint or guard waits for the owner (#783) [PROTECTED: owner merges] #820 lands second must add one traced entry for it, with a region forweakeningReasons(andweakeningOf's catch). - CHANGELOG (measured in memory): the bullet and its 4 sub-bullets survive
withSection(changelogSection('0.5.3', ...)).
Required changes
-
Bound the parse, so no file under the cap can stall or crash the coordinator (principle 1; measured above: 106 s, over 120 s, and a fatal out-of-memory error, all at or under 500,000 characters).
- Match brackets once, in one stack pass, instead of
closeOfper test call. - Bound each assertion's
whereand each test'saround(for example, the innermost brackets up to a fixed number of characters). Or hold the file as unreadable when the nesting depth or the total key text passes a fixed limit. - Resolve an options const once per name.
- Add a per-PR budget: total characters parsed across files, the rest unread. Skip
cat-filefor a blob overSIZE_CAP(ls-tree -lgives its size). - Add tests with these worst-case shapes at the cap. Prefer an operation-count bound to a wall-clock one, so they aren't flaky.
- Match brackets once, in one stack pass, instead of
-
Skips and runner selection that still pass in a tests-only change, at L0:
- (a) An identifier inside an options object (
{ skip: SKIP_SLOW }): read as its value, like a whole-argument const. - (b) What the test calls bind to: an import of
test/it/describe/suite(any module, includingskip as testfromnode:test, or a swap to a local module), and a declaration that shadows one. Treat them astest-binding, as forassert. - (c) An existing test wrapped in a function or arrow nothing calls: put every enclosing header, not only
CONTROLones, intoruns. - Add a fixture for each.
- (a) An identifier inside an options object (
-
Lint tests this repo has that aren't covered.
- test/brand.test.ts (the product-name rule in CLAUDE.md) and test/testrun-gpu.test.ts (
nvidia-smiovergit ls-files src) pass as L0 tests-only weakenings. Add both toLINT_TESTS. - Make it structural (principle 4): a registry test fails on any test file that lists
src/orscripts/files (ls-files, orreaddir/glob/walkinto them) and is neither inLINT_TESTSnor in a named not-a-lint list. - Widen
lintShapedtols-filesand to roots not namedrepoRoot.
- test/brand.test.ts (the product-name rule in CLAUDE.md) and test/testrun-gpu.test.ts (
-
Suppressions:
- In JS/TS, count a mark that isn't inside a string, so a multi-line block comment counts.
- Add ESLint inline config (
/* eslint <rule>: "off" */or0). - Add
#![allow(,@SuppressWarnings,nosemgrep,NOSONAR,stylelint-disable,tslint:disable,oxlint-disable,pyright: ignore,yamllint disableandhadolint ignore. - Add fixtures.
-
Guard files to add (a list change each):
- In every scope:
.github/actions/**(L1 in a managed app),**/vite.config.*,**/cypress.config.*,.lintstagedrc*andlint-staged.config.*,**/.semgrep.y*mland.semgrep/**,**/.rubocop.yml,**/deno.json*,.github/CODEOWNERS, and<configDir>/checks/**(the project lints the PR-gate tier runs). - In the engine:
scripts/ci/**(affected.mjs and shard.mjs choose which tests a PR's CI runs).
- In every scope:
-
A tests-only PR that only adds tests must not wait. Each of these is held now (measured):
- (a) a new
it()witht.skip()in an existingdescribe: count body skips outside nested test calls; - (b) a row added to a loop's case table: read it as never-shrink (hold a row removed or changed, not one added);
- (c) a new test appended to a
LINT_TESTSfile, on a line with a bypass word: ignore added lines inside a test call new in the change, or name it as a known over-hold with its measured cost. - Add fixtures.
- (a) a new
-
Name these as known gaps in the PR body, the docs/design.md paragraph and the CHANGELOG (principle 5), or close them:
- an actual value overwritten or recomputed before an existing assertion (
got = 3;inserted;let got = (add(1, 2), 3)); - an imported test helper that throws (not
assert*) emptied; - an expected value moved into an imported JS fixture. A cheap close: treat a JS file under test paths with no tests like an expected-output fixture when code changes. Measure the cost first;
- a lint-shaped test's string rule (
includes) changed; - a registered lint test narrowed by a pure insertion with no bypass word (
files.splice(...), a copied Set grown); - a guard source's pure insertion with no bypass word (L3 in the engine).
- an actual value overwritten or recomputed before an existing assertion (
-
Correct the speed claims (principle 5):
- the body's "
testFactsis now linear" and "0.20–0.23 s at the cap"; - docs/design.md's "a parse at the cap measured about 0.2 s";
SIZE_CAP's comment, "well under a second".
Each is false for nested shapes (measured above). Add round 2's attacks to the Adversarial pass table, and the stall or crash to "Compromised component".
- the body's "
For the owner (not decided here)
- Still pending, as before: a reworded or swapped assertion in a tests-only PR (isolated in
changedHolds). The mocked-unit row above is one more case of it. - Pending: the package size budget. main now includes Project commands run as the agent user only in a sandbox that hides its credentials; their output is redacted (#713) [PROTECTED: owner merges] #787, which trimmed comments for the same budget.
- Merge order (not a decision): First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816 and Merge: a PR that removes or weakens a test, lint or guard waits for the owner (#783) [PROTECTED: owner merges] #820 each pass alone. The second to land needs the one
TRACEDentry described above.
…tion changes wait (#783) The round-2 review of #820 measured the parse stalling the coordinator's synchronous merge path: 106 s on nested describes at the cap, a fatal out-of-memory after 223 s on nested calls, 15 s for 50 near-cap files. Now: - brackets are matched once per text (kept on the scanned text, never keyed by the string), a depth-zero scan jumps bracketed spans, and an assertion's or test's context is built once per open bracket; - a file holds unread past MAX_DEPTH (100) of nesting, MAX_KEY (8,000) characters of context, or WORK_PER_CHAR (40) parse steps per character; a change parses at most PARSE_BUDGET (2,000,000) characters, the rest unread; filesAt reads no blob over SIZE_CAP (`ls-tree -l`); - measured at the cap: every shape takes at most 12 steps per character, each worst shape holds in under a second, and 20 near-cap files take about a second. Owner's decision (2026-10-19): a tests-only change that changes an existing assertion's expected value or matcher waits; an added assertion and a pure rename (a test's title, or a test's own local) don't. Through changedHolds. The review's other gaps, each closed by its class: - skips: a const named inside a test's options is read as its value; an import or shadow of test/it/describe is a binding (`import { skip as test }`, a swap to a local no-op); every header around a test is how it runs (a function nothing calls); - lint tests: brand.test.ts, testrun-gpu.test.ts and seven more are in LINT_TESTS; a registry test fails on a test that lists src/ or scripts/ files and is in neither LINT_TESTS nor NOT_A_LINT (with why); - suppressions: a marker counts anywhere but in a string (a bare line of a block comment), ESLint inline config turning a rule off, and ten more markers; - guard files: .github/actions, CODEOWNERS, vite, cypress, lint-staged, semgrep, rubocop and deno configs, the project lints (<configDir>/checks) and, in the engine, scripts/ci; - only adding never waits: a skip counts for the innermost test only, a case table is compared by rows (one added runs more), and a lint test's new tests and new helpers aren't read as bypasses. Measured on the last 150 merged PRs (to 128453a): 9 newly held, the 5 before plus #790, #777, #684 and #656 (tests-only changes of existing assertions, the owner's decision). Test (fails without): test/merge-policy.test.ts "weakening (round 3, ...)" and the owner's-decision test fail with src/merge-policy-weakening.ts as at a8d68ed. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com> # Conflicts: # test/package-files.test.ts
Evaluator verdict: CHANGES NEEDED (round 3)I reviewed head Most round-2 items are closed. The skip, import and header bindings,
1. Parse safety (measured: scratch processes, files at or under 500,000 characters)The round-2 shapes are fixed.
Shapes at the edge of the limits behave.
The work bound is not enforced in every loop.
2. Owner's decision 1 (measured: fixtures fed to
|
| Change | Result |
|---|---|
Expected 3 to 4 |
HELD |
deepEqual to ok |
HELD |
equal to notEqual |
HELD |
| An assertion removed | HELD |
| One removed and another added (count kept) | HELD |
A rename to an imported name (add(1, 2) to sub(1, 2)) |
HELD |
A local renamed with its literal changed (want = 3 to wanted = 4) |
HELD |
| An added assertion | pass |
A local renamed (got to actual) |
pass |
| A test title renamed, tests only | pass |
A title renamed with code changed still holds, as the module's comment says.
These still break decision 1:
| Change | Tests only | With code |
|---|---|---|
Python assert add(1, 2) == 3 to == 4 (tests/test_math.py) |
pass (L0, no other wait) | HELD |
Go != 3 to != 4 (*_test.go) |
pass | HELD |
Snapshot src/__snapshots__/view.test.ts.snap value changed |
pass | HELD |
test/fixtures/expected.json value changed |
pass | HELD |
A literal const changed past its first 400 characters (const EXPECTED = { ...40 fields..., total: 3 } to total: 4), top-level or test-local |
pass | pass |
const eq = assert.deepStrictEqual; then eq(x, 3) to eq(x, 4), or the eq(...) call removed, or = assert.ok |
pass | pass |
- The first four come from the non-JS branch, which still skips unless code changes (
if (!codeChanged) continue;). - The const row comes from
valueOf, which truncates toHEAD(400). The alias row comes from a method alias, which is no assertion name.
Add-only over-holds (owner: "purely added assertions don't wait"):
- A new
it()with an earlyreturnbefore an existingit()in the samedescribe: HELD as "changes a test assertion". - A new
it()holdingassert.throws(() => { throw ... })before an existing one: HELD. - A new assertion with a
throwbefore an existing one in the same test: HELD. - The cause:
exitscounts every return/throw since the outermost bracket, including ones inside closed sibling functions and test calls. - Controls that pass, as they should: the same new
it()appended after the existing one; the round-2 add-only fixtures.
Are #790, #777, #684 and #656 true changes of an existing expected value or matcher? I read each diff.
- dashboard-switch-speed: a false Board check says which part failed and what the page had #656: yes.
assert.ok(r.ms >= 0)becameif (r.ms < 0) assert.fail(...),assert.equal(..., true)became a helper, and one assertion was made conditional (if (onPrs)). - Test: budget health reads the limited credential's reset time once #790: yes, by its text. The expected
at(600)became a localresetLimited = at(600): the same intended value, computed once. - main red: the #566 switch test stamps its pause on the test's clock, not the wall clock #684: no. The actual changed (
table({ dir: switched })toswitchedAt(now + 1000)) and the clock is now mocked. The expected[]is unchanged. - add-project-view test: the stand-in helper holds until released, so a slow runner can't see the run already failed #777: no. No assertion changed. The test body moved into
followTheRun(), which two tests call. It holds on context.
So two of the four are holds broader than the owner's wording. By principle 1, they are fair as fail closed. They need describing accurately (below), and are noted for the owner.
3. The round-2 gaps rerun, and new attacks (measured, round-2 fixtures unchanged)
- Round-2 rows now held:
- the
{ skip: SKIP_SLOW }const, flipped or env-driven; skip as test, and the swap to a local no-op;- an existing test wrapped in a function or arrow;
- vite and cypress configs,
.github/actions,.worklane/checks, lint-staged and CODEOWNERS; - semgrep, rubocop and deno configs;
- all 13 suppression markers.
- the
- Still passing, as named: actual overwritten or recomputed; helper emptied; fixture module made tautological; string rule;
ls-fileslint; the copied Set. - The mocked unit (tests-only) now passes and isn't named.
files.splice(...)in a registered lint test still passes (round-2 row 14).LINT_BYPASS's(?<![\w$.])lookbehind meanssplice,filteranddeletenever match as a method call. So the body's "splice/filterare words now" is false.
Opened by round 3's add-only fixes (HELD at a8d68ed, pass at 10b66b3, measured on both builds):
- The outer
t.skip()called inside a new subtest:await t.test('setup', () => { t.skip(); }). The skip is credited to the new subtest. Node v22.23.2 (measured on a scratch file): the existing test with a failing assertion reports# SKIP,# fail 0, exit 0. - A new helper function in a
LINT_TESTSfile, called from a new top-level line (function trim() { if (files.length) return files.splice(0); } trim();), engine scope.
New weakenings that pass (each measured; "auto" means no other reason waits, also measured):
| Family | Attack | Result |
|---|---|---|
| Skips | A top-level beforeEach((t) => t.skip()) |
pass. Node: # SKIP, exit 0 (measured). It passed at a8d68ed too. |
| Skips | A new test.only(...) beside existing tests |
pass. vitest and jest run only it by default (inferred). Node --test-only: the failing test isn't run, # fail 0 (measured). |
| Skips | { skip: SKIP_SLOW } with SKIP_SLOW imported from test/env.ts and flipped there |
pass |
| Runner and configs | Makefile test: emptied; phpstan.neon level lowered; .swiftlint.yml rule disabled |
pass, L1 auto in both scopes. pyproject.toml, setup.cfg and Cargo.toml already wait as dependency manifests. |
| Lint tests | test/brand.test.ts (the CLAUDE.md product-name lint): offenders.length = 0; inserted before its assertion |
pass |
| Lint tests | A new test or describe placed first that empties the lint's shared list (files.splice(0), or files.length = 0 in a new before hook) |
pass |
| Lint tables | A case table built by a spread (const CASES = [...BASE]), a row removed from BASE |
pass |
| Lint regexes | An unregistered lint-shaped test whose rule words sit in an array (new RegExp(WORDS.join('|'))), one word removed |
pass |
| Suppressions | NOLINTNEXTLINE and NOLINTBEGIN (the \b after NOLINT stops them), #[expect(, swiftlint:disable, /* eslint x: "warn" */, /* eslint x: ["off"] */, rubocop:todo |
pass |
- Case tables are safe against the obvious trick: a row added while another is removed holds, and so does a row changed and reordered. A row only added passes.
4. False positives (measured independently)
I ran changeFiles plus weakeningOf over the last 150 merged PRs to 128453a, with today's review.yaml, using read-only git.
- 99 of 150 get a reason. Newly held: Test: list scripts/bench/hidden.ts on the changed-files reader list (main red after #688 + #757) #797, Test: budget health reads the limited credential's reset time once #790, add-project-view test: the stand-in helper holds until released, so a slow runner can't see the run already failed #777, main red: the #566 switch test stamps its pause on the test's clock, not the wall clock #684, dashboard-switch-speed: a false Board check says which part failed and what the page had #656, web: Needs you is one queue at /needs-you, blocking first then oldest; its badge counts Waiting on you (#585 part 2) #627, web: the sidebar's pages become tabs in six groups; every old route redirects (#585 part 1) #620, Fix: siri-ask cost-leak check uses a value that can't occur by chance (#546) #547 and No late answer moves a control on Settings, Inbox or Decisions (#455) #503. This is the builder's count and list.
- Tests-only PRs: 11 of 16 wait.
- Tests-only PRs that only add tests: slot dir: a test fails if engine code writes a slot-directory file in place (#616) #654 doesn't wait.
- The round-2 add-only fixtures all pass: a new
it()witht.skip()or{ skip }, a new top-level test witht.skip(), a new test in aLINT_TESTSfile, and a new loop case. - The over-holds in section 2 are synthetic, and none occurs in the 150.
5. Package budget (measured: npm ci, npm run build, npm run build:web, npm pack --dry-run --json --ignore-scripts in my own worktrees)
| Tree | Tarball | Unpacked | Files |
|---|---|---|---|
| main d4b4503 | 1,902,158 | 5,934,411 | 512 |
| PR base 128453a | 1,894,384 | 5,911,738 | 512 |
| head 10b66b3 | 1,918,351 | 5,993,616 | 513 |
-
The PR's file-list diff (head against base) is exactly four files:
dist/src/merge-policy-weakening.js+80,220 (new);change-files.js+1,261;merge-flow.js+259;merge-policy.js+138.
That is 81,878 in all. main against base touches none of them.
-
Merged size. main + 81,878 = 6,016,289 unpacked (inferred by addition), the builder's measured figure.
- I did not build a merged tree (rule 12). The builder's merged tarball of 1,926,615 is plausible: by addition I get 1,926,125 (inferred).
-
Arithmetic. 1,926,615 × 1.02 = 1,965,147, rounded up to 1,970,000. 6,016,289 × 1.02 = 6,136,615, rounded up to 6,140,000. Both correct.
-
At 10b66b3 only the budget line changed in test/package-files.test.ts (+1/−1).
-
What's in the 80 KB module (measured with TypeScript's printer):
- about 22.7 KB of it is comments;
- about 57.5 KB is code;
- 12.9 KB is string and regex literals, with one literal over 300 characters (the
ASSERT_WORDregex). - The biggest parts:
weakeningReasons18.6 KB,testFacts13.5 KB,CHECK_TABLES5.0 KB,GUARD_FILES2.2 KB,NOT_A_LINT1.1 KB. - No fixture or test data is embedded. The registries name this repo's test paths, but they are runtime data the hold reads.
testFactsWorkis exported for tests only, and it is small.- The comments are the trimmable part, as Project commands run as the agent user only in a sandbox that hides its credentials; their output is redacted (#713) [PROTECTED: owner merges] #787 did.
-
Now moot. test: raise the package budget to main's size + 2% after #805 and #787 [PROTECTED: owner merges] #823 merged main's own raise (1,950,000 / 6,060,000). The current head 3871c80 takes main's line, and the PR no longer touches test/package-files.test.ts (measured with
git diff d5674d04 3871c80d --stat). CI's budget test passes there (below). By owner decision 2 no raise is needed, since the PR isn't over budget. The body's "Package budget" section is stale.
6. The usual checks
- Fail-without (measured). I used a scratch worktree at 10b66b3 with
src/merge-policy-weakening.tsandsrc/coordinator/change-files.tsfrom a8d68ed (git checkout a8d68edb -- ...), then rannpm run build(5 TS errors for the new exports; it emits anyway) andnode --import ./dist/test/isolate.js --test dist/test/merge-policy.test.js. Result:# tests 58 # pass 47 # fail 11, the builder's count.- 10 fail on an assertion.
- Intake: an edit to a refused issue's done_when gets judged again #43 ("no file under the cap can stall the parse") fails on
weakening.testFactsWork is not a function: a missing export, not the behaviour.
- The two changed expectations (rule 10). Engine install ships the dashboard's web UI #32 (adversarial pass) and CI fix runs: a failed required check on a harness PR gets a bounded fix #39 (round-2 adversarial) now expect
[ASSERTION, 'removes or changes code in a guard or lint']where they expected only the second. They are stricter, not weaker: the regex named by the assertion is its matcher, and the insertedcontinuemakes the assertion conditional, which the owner's rule holds.- The decision test's reworded
[]to[ASSERTION]follows the owner's decision. - One more loosened expectation isn't named in the body. The round-2 test "every
LINT_TESTSentry reads as a lint" becamelintShaped(text) || LISTS_SOURCES.test(text). It is the PR's own test, not main's, so rule 10 doesn't bind, but principle 5 asks to name it.
- The decision test's reworded
- At head (measured):
- merge-policy, change-files-renames, review, slot-path, package-files and brand:
# tests 179 # pass 179 # fail 0(scratch HOME, TMPDIR and state dirs); npm run typecheckis clean;check:denylistreports "1306 files clean";scripts/check-dco.mjs d4b45039 HEADreports "all commits signed off".
- merge-policy, change-files-renames, review, slot-path, package-files and brand:
- Hygiene (measured).
- All 4 PR commits have a
Signed-off-by. - There is no attribution in the commits or the body.
- No new dependencies, no product name in the module, and no home or state path in the tests (grep). "Closes Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783" is present.
- All 4 PR commits have a
- Protection (measured).
- The title is
[PROTECTED: owner merges], the label isneeds-owner, the risk class isprotected, and both sections are real. - With
computeLevel, every src file of the PR is L3 byself-modification, and so are docs/design.md, test/merge-policy.test.ts and test/package-files.test.ts. - "Make anything merge that waited before" still holds (inferred: only appends reasons).
- "Compromised component" is false on the stall (section 1).
- The title is
- Adversarial pass. The section is present, with a round-3 table. It missed the uncounted regex and pairing costs, and the holes in section 3.
- CI.
- Overlaps (measured with
git merge-tree --write-tree).- 10b66b3 against d5674d0 conflicted only in test/package-files.test.ts, the expected budget line.
- 3871c80 is clean with main and with every open PR except Remove the desktop app, its source, Tauri lockfile and docs; keep the desktop check one more release as a no-op (#744) [PROTECTED: owner merges] #746, which conflicts in the same four files it already conflicts in with main.
- First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816 still has no
TRACEDentry forreasons.push(...(i.weakening ?? []))(inferred from its test). Whichever of the two lands second adds it.
- CHANGELOG (measured in memory). The bullet and its 5 sub-bullets survive
withSection(changelogSection('0.5.3', ...))at both 10b66b3 and 3871c80.
Required changes
-
Bound the time the hold actually takes, not just the characters it walks. Principle 1; section 1 measured 26–39 s per parse, 132 s and over 200 s per change, and an out-of-memory crash under a 1 GB heap.
- Make each whole-text regex linear:
ASSERT_CALL: no lazy identifier-chain prefix. Matchassert|expectand walk back.- The declarations regex: bound its type annotation, e.g.
[^=;\n]{0,200}. assertingFunctions:[^(]{0,200}.
- Count
spanFrom's scan inworkand stop it at a bound. - Pair changed assertions without gone × left: bucket them by token shape, or hold without pairing past a fixed count.
- Cap the total key characters per file, and hold unread past it.
- Add the four shapes, the all-changed file and the long-key pair as tests. A coarse wall-clock ceiling is acceptable as a backstop, since
workcan't see regex backtracking.
- Make each whole-text regex linear:
-
Owner's decision 1, as decided:
- (a) A tests-only change of a non-JS test, a snapshot or an expected-output fixture holds on a removed or changed line. Drop the
codeChangedgate there; added lines still don't hold. - (b) An assertion carries the whole literal value of each const it names (or a hash of it), not its first 400 characters. The same goes for case-table rows.
- (c) A method alias of an assertion (
const eq = assert.deepStrictEqual,.bind(assert)) is an assertion name, like a destructured one. - (d) Adding never changes an existing assertion. Count only exits that can stop it: none inside a function body or test call that closed before it.
- Add a fixture for each.
- (a) A tests-only change of a non-JS test, a snapshot or an expected-output fixture holds on a removed or changed line. Drop the
-
Close the two holes round 3's add-only fixes opened. Both are measured HELD at a8d68ed and pass now.
- The outer
t.skip()inside a new subtest: a body skip counts for the test whose context it calls, so a skip on a name bound outside the innermost test call counts for the test that binds it. - In a
LINT_TESTSfile, a new function called from a new line outside a new test: read its lines as bypasses. - Also hold a line in a new test, hook or function of a lint test that writes a name declared outside it (
files.splice(0),files.length = 0).
- The outer
-
The new weakenings in section 3. Close the cheap ones:
- a top-level
beforeEach/beforehook with a skip; .onlyon a new test: anonlyanywhere in an existing file is a change to every test in it;LINT_BYPASS's lookbehind, so.splice(,.filter(and.length =count;- in a registered lint test, hold every added line inside an existing test or at top level (web: while paused, start actions say they take effect on resume, File it and Add ready say they happen now, and land approvals say they land now #664's lines sit in new tests and functions, so they stay exempt);
- the markers
NOLINT(prefix:NOLINTNEXTLINE,NOLINTBEGIN),#[expect(,swiftlint:disable,rubocop:todo, and ESLint inline config towarnor1and its array form; **/Makefile,**/justfile,**/phpstan.neon*and**/.swiftlint.ymlas guard files.
Name the rest in the body, docs/design.md and the CHANGELOG (principle 5):
- a skip const imported from a helper;
- a case table built by a spread;
- a lint's rule words in an array;
- a mocked unit in a tests-only change.
- a top-level
-
Correct the body, docs and CHANGELOG (principle 5):
- the speed claims in section 1 and "Compromised component";
- "
splice/filterare words now"; - the stale "Package budget" section (3871c80 takes main's 1,950,000 / 6,060,000, and CI passes);
- the unnamed
lintShaped || LISTS_SOURCESloosening; - the four decision-1 PRs as they are: dashboard-switch-speed: a false Board check says which part failed and what the page had #656 changed matchers; Test: budget health reads the limited credential's reset time once #790 put its expected value in a local; main red: the #566 switch test stamps its pause on the test's clock, not the wall clock #684 changed the actual and mocked the clock; add-project-view test: the stand-in helper holds until released, so a slow runner can't see the run already failed #777 moved assertions into a helper.
- Add the round-3 attacks above to the Adversarial pass table.
For the owner (not decided here)
- The hold is wider than decision 1's wording. It holds any change to an existing assertion's text or context, not only its expected value or matcher. Two of the four tests-only PRs it newly holds (main red: the #566 switch test stamps its pause on the test's clock, not the wall clock #684, add-project-view test: the stand-in helper holds until released, so a slow runner can't see the run already failed #777) changed neither: a fail-closed hold by principle 1 (inferred). If you want only expected-value and matcher changes to wait, that is a narrower rule and a new trade-off.
- The budget needs no decision now. Under decision 2, the merged head fits main's current budget (CI green on 3871c80).
…rser (#783) The owner decided (2026-10-19) to replace the 80 KB parser with a line-based rule read from the PR's own diff: - a test file: a removed or changed line with an assertion or a test on it (CHECK_LINE, one regex of literal alternatives, no quantifier), and a deleted test file; - a snapshot, fixture or other expected output: any removed or changed line; - a guard or lint file (GUARD_FILES, LINT_TESTS): any removed or changed line; - a registered table (CHECK_TABLES), read as its lines: an entry gone from a rule or guard list, or new in an allowlist; - an allowlist file: any added line; - any file but docs: an added line with a skip, only or suppression mark (MARKERS, plain substrings; the owner accepts the false holds). Bounded and fail closed: one pass over the changed lines; over 4,000,000 characters, a diff that doesn't match git's counts, a file that isn't text, a table file over 500,000 bytes or a git failure, the PR waits. The registry lints (check-shaped tables, tests that list the sources) stay as backstops in test/merge-policy.test.ts, next to the merge-time check. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Evaluator verdict: CHANGES NEEDED (round 4)I reviewed head What works (measured):
What blocks a pass:
1. Does the rule match the owner's decisions?Decision 1 (the line rule): met, with one stricter point and several looser ones.
Decision 2 (marks): the owner's own examples are covered, but "and similar" isn't.
Decision 3 (lints and guards, same line shape): met in shape, with one hole.
No parser left (inferred, from reading all 455 lines). The module uses substrings, one quantifier-free regex, path globs, and 2. Fail closed and boundedAttacks (measured, real git:
Worst-case time (measured,
3. MarkersSee section 1 for the owner's forms, the "similar" forms that pass, the hidden-mark bypass and the docs exclusion. 4. Round 3's attacks against the new rule (measured: round 3's 51 fixtures, unchanged, fed to the head module)
5. Fail-without (measured, owner method (b))I made a fresh worktree at base 6e0d7d8. I wrote the PR's
6. The 150 PRs (measured: all 150, the last 150 merged to 128453a, review.yaml at 128453a, read-only git)
The named PRs:
7. Package size (measured:
|
| Tree | Tarball | Unpacked | Files |
|---|---|---|---|
| main 6e0d7d8 | 1,903,282 | 5,938,018 | 512 |
| head ca0b16f (main merged in) | 1,909,332 | 5,963,338 | 513 |
- The delta is +6,050 / +25,320, the builder's exact figures.
merge-policy-weakening.jsis 23,662 bytes. - It fits main's
PACKAGE_BUDGET(1,950,000 / 6,060,000), so no raise is needed.
8. Rule 10 (measured)
- main's file has 22 tests, 3871c80 had 58, and the head has 40.
git diff 6e0d7d81 ca0b16f1 -- test/merge-policy.test.tsis +415/−0, so no line of main's tests changed.- The 36 removed or rewritten tests are 58 − 22, all this PR's own. Each of the 33 removed
test(titles maps to the body's "kept" or "removed" lists (inferred, by reading them side by side). - One of the removed tests is Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783's own acceptance check for renames: "renames a test file with the same text, does not wait". Rule 10 doesn't bind the PR's own tests, but the issue's Deliverable does (required change 2).
- Main's
test/fix-forward-coordinator.test.tsnow fails (section 9). Making CI green must not weaken it. Any change to it is a change to an existing main test, so the body names it (rule 10).
9. The usual checks
- CI on ca0b16f: red, caused by this PR (measured). Run 39326108484,
test shard 2/4fails on node 22 and node 24, with# fail 2each. The failing test is test/fix-forward-coordinator.test.ts: "two PRs green alone, merged one after the other ..." (merged one after the other: expected[5, 6], actual[6]) and "the fix PR fails its CI ...".- Locally the file gives 0/2 at the head and 2/2 at main 6e0d7d8 (measured).
- The cause (inferred from the test): its "rename" PR runs
replaceAll('totalCents', 'totalInCents')ontest/price.test.js. That changes assertion lines, so the PR now waits and is never merged. - Every other check passed: shards 1, 3 and 4, scan, scope, push-gate, new-deps, desktop, denylist, dco and agentshield.
- Targeted tests at the head (measured): merge-policy, change-files-renames, review, slot-path, brand and package-files give
# tests 161 # pass 161 # fail 0. - Checks at the head (measured):
npm run typecheckis clean (both projects), andcheck:denylistreports "1310 files clean". - Commits (measured): all 7 commits since main have
Signed-off-by. No attribution appears in the commits or the body (grep: 0). - Protection (measured with
computeLeveland the head's review.yaml):- every
srcfile, docs/design.md and test/merge-policy.test.ts is L3self-modification, and CHANGELOG.md is L0; - the title is
[PROTECTED: owner merges], the label isneeds-owner, and the risk class isprotected; - "What this can never do" and "Compromised component" are real.
- "Make anything merge that waited before" holds (inferred: the diff outside the new module only appends a reason).
- "Stall the merge path ... worst case 464 ms" is wrong (section 2).
- every
- Durability: met (inferred). The protection is a runtime check in
mergeDecisionFor, and the two lints are named as backstops. - Overlaps:
git merge-tree --write-tree origin/main ca0b16f1is clean, and origin/main is 6e0d7d8 (measured). - The
TRACEDnote for First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816 is accurate. First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816 (head 640b6cd, open) has noweakeningentry in test/first-task-progress.test.tsTRACED(measured with grep). Its own fixturereasons.push(...(i.instructionEvalsExtra ?? []))is flagged "untraced", soreasons.push(...(i.weakening ?? []))will be too (inferred). Whichever lands second adds the entry. - Claims: labelled. The corrections are in required change 9.
Required changes
-
Make CI green without weakening the fix-forward test (measured red, section 9). Under decision 1 the fixture's rename PR now waits, because it changes assertion lines. Change the scenario so that it still proves fix-forward. For example, the rename PR could leave the test's assertion lines alone (an alias kept for the old name), or the test could approve the held PR. Name the change to this existing main test in the body. If it can't be done without loosening what the test proves, keep the test and take it to the owner (rule 10).
-
A pure rename doesn't wait (Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783's Deliverable, and the owner's decision 1).
- A deleted test path whose exact blob reappears at a new path that is still a test path with the same runner suffix counts as neither deleted nor removed lines.
- A rename that edits the content, or leaves the runner's naming, still waits (round 2's Run a coordinator per instance, with only that instance's logins #3).
- Restore an acceptance test for both through
decideChange(real git).
-
A non-text line must not hide marks.
- Read
MARKERSon every added line of a non-doc file, whatever else the file holds. - Or make a non-text change to a file that isn't an asset (a code or config extension) wait as unreadable.
- Add fixtures: NUL, a literal U+FFFD, and a Latin-1 byte, each beside a mark.
- Read
-
Make the table reader fail closed. When the table's name is declared more than once on either side (a decoy in a comment or string), or when the line that ends it isn't a closer (
],},)), treat the table as unreadable, so the change waits. Add the four fixtures from section 1. -
Add the common "similar" marks, or name each one you leave out.
- Go
Skipf(; - Python
skipTest,SkipTest,importorskip; - Rust
#[ignore; - Java
@Disabled,@Ignore; - vitest
.skipIf(,.runIf(,.skip.each; - mocha
xcontext(; jasminepending(; - Swift
XCTSkip; C#Skip =,#pragma warning disable; - Kotlin
@Suppress(; PHPphpstan-ignore,psalm-suppress; pylint:disableandtype:ignorewithout the space;- RSpec
xit/skipwithout parentheses, if a substring for them doesn't flood the false holds.
Give each a fixture, and re-measure the 150 PRs.
- Go
-
Assertion words for every extension in
TEST_CODE. Add C#Assert./[Fact/[Test, SwiftXCTAssert/XCTFail, Rubyrefute/must_/wont_, and bats@test. Or take those extensions out ofTEST_CODE, so every removed line of such a test waits (principle 1). Add a fixture each. -
Nested test paths. Add
**/test/**,**/tests/**and**/spec/**toTEST_PATHS, and a nestedfixtures/under them. Or name the gap. Re-measure the 150 PRs. -
Marks in a test or guard file under
docs/, or in a.md/.txttest file: read them. Skip marks only where the file is skipped (isDoc && !strict). -
Correct the timing claims (principle 5) in the body's bound table and "What this can never do".
- The measured worst case at the cap is 0.8–1.3 s (load 11–13), and 2.3–2.7 s end to end at load 47–48, with +141 MB heap.
- "Over the cap waits in 22 ms" is true only for one file; four files took 258 ms.
- Optional: keep only the first three places and a count per kind, which removes the per-line map cost.
-
Name the rest in the body, docs/design.md and the CHANGELOG (or close them):
- a mode-only change to a hook or guard (exec bit dropped);
- a gitlink outside the test paths;
- Rust's inline
#[cfg(test)]tests insrc/(narrow "covers Rust"); - the stricter holds: a deleted helper with no assertion, and a whole-file line-ending change.
-
Add each attack above to the Adversarial pass table.
For the owner (not decided here)
- The
xit(false hold.process.exit(holdsxit(in 25 of the 150 PRs (builder's count). In those 150 it newly held none: each such PR waits anyway (measured, section 6). In a managed app withprocess.exit(in ordinary code, it would newly hold PRs (inferred). The plain-substring rule would need a one-character look-behind forxit(/fit(. That is a small change to the "plain substrings" design, so it's your call. - Renames. Required change 2 follows Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783's Deliverable and decision 1 as I was briefed. If you want renames of test files to wait too, say so, and change 2 drops.
- The fix-forward test. Decision 1 holds a rename that touches assertion lines. So the engine's own fix-forward scenario test (a rename PR plus a caller PR) no longer auto-merges its first PR. The test's fixture must change (required change 1).
Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…found it open (#783) - A pure rename of a test file or folder doesn't wait: exactMoves (the changed-files helper) pairs a deleted path with an added one of the same blob and mode; both are skipped when they are test paths with the same runner suffix. A rename with an edit is still a delete and an add. - Marks are read on every added line; a changed file that isn't text (a NUL, a U+FFFD, a byte that isn't UTF-8) waits, unless it is an asset by its extension. - A registered table declared twice, with a line at its indent inside it, or with no close makes the change wait; a line that closes every bracket it opens is a one-line table. - More marks (Go, Python, Rust, Java, vitest, mocha, jasmine, Swift, C#, Kotlin, PHP, RSpec) and assertion words (C#, Swift, minitest, bats); shell test files wait on any removed line; test folders count at any depth, and a test file under docs/ is a test file. - The cap is checked before any line is read, and each kind keeps only its first three places and a count. - test/fix-forward-coordinator.test.ts: the rename PR imports the renamed function under its old local name, so its assertion lines stay as they are and the scenario still auto-merges it. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Evaluator verdict: CHANGES NEEDED (round 5)I reviewed head The follow-up commit (measured): Most of round 4 is fixed (measured).
What blocks a pass is the new pure-rename exemption. It also lets through renames that stop a test from running, and moves of guard and lint files. It's too wide. There is also one table trick left. 1. Round 4's attacks, rerun (measured: the same fixture scripts, at this head)
2. Attacks on the new pieces (measured, real git:
|
| Change | Result |
|---|---|
test/math.test.ts to test/sum.test.ts; test/unit/ to test/core/ |
passes (as decided) |
Rename with an edit; to .test.ts.bak; to .spec.ts; pkg/math_test.go to pkg/math.go |
waits |
| Same blob, exec bit dropped in the move (mode differs) | waits |
| A copy (same blob added, original kept) | passes (nothing is lost) |
| Two identical tests on main: one deleted, one renamed | waits (the second delete stays a delete) |
| Two tests swap names | waits (a stricter hold, fine) |
A gitlink moved from test/suite to vendor/suite, same commit |
waits |
tests/test_math.py to tests/math.py, or to tests/math_helpers.py |
passes. pytest only collects test_*.py/*_test.py by default (inferred), so the test stops running. |
src/test/java/MathTest.java to MathCase.java |
passes. Surefire selects *Test.java, Test*.java and *Tests.java by default (inferred). |
spec/math_spec.rb to spec/math.rb |
passes. rspec's default pattern is *_spec.rb (inferred). |
Engine: test/math.test.ts to scripts/math.test.ts |
passes. The engine runs only dist/test/**/*.test.js (measured: package.json test), so the test stops running (inferred). |
test/math.test.ts to docs/math.test.ts, or to test/fixtures/math.test.ts |
passes |
tests/conftest.py to tests/old/conftest.py (a guard file) |
passes. A conftest only applies to its own folder and below (inferred), so an autouse fixture or hook stops covering the tests it covered. |
test/tsconfig.json and test/jest.config.js, moved into test/old/ (guard files) |
passes |
Engine: test/brand.test.ts (in LINT_TESTS) to test/brand-name.test.ts |
passes the hold. The backstop "every LINT_TESTS entry is a real file" would then fail in CI (inferred). |
Engine: test/isolate.ts (a guard file) to test/isolate-old.ts |
passes the hold. It is L3 by the lock anyway (inferred). |
Why: runnerSuffix is only the file extension unless the name holds .test./.spec., and TEST_PATHS is much wider than what any runner selects (every file under test/, tests/, spec/, plus **/*.test.* anywhere). The exemption also covers files that are guard, lint, allowlist or table files as well as test paths. #783's deliverable is "a rename with the same content doesn't hold". A move that takes a test out of what its runner runs is a deleted test (decision 1), and principle 1 says to fail closed. Required changes 1 and 2.
The asset exemption (principle 9).
- It narrows round 4's skip: round 4 skipped every non-text, non-strict file, while now only a known asset extension is skipped, and every other non-text file waits.
- It loosens no check on main, because the hold is new in this PR (inferred).
- A test or guard file can't use it, because
asset()requires!strict(path): a binarytest/data.binchange waits (measured). - A non-test
app/run.binholding// eslint-disableand a NUL passes (measured). Linters don't read those extensions, so the mark there turns nothing off (inferred). - No change needed.
The table rules.
- A duplicate declaration, a column-0 line inside the table, and a table that never closes each wait (measured, the PR's fixtures and mine).
- A one-line table: changing the declaring line to fake a close (
= { // }) changes that line, which waits in both kinds (inferred frommissing). - Still open (measured): a decoy copy of the table in a
/* */comment above it, plus the real declaration made unreadable to the rule (var ALLOWED, orconst ALLOWEDwith two spaces), plus an entry added to the real table. This passes fortest/instance-name.test.tsALLOWEDin engine scope.- Without the decoy, the same
varchange waits ("removes or renames a check's or guard's table"). - The rule then sees only one declaration, the decoy, and the decoy is identical to the base's table.
- Required change 3.
- Without the decoy, the same
The single marks regex. It is escaped literals joined by |, with no quantifier.
- 4 files of near-miss prefixes of the longest marks (
@pytest.mark.ski,#pragma warning disabl, ...), as single lines and as many lines, 3.8 M characters: 25 ms (measured). No catastrophic case found (inferred: no quantifier means no backtracking). - The RSpec forms
xitandskipwiden the false holds (section 4).
3. Bound and timing (measured, load average 14–20, weakeningReasons alone, one process each)
| Shape (4 files, together under 4,000,000 characters) | Time | Heap |
|---|---|---|
| Distinct short removed lines in guard files (round 4's worst, 0.8–2.3 s) | 52 ms | +17 MB |
| One-character lines in guard files | 514 ms | +7 MB |
| Every removed line an assertion | 43 ms | +10 MB |
| Every added line a mark | 48 ms | +13 MB |
| Near-misses for every word and mark | 22 ms | +5 MB |
| Marks-regex near-prefix lines | 25 ms | +1 MB |
| 40,000 test files | 163 ms | +9 MB |
| 4 table files of 499,000 bytes | 208 ms | +30 MB |
| Over the cap, four files | 44 ms | +2 MB |
- End to end at the cap, in a scratch repo (4 guard files):
changeFilestook 277 ms andweakeningOf89 ms. - These are consistent with the body's 0.41–0.67 s worst case and 38–39 ms for my round-4 shape. My 44 ms against the body's 17–20 ms for over the cap is load.
- A minor point, not required: once a kind has three places,
hitcounts repeat places too, so "and N more" can overcount.
4. The 150 PRs (measured: all 150, the last 150 merged to 128453a, review.yaml at 128453a)
| Builder | Mine | |
|---|---|---|
| With a reason | 114 | 114 |
| Newly held | 12 | 12: #797, #790, #777, #664, #653, #627, #684, #656, #620, #594, #547, #503 |
| Tests-only PRs held by this rule | #797, #790, #777, #692, #653, #654, #652, #684, #656, #547 | |
| #654 | reason from skip, waits anyway |
the same: const skip = process.platform === 'win32' ? ... |
| Held only by a mark | 29, 3 of them new (#664, #594, #503) | |
Slowest weakeningOf |
62 ms (#802) |
The named PRs:
- main red: the #566 switch test stamps its pause on the test's clock, not the wall clock #684, dashboard-switch-speed: a false Board check says which part failed and what the page had #656, Test: budget health reads the limited credential's reset time once #790: assertion lines.
- add-project-view test: the stand-in helper holds until released, so a slow runner can't see the run already failed #777: an assertion line, plus
process.exit(matchingxit(. - escape fixture: refuses to run outside the MCP broker's sandbox, and never as root (#608) #653: the fixture server's lines.
- first run: an open panel stays put across a poll that changes the detail lines above it #594: a platform
skip:, a false hold. - web: while paused, start actions say they take effect on resume, File it and Add ready say they happen now, and land approvals say they land now #664:
pending(, a false hold.
The 8 PRs that gain a reason over round 4 all wait anyway except #664. Several come from skip in ordinary text or names: #638 (a string, "Totals skip zero quantities"), #654 and #539 (const skip = ..., the latter in src/coordinator.ts).
5. Fail-without (measured, owner method (b))
I used a fresh worktree at base 71316eb. I wrote the PR's test/merge-policy.test.ts and test/fix-forward-coordinator.test.ts into it with Edit (each cmp-identical to the head copy), ran npm run build, then ran node --import ./dist/test/isolate.js --test dist/test/<file>.test.js.
- With the tests alone:
- merge-policy fails to load (
ERR_MODULE_NOT_FOUND: the module is new, so this is unavoidable); - fix-forward passes 2/2, so the new fixture works on main as well.
- merge-policy fails to load (
- With round 4's module (sha256 matches
ca0b16f1), the head'schange-files.ts, and the wiring (eachcmp-identical):# tests 44 # pass 37 # fail 7. The 7 are all merge-policy, and fix-forward passes 2/2. That is the builder's 7 of 42. The seven:- other languages;
- pure rename;
- the table tricks;
- the marks;
- bounded and fail closed;
- the rename acceptance test;
- the non-UTF-8 acceptance test.
- At the head: merge-policy, change-files-renames, review, slot-path, brand, package-files, fix-forward-coordinator and changelog give
# tests 175 # pass 175 # fail 0.
6. Rule 10: main's fix-forward test (measured diff, plus inferred reading)
- The only change is the worker's rename step.
test/price.test.jsnow importstotalInCents as totalCents, so its assertion lines stay, and a newassert.notEqual(renamed, t, ...)checks that the import line was there to change. - No assertion of the fix-forward test was removed or changed, and the scenario still breaks main:
src/report.jsstill imports the old name (inferred). - It passes on main with the new fixture (2/2, measured), so it doesn't depend on the hold to pass.
- The body names it under "Rule 10: main's fix-forward test".
- The file is L0 (
tests) under the engine's review.yaml (measured withcomputeLevel). The PR is protected anyway. - One stale line: the body's older Rule-10 list still says "(round 2, Run a coordinator per instance, with only that instance's logins #3) renamed with another runner suffix: every move deletes a path, and waits". Pure renames no longer wait.
7. The usual checks
- CI on 7c632eb: green (measured). 19 checks succeeded, including all 8 test shards on node 22 and 24, build, desktop, push-gate, dco, denylist, scan and agentshield. 7 were skipped (the real-systemd jobs), and 1 was cancelled: one of two
new-depsruns, the other succeeded. 9096a3f's run was cancelled by the new push, but its shard 2/4 (node 24), which holds the fix-forward file, passed before the cancel (measured, check runs). - CHANGELOG (measured,
git diff 71316ebd 7c632eb4 -- CHANGELOG.md): it only adds a## v0.5.4 (unreleased)section above## v0.5.3 (2026-10-19), which ends with the empty### Guardrail changes. Main's v0.5.3 section is untouched, and test/changelog.test.ts passes. - Package (measured):
npm ci,build,build:webandpack --dry-run --json --ignore-scriptsgive:- main 71316eb: 1,903,282 / 5,938,018 / 512 files;
- head: 1,911,634 / 5,967,997 / 513 files;
- delta +8,352 / +29,979, the builder's exact figures, within main's budget (1,950,000 / 6,060,000).
- Checks at the head (measured): typecheck is clean, and
check:denylistreports "1310 files clean". - Commits (measured): all 10 commits since main are signed off, and grep finds no attribution in the commits or the body.
- Protection (measured with
computeLevel): everysrcfile, docs/design.md and test/merge-policy.test.ts is L3self-modification. The title, label, risk class and both sections are present. - First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816
TRACED: First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816 is still open at 640b6cd with no entry forreasons.push(...(i.weakening ?? [])). Whichever lands second adds it (inferred, as round 4).
Required changes
- A pure rename may skip the hold only if it keeps the test selected by its runner.
- Compare the runner's naming class, not just the extension:
test_*.py/*_test.py,*Test.java/Test*.java/*Tests.java,*_spec.rb,*_test.go,*.test.*/*.spec.*with the same suffix,*.bats. - Keep the old path's test root: the nearest
test/,tests/,spec/,__tests__/orTests/folder, or the top folder for a name-matched file. - Never move into
fixtures/,__snapshots__,testdata/ordocs/. - Anything else stays a delete and an add (principle 1).
- Add the measured cases above as fixtures, with real git for at least the Python and
scripts/ones.
- Compare the runner's naming class, not just the extension:
- Never exempt a rename of a guard, lint, allowlist or registered-table file, even when it is also a test path: conftest.py, tsconfig and jest configs under
test/,LINT_TESTS,test/isolate.ts. Restrict the exemption toisTest && !isGuard && !isAllow && !table. Add a fixture each. - Close the decoy-plus-unreadable-declaration trick.
- When the base's declaring line is among the file's removed lines, treat the table as gone or unreadable, so the change waits.
- Also count a declaration in any of
const,letorvar, with any run of spaces. - Add the measured fixture: a decoy comment, then
var ALLOWEDwith an entry added.
- Correct the body (principle 5):
- remove the stale Rule-10 line "(round 2, Run a coordinator per instance, with only that instance's logins #3) ... every move deletes a path, and waits";
- state the rename rule's real reach after changes 1 and 2;
- add these attacks to the Adversarial pass table.
For the owner (not decided here)
- The false holds grew. 114 of 150 PRs now get a reason (106 in round 4, 99 under the parser). In the 150 there is one new false hold, web: while paused, start actions say they take effect on resume, File it and Add ready say they happen now, and land approvals say they land now #664 (
pending(); the rest wait anyway.skipmatches prose and names (const skip = ...insrc/coordinator.ts, a test-data string "Totals skip zero quantities"), so in a managed app it will hold ordinary PRs (inferred). Narrowing it, for example to a line that starts withskipafter indentation, is a small change to the plain-substring design, so it's your call. The same applies toxit(matchingprocess.exit((25 of 150). - Renames. Required changes 1 and 2 keep pure renames passing as decided. They only stop moves that drop a test from its runner, or that move a guard file. If you'd rather every rename wait (simpler and fail closed), say so, and those two changes collapse into "renames wait".
…t; a table's declaring line is pinned (#783) - A pure rename is exempt only when both names match the same runner's default pattern with the same suffix, under the same test-folder root, the new one under no fixtures, snapshots, testdata or docs folder, and neither is a guard, lint, allowlist or table file. Any other move waits. - A table is declared by const, let or var with any spacing; the change removing or changing the base's declaring line makes it wait as unreadable (a decoy can't stand in for it). Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Evaluator verdict: PASS (round 11)I reviewed head 1.
|
| Added line | Result (measured) |
|---|---|
eval('ALL' + 'OWED')['x'] = 1; |
waits (changes a check's or guard's table outside its own list) |
eval ('ALL' + 'OWED')['x'] = 1; (round 10's hole) |
waits |
eval/**/('ALL' + 'OWED')['x'] = 1; |
waits |
eval with (...) on the next line |
waits (the eval line holds the word) |
new Function('t', 't.x = 1')(globalThis); |
waits |
const s = 'see Function docs'; |
waits: the accepted, harmless over-hold for a word in a string |
const n = 1; (control) |
passes |
The matcher is (?<![\w$])(?:eval|Function)(?![\w$]), a whole word, so any direct-eval spelling waits. The indirect forms I measured in round 10 ((0, eval), an alias, globalThis['ev'+'al']) can't reach a module-scope table: each throws ReferenceError.
2. Package-manager settings (weakeningReasons, managed-app scope)
| Change | Result (measured) |
|---|---|
.npmrc + script-shell=true |
waits (adds a line to an allowlist file or a package manager's settings file) |
.yarnrc + ignore-scripts true |
waits |
.yarnrc.yml + enableScripts: false |
waits |
A nested packages/web/.yarnrc.yml + a line |
waits |
bunfig.toml + [test] root = ... |
waits |
A nested apps/x/.bunfig.toml + a line |
waits |
pnpm-workspace.yaml: a package added |
passes, as intended |
pnpm-workspace.yaml: a line changed (scriptShell: bash to scriptShell: true) |
waits (removes or changes a line of a guard or lint) |
pnpm-workspace.yaml: a new setting added (scriptShell: true) |
passes. This is the named gap ("an added pnpm script setting"). |
3. Fail-without (measured, owner method (b))
I used a fresh worktree at base 71316eb. I wrote the PR's test/merge-policy.test.ts and test/fix-forward-coordinator.test.ts into it with Edit (each cmp-identical to the head), plus round 10's module (sha256 045f3500..., matching 2e9ea4fb), its change-files.ts and the wiring (each cmp-identical). Then npm run build (no type errors), and node --import ./dist/test/isolate.js --test on both files.
-
Result:
# tests 45 # pass 43 # fail 2. Both failures are in merge-policy, matching the body's "41 pass, 2 fail":- Dashboard: only a decision's owner or a writer answers it; costs are today's estimates #30, the table test (
eval (as a whole word); - Push limits: every push of agent-written code is checked first #31, the allowlist and settings test (yarn and bun,
pnpm-workspace.yaml, the reason's new wording).
Fix-forward passes 2/2.
- Dashboard: only a decision's owner or a writer answers it; costs are today's estimates #30, the table test (
-
At the head: merge-policy, change-files-renames, review, slot-path, brand, package-files, fix-forward-coordinator and changelog give
# tests 176 # pass 176 # fail 0.
4. The usual checks
- CI on 9c79a3b: green (measured). 20 checks succeeded, including all 8 test shards on node 22 and 24, and 7 real-systemd jobs were skipped. Nothing failed or was cancelled.
- The 150 PRs (measured, all 150 to 128453a): unchanged. 114 get a reason, the same 12 are newly held, and the slowest
weakeningOftook 36 ms. - Body (measured): round 11's section matches what I measured. That covers the whole-word
eval/Function, the yarn and bun settings files,pnpm-workspace.yamlas a guard with the added-setting gap named, and "41 pass, 2 fail". - CHANGELOG (measured): against main it removes no line; only the v0.5.4 section changes, and v0.5.3 is untouched. The changelog test passes.
- Checks at the head (measured): typecheck is clean, and
check:denylistreports "1310 files clean". - Commits (measured): 9c79a3b is signed off, and grep finds no attribution in any commit since main.
- Protection: unchanged. The PR is still marked protected (
[PROTECTED: owner merges],needs-owner), and the owner merges.
Named gaps (as the body names them; confirmed where I probed)
- Changes the line rule doesn't see:
- a changed line with no assertion on it;
- a pure insertion with no mark;
- a config line added with no mark.
- Ways around a registered table:
- a copied table grown (
copy.set); - mutation from another module (L3 in the engine, so it waits anyway).
- a copied table grown (
- Unread paths and files:
- Rust's inline tests in
src/; - a hook losing its executable bit;
- a submodule outside the test paths;
- an added pnpm script setting;
- Makefile,
phpstan.neonand.swiftlint.yml.
- Rust's inline tests in
- Marks: the long tail of other runners' and linters' forms.
For the owner (unchanged, nothing new)
- Renames: 0 renames to date in this repo (measured in round 8), so "every rename waits" would cost nothing here. Your call.
- Fixture moves now wait.
- One-line-table reasons no longer name the entry.
- The false holds from
skip,pending(andxit(: 114 of 150 PRs get a reason, and web: while paused, start actions say they take effect on resume, File it and Add ready say they happen now, and land approvals say they land now #664 is the one newly held false hold.
…-783-weakening-hold Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com> # Conflicts: # CHANGELOG.md
Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com> #816 landed first: its scan of merge.decided's reason sources needs the entry for reasons.push(...(i.weakening ?? [])) in mergeDecision. Test data only.
…ain (#783) Main's machine-path lint names its guard (file and function) in a const GUARD; the backstop asks every check-shaped table to be registered. Changing which helper counts as the guard narrows the lint: never-shrink. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…g-hold Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com> # Conflicts: # CHANGELOG.md
) Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com> #856's golden runner settings: the deny list and hooks a run's --settings must carry. The backstop asks every check-shaped table to be registered; shrinking it weakens what a run is held to: never-shrink.
…owner) Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…owner) Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…owner) Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…owner) Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…owner) Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…owner) Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…owner) Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
main: docs-guardrails test turns #820's weakening input on, and the page lists its hold [PROTECTED: owner merges]
… at the top level Round 1's review measured two gaps in #881's declarations: - #820's weakening hold let an added `// runtime-import:` line through in any file. Each declaration turns tests that ran on every change into selected ones, which is a loosening (decision guide 9). `runtime-import:` joins MARKERS in src/merge-policy-weakening.ts, so a change that adds or changes a declaration line waits for the owner. Taking one out does not wait: its tests run on every change again. - With no `{…}` around the import (a top-level `const w = () => import(…)`), the check fell back to every string in the file, so a declaration of a file that other code there names was taken (route C). It is now refused at the top level (fail closed). Docs and the changelog fragment say both. Test (fails without): test/merge-policy.test.ts "a test-selection declaration ... waits" (an added declaration gave [] in scripts/up.cjs); test/ci-affected.test.ts top-level cases (route C gave an edge to src/c.ts). Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…eck table (#884 round 2) #820's weakening backstop found scripts/bench/pins.ts SELECTION_RULES, a check-shaped table, neither registered nor excused, and failed test shard 4/4. It is the list of rule names pins.json may name: a new name widens what the pins accept. So a line added to it holds for the owner (never-grow). One line added to CHECK_TABLES, nothing else changed. Test (fails without): test/merge-policy.test.ts "weakening backstop: each registered table is found where it is registered, and every check-shaped table in test/ and scripts/ is registered or says why not" (failed in CI on c0529b1). Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Intent: Make the owner's rule that no agent removes or weakens a check structural (decision guide principles 7 and 9, owner rule 10). When a PR removes or changes a test's assertion or test line, deletes a test, changes an expected output, removes a line of a guard or lint, grows an allowlist, or adds a skip, only or suppression mark,
mergeDecisiongets adesign: may weaken a check: ...reason and the PR waits for the owner, whatever its level. The reason comes from the PR's own diff, never from the evaluator's text.Risk class: protected. Every path under
src/isenginein this repo's review.yaml.src/merge-policy-weakening.tsfalls under the lockedsrc/merge-policy*glob ("the merge policy"), and so dosrc/merge-policy.tsandsrc/coordinator/merge-flow.ts.src/coordinator/change-files.tsis "the coordinator", anddocs/design.mdandtest/merge-policy.test.tsare locked too.Closes #783
The by-design protection (owner rule, 2026-10-19)
A runtime behaviour check in the merge gate.
weakeningOfruns inside the coordinator'smergeDecisionForon each PR's own diff, and its reasons decide whether the PR may auto-merge. An agent can't skip it or edit it: the PR it would need is itself held (the module, its registries and its tests are locked,src/merge-policy*andtest/merge-policy.test.ts, L3), and removing a line of the module is a removed guard line.Backstops (lints that read the repo), named as such: two tests in test/merge-policy.test.ts keep the gate's registries whole. One fails on a check-shaped table in
test/orscripts/that is neither registered nor inNOT_A_CHECK. The other fails on a test that lists the sources and is in neitherLINT_TESTSnorNOT_A_LINT. They protect nothing on their own.No model-compensating scaffolding, and no Claude Code-specific parsing (none at all: the rule reads diff lines).
Main db2e3df merged in (#855, #856, #819)
## v0.5.3 (unreleased), neither reworded.git diff origin/main -- CHANGELOG.mdshows only this PR's seven lines.test/runner-golden.test.tsconst GUARDholds the golden runner settings, the deny list and hooks a run's--settingsmust carry. The backstop found it, and it is now registerednever-shrink(coordinator's call: shrinking it weakens what a run is held to). The change issrc/merge-policy-weakening.ts | 2 ++: the entry and its comment.test/adversarial/adversarial.test.tsand changelog give# tests 217 # pass 210 # fail 0 # todo 7(the 7 are the adversarial suite's own todo cases);Main 2ce7726 merged in (the CHANGELOG fold, #816)
Round 11's CI was green: 19 passed and 7 skipped, measured at 9c79a3b. Main 2ce7726 is merged in, not rebased (merge commit 4c25810).
## v0.5.3 (unreleased)section. The Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783 bullet moved from the deleted## v0.5.4 (unreleased)into v0.5.3's "Guardrail changes: what each one does", above its closing empty### Guardrail changes.git diff origin/main -- CHANGELOG.mdshows added lines only, inside v0.5.3, and no 0.5.4 (measured).test/changelog.test.tspasses.TRACEDentry (a9d1601, test data only):['...(i.weakening ?? [])', null]in test/first-task-progress.test.ts.weakeningReasonswith thedesign: may weaken a check:prefix.test/machine-path-writes.test.tsnames its guard in a check-shapedconst GUARD = { file: 'test/isolate.ts', fn: 'onRealMachine' }. The backstop asked for it to be registered.never-shrink: changing which helper counts as the guard narrows that lint.CHECK_TABLES, the only code change since round 11.# tests 203 # pass 203 # fail 0;Round 11 (the round-10 review, CHANGES NEEDED)
Round 10's CI was green: 19 passed and 7 skipped, measured with
gh pr checksat 2e9ea4f. The fixtures were written first and run on round 10's module.evalandFunctionare matched as whole words, noteval(/Function(:(?<![\w$])(?:eval|Function)(?![\w$]). The fixtureeval ('ALL' + 'OWED')['x'] = 1(with a space) waits..yarnrc,.yarnrc.yml,bunfig.tomland.bunfig.tomlare treated like.npmrc: guard files where any added or removed line waits. They share one list,PACKAGE_MANAGER_SETTINGS. The reason now reads "adds a line to an allowlist file or a package manager's settings file". There are fixtures for each, in both scopes.pnpm-workspace.yamlis a guard file: a removed or changed line waits, and an added one doesn't, since ordinary PRs add packages there. Fixtures cover a removedonlyBuiltDependenciesentry (waits) and a package added (passes). Named gap: an added pnpm script setting.Fail-without, round 11 (measured): the new fixtures on round 10's module give test/merge-policy.test.ts 43 tests, 41 pass, 2 fail:
eval (with a space);pnpm-workspace.yaml, and the reason's new wording, which the.denylist-allowassertions there also use).At this head, 9c79a3b (measured):
npm run typecheckexit 0; check-denylist "1310 files clean"; DCO and the push gate clean; the CHANGELOG diff against main is additions only, andtest/changelog.test.tspasses 10/10;# tests 176 # pass 176 # fail 0;Round 10 (the round-9 review: the body, plus two fail-closed tightenings)
Round 9's CI was green: 19 passed and 7 skipped, measured with
gh pr checksat db5c355. The reviewer found no code hole. The fixtures were written first and run on round 9's module.eval(andFunction(wait in a registered table's file. A name built at run time, such aseval('ALL' + 'OWED')['x'] = 1, has no identifier to match. So an added line holding either one waits (table-used). There are fixtures foreval(...),new Function(...)andFunction(...)()..npmrcis a guard file, and any line added to it waits. Its settings (script-shell=,ignore-scripts=) change what install and every npm script run under.**/.npmrcis both inGUARD_FILES(a removed line waits) and inNEVER_GROW_FILES(an added line waits).script-shell=trueadded and a line removed, at the root and inweb/, in both scopes.Fail-without, round 10 (measured): the new fixtures on round 9's module give test/merge-policy.test.ts 43 tests, 41 pass, 2 fail: the table test (
eval(/Function() and the allowlist-file test (.npmrc).At this head, 2e9ea4f (measured):
npm run typecheckexit 0; check-denylist "1310 files clean"; DCO and the push gate clean; the CHANGELOG diff against main is additions only, andtest/changelog.test.tspasses 10/10;# tests 176 # pass 176 # fail 0;.npmrc. The eval/Function words add no hold: Slot dir: one writeSlotFile helper for every slot-dir write; the lint fails on any other [PROTECTED: owner merges] #766 already hadtable-used, and it waits anyway.Round 9 (the round-8 review, CHANGES NEEDED)
Round 8's CI was green: 19 passed and 7 skipped, measured with
gh pr checksat 4dd1abd. This round covers decision 3's case. The tests were written first and run on round 8's module.table-used, "changes a check's or guard's table outside its own list"). It is a plain identifier match,(?<![\w$])NAME(?![\w$]), with no parse.package.jsonscripts) is data, never used by name in its file.Object.assign,.push,.addand a spread declared again, intest/no-clock-helpers.test.ts, plus the same check inchange-files-renames.test.tsandno-wall-clock-durations.test.ts. An entry added inside the literal is stilltable-grown.withoutCommentsstrips//and/* */comments, but not inside a string, in one pass before the reader counts brackets. It also skips an escape (a regex literal's\/).const ALLOWED = // see [docs]with the list on the next line at the same indent now waits as unreadable. A//inside a string is no comment.GUARDRAIL_PATHS, a list of regex literals, needed the escape rule.Fail-without, round 9 (measured by the reviewer; corrected in round 10): on round 8's module, test/merge-policy.test.ts gives 43 tests, 41 pass, 2 fail. The two are:
weakening.withoutComments, an export round 8 didn't have.I first wrote "1 fail" from a run made before I added the backstop's comment stripping (principle 5).
At this head, db5c355 (measured):
npm run typecheckexit 0; check-denylist "1310 files clean"; DCO and the push gate clean; the CHANGELOG diff against main is additions only, andtest/changelog.test.tspasses 10/10;# tests 176 # pass 176 # fail 0;table-usedreason appears in 3 of them, Slot dir: one writeSlotFile helper for every slot-dir write; the lint fails on any other [PROTECTED: owner merges] #766, Review levels: a renamed file counts under its old and its new path, in changeFiles and every path reader [PROTECTED: owner merges] #688 and review: lock the tool guard's whole import tree; engine-source locks stand aside only where root names a managed app [PROTECTED: owner merges] #691, all reading a table in a loop or afind. All 3 wait anyway, so it holds no new PR.Round 8 (the round-7 review, CHANGES NEEDED)
Round 7's CI was green: 19 passed and 7 skipped, measured with
gh pr checksat eb37ee8. Round 8 has two small items, each failing closed. The tests were written first and run on round 7's module.dist/testglob skip it.test/x.test.tstotest/.x.test.tswaits, and there is a fixture for it.=.tableLinesnow reads the declaring line's value: what follows its=(not an arrow's=>), or a JSON key's:.const ALLOWED: string[] =, orconst ALLOWED:with the list below.= // see [docs]with the list below, or= new Set()with.add(...)below.=, every bracket closed, and the next line no deeper. Every registered table passes it today (measured).Fail-without, round 8 (measured): the new tests on round 7's module give test/merge-policy.test.ts 43 tests, 41 pass, 2 fail: the rename unit test (the leading dot) and the table test (the three shapes). The backstop's new assertion passes on round 7's module, because no registered table has these shapes. It is a backstop.
At this head, 4dd1abd (measured):
npm run typecheckexit 0; check-denylist "1310 files clean"; DCO and the push gate clean; the CHANGELOG diff against main is additions only, andtest/changelog.test.tspasses 10/10;# tests 176 # pass 176 # fail 0;Round 7 (the round-6 review, CHANGES NEEDED)
Round 6's CI was green: 19 passed and 7 skipped, measured with
gh pr checksat 84ebfa8. Round 7 closes the review's three findings, each failing closed. The tests were written first and run on round 6's module (nogit checkout <ref> -- <paths>).node_modules/,__mocks__/or any dot folder (.old/,.cache/, ...) is no longer exempt, alongside fixtures, snapshots, testdata and docs. There is one fixture each:test/x.test.tstotest/node_modules/x.test.ts,test/.old/x.test.tsandtest/__mocks__/x.test.ts.exactMovespairs only modes 100644 and 100755, so a symlink (120000) or a submodule (160000) is never paired.test/link.test.tspointing atreal.test.ts, is moved totest/sub/with its link text kept, so it now points at a new near-emptytest/sub/real.test.ts. The move waits as a deleted test file.const ALLOWED:withstring[] = [on the next line) now makes the table unreadable, so the change waits. It used to be read as a one-line table. There is a test, and every registered table still reads (the backstop test).Fail-without, round 7 (measured): the new tests on round 6's module give test/merge-policy.test.ts 43 tests, 40 pass, 3 fail: the rename unit test (the three folders), the table test (the split declaration), and the symlink test.
At this head, eb37ee8 (measured):
npm run typecheckexit 0; check-denylist "1310 files clean"; DCO and the push gate clean; the CHANGELOG diff against main is additions only, andtest/changelog.test.tspasses 10/10;# tests 176 # pass 176 # fail 0;Round 6 (the round-5 review, CHANGES NEEDED)
Round 5's CI was green, and every round-4 attack was closed (measured by the reviewer). Round 6 closes what the review found open in round 5's new pieces. The tests were written first and run on round 5's module.
The rename exemption's real reach. A pure rename (the same blob and mode,
exactMoves) passes only when all of these hold:RUNNER_NAMES), with the same suffix:*.test.*/*.spec.*in JS or TS;test_*.py,*_test.py;*_test.go;*_spec.rb,*_test.rb,test_*.rb;Test*.java,*Test.java,*Tests.java,*TestCase.java, and Kotlin's;*Test(s).cs; XCTest:*Test(s).swift; bats:*.bats;test/,tests/,spec/,Tests/or__tests__/, or none for both;Anything else is a delete and an add, and waits. That covers:
tests/test_math.pytotests/math.pyortests/math_helpers.py;MathTest.javatoMathCase.java;spec/math_spec.rbtospec/math.rb;test/x.test.tstoscripts/x.test.ts,docs/x.test.ts,test/fixtures/x.test.tsortest/__snapshots__/x.test.ts;Renames that pass:
test/math.test.tstotest/sum.test.ts, a folder of tests (test/unit/totest/core/),test_sum.pytotest_total.py,MathTest.javatoSumTest.java,math_spec.rbtosum_spec.rb,math_test.gotosum_test.go, and a co-locatedweb/src/a.test.tsxtob.test.tsx.Tests: one case each, plus acceptance through real git (
scripts/,test/fixtures/and a name no runner selects, each waits).A guard, lint, allowlist or table file is never rename-exempt, even inside the test paths. These now wait:
tests/conftest.pytotests/old/conftest.py,test/tsconfig.jsonandtest/jest.config.jsintotest/old/,test/brand.test.ts(a lint test) renamed,test/isolate.tsrenamed, and a lint test with a registered table renamed.The table trick.
const,letorvarand any spacing (^(?:export\s+)?(?:const|let|var)\s+NAME, or a JSON key).var ALLOWED,const ALLOWEDorlet<tab>ALLOWED, and an entry added. So does each declaration change alone.var ALLOWEDwith two spaces is read, and an entry added to it istable-grown.table-grown/table-shrunk). It still waits. Two of my own test expectations changed for it: the one-line allowlist test, andsrc/delegation.tsNEVER_LOOSENin the guard-list test.Body: the Rule-10 line is corrected, this section states the rename rule's reach, and the Adversarial table has the round-5 review's attacks.
Fail-without, round 6 (measured): the new tests on round 5's module (this branch before the module changed) give test/merge-policy.test.ts 42 tests, 39 pass, 3 fail:
var, two spaces, tab);scripts/,test/fixtures/, an unrun name).At 84ebfa8 (measured):
# tests 175 # pass 175 # fail 0;npm run typecheckexit 0;My first push of this round was stopped by the push gate's denylist. A conftest fixture string held
\n@pytest.fixture, which reads as an email address. The fixture now uses apytest_configurehook instead, and the commit was amended before any push.CI: the head is 84ebfa8, and its CI was still running when this body was written. Round 5's 7c632eb was green (19 succeeded, 7 skipped, 1 duplicate
new-depscancelled; the reviewer measured it).150 PRs, round 6 (measured, the same method): decisions are identical to round 5's in all 150. 114 get a reason; 12 are newly held (#797, #790, #777, #664, #653, #627, #684, #656, #620, #594, #547, #503); 10 of 16 tests-only PRs are held. The slowest
weakeningOftook 37 ms, at load 2.Worst case, round 6 (measured on the final build,
weakeningReasonsalone, two runs each, load average 2.4):Round 5's figures at load 8–33 are below. These are lower mainly because the load is (inferred): round 6 adds no per-line work.
Round 5 (the round-4 review, CHANGES NEEDED)
The design is unchanged: plain lines, no parser. Each item below fails open no longer. The tests were written first and run on round 4's module (fail-without below).
CI is green again without weakening main's fix-forward test.
test/fix-forward-coordinator.test.ts's "rename" PR used to renametotalCentsintest/price.test.js's assertion lines, so under decision 1 it waited and never merged (CI run 39326108484, shard 2/4, measured by CI and the reviewer).src/price.js, and the test imports it under its old local name (import { applyDiscount, totalInCents as totalCents }), checking that the import line was there to change.test/price.test.jsis the same, and still checks 598 cents through the renamed function;/issue-(\d+)/). My scratch env's path heldissue-783, so it failed locally at "both PRs open" on main too. That was the "can't run it here" of round 1. Measured with a probe, and fixed by an env path withoutissue-.A pure rename doesn't wait (Merge: a PR that removes or weakens test assertions, lint rules or guards waits for the owner #783's deliverable, decision 1).
exactMoves(insrc/coordinator/change-files.ts, the changed-files helper) pairs each deleted path with an added path of the same blob id and mode (diff --raw, content match, no similarity guess).decideChange): a file renamed and a folder renamed don't wait; a rename with an expected value edited waits.A non-text line no longer hides a mark.
The table reader fails closed and doesn't read around tricks. The change waits when any of these is true:
A declaring line that closes every bracket it opens is a one-line table. Fixtures: the review's four tricks, and a one-line table.
Marks: the review's measured list, plain substrings:
t.Skipf(,t.SkipNow(;skipTest(,SkipTest,pytest.importorskip,pylint:disable,type:ignore;#[ignore;@Disabled,@Ignore;.skipIf(,.runIf(,.skip.;xcontext(; jasmine:pending(;XCTSkip;Skip =,Skip=,#pragma warning disable;@Suppress(;@phpstan-ignore,@psalm-suppress;xitandskip(each line is read with a space before it, so the start of a line counts).The long tail is a named gap. Each form has a fixture, plus negatives (
exit 0,skipped,skips,skipper).Assertion words:
Assert.,[Fact,[Theory,[Test;XCTAssert,XCTFail;must_,wont_,refute(assert_was already there);*.bats, now a test path):@test,runand[.Shell test files (
.sh) are no longer read as code, so every removed line of one waits, as an expected output's. Rust is narrowed to what's true:*.test.rsand test folders count, but inline#[cfg(test)]tests insrc/don't (a named gap, with a test).Test paths at any depth:
**/test/**,**/tests/**,**/spec/**and**/Tests/**(Swift, C#). Nestedfixtures/count as expected outputs. A test-file name pattern (*.test.*,test_*.py, ...) counts anywhere.Docs: a test file under
docs/is a test file, and its marks are read. Only a doc that isn't a test file is skipped.Timing. The bound is now checked first: the changed lines the rule would read are added up, and over 4,000,000 characters the change waits before any line is read. Each kind keeps only its first three places and a count, with no map per line. Measurements are in the bound table below.
Named (body, docs/design.md, CHANGELOG):
The Adversarial pass table has each of the review's attacks.
Also: main 71316eb (the v0.5.3 release) is merged in, not rebased. The CHANGELOG bullet moved out of the released v0.5.3 section into a new
## v0.5.4 (unreleased)section. One risk there, measured in memory:scripts/lib/changelog.mjswithSectionkeeps a section's hand-written top only above a generated list (### Features, ...). So the section ends with an empty### Guardrail changesheading, as #834 and #839 do (commit 7c632eb). Measured in memory: regenerating v0.5.4 keeps the bullet, and v0.5.3 is unchanged.test/changelog.test.ts: 10/10.What changed (round 4: the owner's line rule replaces the parser)
The owner decided (2026-10-19) to replace the 80 KB parser with a simple line rule, read from the diff, with no parser. Then the owner added that marker lines wait. Round 4 does both.
src/merge-policy-weakening.ts, rewritten. Before: 1,438 lines and 79,978 bytes of source. Now: 455 lines and 24,013 bytes, most of them registries.weakeningReasonsis pure, andweakeningOfreads the change. The rule, file by file:A test file (
TEST_PATHS, every repo, tests-only changes too):CHECK_LINE:assert,self.assert,expect,should,fail(,t.X,require.,pytest.,.toBe-style and chai.to.matchers,test(/it(/describe(/suite(and their.xforms,def test,class Test,func Test,@Test);TEST_PATHSgained**/__snapshots__/**,**/*.snapand**/testdata/**. Round 5 added test folders at any depth,Tests/and*.bats.An expected output (a snapshot, a file under
fixtures/,__snapshots__/ortestdata/, or any test file that isn't code): any removed or changed line waits. This closes round 3's hole 2(a).A guard or lint file (
GUARD_FILESandLINT_TESTS, the same file set as before): any removed or changed line waits.A registered table (
CHECK_TABLES, the same tables) is read as its lines, from its declaration to the line that closes it at its indent:never-shrink;package.jsonscriptsis one);never-grow).A table only after the change is new and passes. A table gone waits.
An allowlist file (
.denylist-allow,.gitleaksignore,.eslintignore,.semgrepignore): any added line waits.Any file but a doc (a test file under
docs/isn't a doc, since round 5): an added line holding a mark fromMARKERSwaits.MARKERSis the one list, plain substrings, owner 2026-10-19:.only(,test.only,it.only,describe.only,fit(,fdescribe(,.skip(,skip:,t.skip,test.skip,it.skip,xit(,xdescribe(,@pytest.mark.skip,@unittest.skip,t.Skip(,eslint-disable,eslint,@ts-ignore,@ts-expect-error,@ts-nocheck,NOLINT,noqa,type: ignore,#[allow(,#[expect(,swiftlint:disable,rubocop:disable,rubocop:todo,nolint,istanbul ignore,c8 ignore;xtest(,.todo(,todo:,@pytest.mark.xfail,t.SkipNow(,biome-ignore,oxlint-disable,tslint:disable,deno-lint-ignore,pyright: ignore,mypy: ignore,pylint: disable,#![allow(,v8 ignore,pragma: no cover,nosec,nosemgrep,NOSONAR,gitleaks:allow,@SuppressWarnings,phpcs:ignore,shellcheck disable,stylelint-disable,yamllint disable,hadolint ignore.Any other pure addition passes.
Bounded and fail closed.
MAX_CHANGED_CHARS(4,000,000) characters (since round 5, checked before any line is read);MAX_TABLE_FILE(500,000 bytes, not even read);Scope: the engine's own source paths stand aside in a managed app's repo, as the locks do (review.ts
lockedCategories, never named here).weakeningOfreads three things:changedNameswith--diff-filter=D);filesAt;changeFilesalready givesmergeDecisionFor.src/merge-policy.ts,src/coordinator/merge-flow.tsandfilesAtinsrc/coordinator/change-files.tsare unchanged since round 3.NOT_A_CHECK,NOT_A_LINT,checkShapedTablesand the source-listing pattern.NOT_A_CHECKandNOT_A_LINTare registerednever-growtables there.withSectionregeneration (measured in memory with scripts/lib/changelog.mjs).Owner decisions this PR follows
How it was verified
All runs used a scratch HOME, TMPDIR,
WORKLANE_STATE_DIRandWORKLANE_HELD_OUT_DIRunder the session scratchpad. No real state was touched. Times were taken on a shared machine under load (load average 22–54).Fail-without, round 5 (measured). I wrote the new tests first and ran them on round 4's module (this branch, before the module changed; no
git checkout <ref> -- <paths>).Fail-without, round 4 (measured; scratch worktree at f8a7010, test/merge-policy.test.ts):
src/merge-policy.tsandsrc/coordinator/merge-flow.tsas on main (no wiring):# tests 40 # pass 35 # fail 5. The five are the merge-decision test and four acceptance tests: a deleted assertion, an added.skip, an allowlist entry and a weakened guard. The no-over-hold acceptance test ("only adds tests") passes both ways.src/merge-policy-weakening.tsas at 3871c80 (the old parser):# tests 40 # pass 26 # fail 14. Every rule test fails: the line rule, the other languages, deleted files, expected outputs, guard lines, tables, marks, bounds andweakeningOf. So do the backstop (itstableLinesis new) and the four acceptance tests, on the new reasons' wording.Targeted, at the merged head (measured): merge-policy, change-files-renames, review, slot-path, brand and package-files give
# tests 161 # pass 161 # fail 0. tsc is clean (exit 0).The bound, round 5 (measured on the final build:
weakeningReasonsalone, one process each,--max-old-space-size=1024, two runs each, load average 8–18):CHECK_LINEword, removedskipin the near-miss text holds)MARKERS' escaped literals, with no quantifier.changeFiles(unchanged, before this check) took 0.7–1.0 s there (reviewer).The bound, round 4 (measured; kept for the record, and its worst case was wrong, see Corrections):
x . x . ...dot chain in a test body (one side 500,000)const a: ...with no=const a = function ...with no(const vK: < = K;and one assertion changedCHECK_LINEalternative, removed, at the capweakeningOf(git calls included) took at most 79 ms for a PR. The parser took at most 1,120 ms.Checks (measured): see "Local results and CI" below.
150-PR comparison, round 5 (measured: the same 150 PRs, the same method)
weakeningOf, git included)skipis in 26 PRs' added lines. Most are prose in strings and comments ("totals skip zero quantities"), plusconst skip = ...in slot dir: a test fails if engine code writes a slot-directory file in place (#616) #654 and Optional cross-family evaluator: a second verdict on protected changes; disagreement holds the PR #539. It added intake: a GitHub-sourced report is worked from its checked draft; one done_when block per body [PROTECTED: owner merges] #671, review.yaml: the tests' own machine, the slot-write check and the tool-guard denial tests wait for the owner [PROTECTED: owner merges] #681, push gate: refuse a commit whose line 2 isn't blank or whose subject is over 180 characters [PROTECTED: owner merges] #642, chat: a refused issue draft is asked for once more, with the reason, inside the answer's caps [PROTECTED: owner merges] #638, slot dir: a test fails if engine code writes a slot-directory file in place (#616) #654, Optional cross-family evaluator: a second verdict on protected changes; disagreement holds the PR #539 and Chat: model and effort picker limited to routing, per-conversation budget cap #514. All of them wait anyway for other reasons.pending(matched web: while paused, start actions say they take effect on resume, File it and Add ready say they happen now, and land approvals say they land now #664's ownpending(decision)helper in test/web-buttons.test.ts. That is the one newly held, a false hold.skiponconst skip = process.platform === ...). It still waits anyway for its locked paths, as before, and nothing else in it is held. The owner's "slot dir: a test fails if engine code writes a slot-directory file in place (#616) #654 doesn't wait" check no longer holds for the rule alone. Ifskipshould be RSpec-only (.rbfiles), that is a one-line narrowing. I left it as briefed: plain substrings, prefixed by a non-identifier character.pending(, false), first run: an open panel stays put across a poll that changes the detail lines above it #594 (skip:platform skip, false) and No late answer moves a control on Settings, Inbox or Decisions (#455) #503 (eslint-disable-next-line, true).skip:is in 57 PRs,skipin 26 andxit(in 25;eslint-disableis in 5 andshellcheck disablein 4;eslintand@ts-expect-errorare in 2 each;pending(,biome-ignore,istanbul ignore,todo:,nosecand.skip(are in 1 each.150-PR comparison, round 4 (measured: last 150 merged PRs to 128453a, each PR's own change, today's review.yaml)
The two newly held, beyond the parser's nine:
test/fixtures/plugins/escape/server.js): an expected-output change, by the rule.{ skip: ... }(platform skip): a mark, the accepted false hold.Holds more than the parser (16 PRs) come from two sources:
{ skip: process.platform ... }, and lines withprocess.exit(thatxit(matches. Fix: claude-failover 'back after the reset' deterministic (flake on main) #537 changed a test's title line.Holds less than the parser (9 PRs), all waiting anyway:
PACKAGE_BUDGETchanged under an unchangedassert.matchline. The parser followed the const.The marks (measured over the same 150, added lines outside docs):
skip:is in 57 PRs, almost all platform skips on new tests;xit(is in 25, almost allprocess.exit(;eslint-disableis in 5,shellcheck disablein 4, andeslintand@ts-expect-errorin 2 each;biome-ignore,istanbul ignore,todo:,nosecand.skip(are in 1 each.21 PRs are held only by a mark: 19 wait anyway, and 2 are newly held (first run: an open panel stays put across a poll that changes the detail lines above it #594, a false hold; No late answer moves a control on Settings, Inbox or Decisions (#455) #503,
eslint-disable-next-line, a true positive).main red: the #566 switch test stamps its pause on the test's clock, not the wall clock #684 and add-project-view test: the stand-in helper holds until released, so a slow runner can't see the run already failed #777: both still wait, by the owner's acceptance. main red: the #566 switch test stamps its pause on the test's clock, not the wall clock #684 changed
table(...)lines inside assertions. add-project-view test: the stand-in helper holds until released, so a slow runner can't see the run already failed #777 moved assertion lines into a helper, so they changed.Rule 10: main's fix-forward test (round 5)
One existing main test changed: test/fix-forward-coordinator.test.ts, its worker's rename step. It now leaves
test/price.test.js's assertion lines alone and imports the renamed function under its old local name. Why it isn't weaker is under Round 5, item 1. No other main test changed.git diff origin/main -- test/touches only that file and test/merge-policy.test.ts, where main's 22 tests are unchanged.Two of this PR's own round-4 tests changed fixtures:
.github/actions/setup/action.ymlinstead of.github/actions/test/action.yml. The old path is now also a nested test path, so it gets an extra fixture reason;Tests/, which is now a test path.Rule 10: every test the parser had, kept or not
All 36 weakening and acceptance tests were this PR's own (none on main). Each behaviour the line rule keeps has a test. Those it no longer has were the parser's, and a test can't assert them for a rule that doesn't have them.
Kept, rewritten for the line rule:
The "only adds tests" acceptance test no longer adds a test with
{ skip: ... }(now a mark), nor renames a test file (now a deleted path). The same fixtures are asserted the other way in the marks and deleted-file tests.Removed, with the reason each no longer applies:
asserton it.].slice(1)on a table's literal still waits.if (...) continue;is an addition (named gap).config.yamlandtests.yamlstill wait (guard test).MAX_CHANGED_CHARSandMAX_TABLE_FILEtests, and the weakeningOf test checks a table file over its cap isn't read.t.skip()never waits": the owner now holds every added mark.Named gaps (measured: the reviewers' fixtures fed to the new rule)
Each passes under the line rule. "Waits anyway" means it waits for the owner for another reason in this repo.
got = 3;inserted,let got = (add(1, 2), 3));if Add(1, 2) != 3(itst.Fatalline waits);eq(x, 3)changed or removed; the alias's own line waits).return;,if (x) return,process.exit(0);assert;import { skip as test };SKIP_SLOW = true);files.splice(0),offenders.length = 0, a new hook or helper that empties the list, or a copied Set grown.continue-on-error: true,test.excludein vite.config, anignorePatterns, a lint step added to.husky. CI workflows are locked in every repo and wait anyway.Makefile,justfile,phpstan.neonand.swiftlint.yml.pyproject.toml,setup.cfgandCargo.tomlwait anyway as dependency manifests..githooks/pre-push,.husky/pre-commitortest/run.shgoing 755 to 644 has no lines, so nothing is read. git skips a hook that isn't executable (inferred);vendor/suite) moved;#[cfg(test)]tests insrc/(a test asserts it passes);MARKERS(the long tail).beforeEach((t) => t.skip()); the outert.skip()from a new subtest; a newtest.only;NOLINTNEXTLINEandNOLINTBEGIN;#[expect(;swiftlint:disable;rubocop:todo; ESLint inline config to"warn"or["off"];Round 3's findings under the line rule (measured with the reviewer's fixtures)
test/fixtures/expected.json, the test-local const, and the alias madeassert.ok.gottoactual) both wait. They change assertion lines, which the owner accepted.return/throware gone: all six add-only fixtures pass.t.skip()from a new subtest waits (a mark). A new helper and its call line in aLINT_TESTSfile pass (a named gap).beforeEachskip,test.only, NOLINT forms,#[expect(,swiftlint:disable,rubocop:todoand ESLintwarn/array config;offenders.length = 0,files.splice(...)inserted, andMakefile,phpstan.neonand.swiftlint.yml.Package size (measured)
No raise.
PACKAGE_BUDGETstays main's ({ size: 1_950_000, unpackedSize: 6_060_000 }), and the PR fits it with room to spare.How it was measured:
npm ci,npm run build,npm run build:webandnpm pack --dry-run --json, in a fresh worktree of main (71316eb) and in this branch at 9096a3f (main merged in).Results:
Delta against main: +8,352 bytes tarball, +29,979 unpacked, from only these files:
dist/src/merge-policy-weakening.js(new): 26,964. The parser was 80,220.dist/src/coordinator/change-files.js: +2,618 (filesAtand round 5'sexactMoves).dist/src/coordinator/merge-flow.js: +259.dist/src/merge-policy.js: +138.Earlier deltas: round 4 was +6,050 / +25,320, and round 3 was +24,457 / +81,878.
No sourcemap,
.d.ts, test, fixture or stray file. The only test-like file in the pack,examples/basic/test/price.test.js, is in main's pack too.A measurement trap I hit (principle 5): a reused worktree's
dist/kept a stalemerge-policy-weakening.jsfrom an earlier build, so its "main" counted 80,220 bytes main doesn't have. The figures above come from a fresh worktree.Why 27 KB: most of it is the registries (
CHECK_TABLES,GUARD_FILES,LINT_TESTS,TEST_PATHS,MARKERS), the same file set as before, and their comments. The rule itself is about 120 lines.Local results and CI
npm run typecheckis clean (both projects, exit 0).node scripts/check-denylist.mjs: "1310 files clean".node scripts/check-dco.mjs origin/main HEAD: "all commits signed off".node scripts/push-gate-hook.mjs --range origin/main..HEAD: "9 commit(s) and 1 tip(s) clean".# tests 165 # pass 165 # fail 0.# tests 3856 # pass 3794 # fail 5. The 5 are twoisolate.testTMPDIR tests (Tests: temporary dirs live under the isolated test machine (TMPDIR in test/isolate.ts), so a run leaves nothing in /tmp #604) and three testrundb-startregressions (Docker). Round 1 measured the same files failing the same way with main's wiring under a scratch env. This PR touches none of them (git diff origin/main --stat), so they come from this machine's env (inferred).test/fix-forward-coordinator.test.tspasses in the full run.dist/once while it ran, to time a change that read the marks as one regex instead of a substring loop (same matches). Later files may have loaded either build. The targeted 165 ran on the final build.Corrections (principle 5)
Round 4's timing claims were false. "The worst case is under half a second", "464 ms (the worst found)" and "over the cap waits in 22 ms" (in the bound table, the bullets under it and "What this can never do") came from a shape that wasn't the worst.
Round 4's "covers Rust" was too wide: Rust's inline tests in
src/aren't a test path. Narrowed above.Round 4's "slot dir: a test fails if engine code writes a slot-directory file in place (#616) #654 passes" was true then. In round 5,
skipgives it a reason (above)."Every shape takes at most 12 parse steps per character", "such a file holds unread in well under a second" and docs/design.md's "under a second per worst-case file" were false. The step counter missed regex backtracking and the assertion pairing: the reviewer measured 26–39 s per parse, 132 s and over 200 s per change, and an out-of-memory crash under a 1 GB heap. The parser is gone. The line rule's measured worst case is 0.41 s at load 8–18 (above).
"
splice/filterare words now" was false.LINT_BYPASS's look-behind stopped them as method calls. That rule is gone too, and aspliceinserted in a lint test is a named gap.The "Package budget" section was stale after 3871c80 took main's 1,950,000 / 6,060,000. It is replaced below.
An unnamed loosened expectation: round 3 changed this PR's own test "every LINT_TESTS entry reads as a lint" to
lintShaped(text) || LISTS_SOURCES.test(text), and the body didn't say so. That test is now the backstop "every test that lists src/ or scripts/ files is registered", and it checks only that each entry is a real file.How four PRs were described. The body said Test: budget health reads the limited credential's reset time once #790, add-project-view test: the stand-in helper holds until released, so a slow runner can't see the run already failed #777, main red: the #566 switch test stamps its pause on the test's clock, not the wall clock #684 and dashboard-switch-speed: a false Board check says which part failed and what the page had #656 all "change existing assertions". As the reviewer read them:
Under the line rule all four still wait, because each changes lines with an assertion on them.
Overlaps (measured with
git merge-tree --write-treeat the merged head)## Unreleased (after v0.5.3)). Whichever lands second puts its bullet into the first one's section.git diff origin/main -- CHANGELOG.md): a## v0.5.4 (unreleased)section directly above## v0.5.3 (2026-10-19), with v0.5.3's sub-heading. Main's v0.5.3 text is untouched.CHANGELOG.md,docs/design.mdandsrc/coordinator/merge-flow.ts, with several open PRs.TRACED) needs one traced entry forreasons.push(...(i.weakening ?? []))inmergeDecision. Whichever of First run: a card follows the first task from File one to its PR; the Start card counts only steps before Start (#741) [PROTECTED: owner merges] #816 and Merge: a PR that removes or weakens a test, lint or guard waits for the owner (#783) [PROTECTED: owner merges] #820 lands second adds it.Adversarial pass (round 4: the line rule)
Each attack is a fixture fed to
weakeningReasons, or tomergeDecisionForon a scratch repo (rule 8).awaitdropped, assertion or test removed, tests only or with codet.Fatal/require., Rustassert_eq!, Java@Test, jest.toBe, chai.should!= 3changed; an alias call changedit./test./t.need a letter after the dottest.skip,.only,t.skip(),beforeEach(t => t.skip()),xit(,@pytest.mark.skip,t.Skip(addedreturn;inserted; a skip const flipped;import { skip as test }MARKERSentry inside an ordinary line, in a source, a test, a workflow, a Python file andCargo.tomlfixtures/*.json,.jsonl, a fixture server,testdata/*.golden,*.stdoutchanged].slice(1)on the literal, table renamed, apackage.jsonscript changedgit mvof a test file, or of a test folder with its fixtures, content unchanged (real git).spec.ts; out of the test pathseslint-disable-next-lineor@ts-nocheckbeside a NUL, a literal U+FFFD or a Latin-1 byte (real git for Latin-1)t.Skipf(,self.skipTest(,raise unittest.SkipTest,pytest.importorskip(,#[ignore],@Disabled,@Ignore,describe.skipIf(,describe.skip.each(,test.runIf(false),xcontext(,pending(),xit 'x' do,skip 'x',[Fact(Skip = ...)],#pragma warning disable,throw XCTSkip(,@Suppress(,@phpstan-ignore-next-line,@psalm-suppress,# pylint:disable=,# type:ignoretest.only(...)added todocs/examples/a.test.jsAssert.Equal/[Fact]/[Theory]removed; SwiftXCTAssertEqualremoved; minitestmust_equal/refute_nilremoved; bats[ ... ]and@testremoved[ ... ]check removed fromtest/cli.shpackages/x/test/math.jsassertion removed;packages/x/test/fixtures/expected.jsonvalue changed;packages/x/spec/math.jssrc/lib.rsinlineassert_eq!removedtests/test_math.pytotests/math.py/tests/math_helpers.py;MathTest.javatoMathCase.java;spec/math_spec.rbtospec/math.rb(out of the runner's naming)test/x.test.tstoscripts/x.test.ts(out of the test-folder root); todocs/ortest/fixtures/ortest/__snapshots__/scripts/andtest/fixtures/tests/conftest.pyintotests/old/;test/tsconfig.jsonandtest/jest.config.jsintotest/old/;test/brand.test.ts(lint test),test/isolate.ts(guard) or a registered lint test renamedtest/math.test.tstotest/sum.test.ts; a test folder moved;test_sum.pytotest_total.py;MathTest.javatoSumTest.java;math_spec.rbtosum_spec.rb;math_test.gotosum_test.go; a co-located*.test.tsxrenamedvar ALLOWED,const ALLOWEDorlet<tab>ALLOWED, and an entry addedconsttovar)var ALLOWED(two spaces) with an entry addedtable-grown(read)test/x.test.tstotest/node_modules/x.test.ts,test/.old/x.test.tsortest/__mocks__/x.test.tsconst ALLOWED:with itsstring[] = [on the next line, an entry addedtest/x.test.tstotest/.x.test.ts(a leading-dot name)const ALLOWED: string[] =with the list below, an entry added=(was: passes)const ALLOWED = // see [docs]with the list below, an entry addedconst ALLOWED = new Set()with.add(...)below, an entry addedALLOWED['test/c.test.ts'] = 1;added totest/no-clock-helpers.test.ts(L0)table-used(was: passes)Object.assign(ALLOWED, {...}),.push(...),new Set(Object.keys(ALLOWED)).add(...),const ALLOWED_TOO = { ...ALLOWED, ... }OUTSIDE_THE_HELPER.b = 2in change-files-renames.test.ts;ALLOWED.b = 2in no-wall-clock-durations.test.tsconst ALLOWED = // see [docs]with the list on the next line at the same indentfor (const c of ALWAYS_L3))eval('ALL' + 'OWED')['test/c.test.ts'] = 1;,new Function(...),Function('return this')()added in a registered table's filescript-shell=trueadded to.npmrcorweb/.npmrc, engine and app scope.npmrceval ('ALL' + 'OWED')['x'] = 1(a space before the parenthesis).yarnrc,.yarnrc.yml,bunfig.tomlor.bunfig.toml, engine and app scopeonlyBuiltDependenciesentry removed frompnpm-workspace.yamlpnpm-workspace.yaml; an added pnpm script setting thereWhat this can never do
mergeDecision(measured:mergeDecision(clean({ weakening: [] }))is unchanged). No existing reason, category, lock, level or limit is changed or removed. No check, test or lint on main was weakened or removed (rule 10): every removed test was this PR's own, listed above. Existing installs keep every hold they had (inferred from the diff: outside the new module, additions only).changedNamesandfilesAtrundiff --name-only,ls-treeandcat-file(read-only), with--literal-pathspecsand replace refs off.weakeningReasonsalone is 0.41 s at load 8–18, and 0.49–0.67 s at load 24–33 (2,000,000 one-character lines). See the bound table.Compromised component
src/merge-policy*,test/merge-policy.test.ts).test_correctand the owner's review of L3 paths still stand.mergeDecisionFor, as it could skip every other reason. Unchanged by this PR.foreignPush), and the hold runs on the pushed head too.New dependencies
None.