Skip to content

Commit a603424

Browse files
committed
ci: run every local-gate drift check in CI, and guard the two lists against drifting again
`npm run test:ci` (the documented local gate) and ci.yml (what actually blocks a merge) are two independently hand-maintained lists, and they had drifted: 22 of the 42 drift checks reachable from test:ci ran in no workflow at all. On a repo whose gate auto-merges on green CI, such a check is a convention rather than a gate, and worse, its presence reads as coverage. Three of the missing ones make that concrete: checkers-wired:check is the #9860 meta-guard that detects unwired checkers and was itself unwired; db:migrations:immutable:check guards the invariant whose violation breaks boot for every upgraded Orb; test-wiring and typecheck-coverage exist to stop coverage silently narrowing. Wired as ONE ungated block rather than 19 individually gated steps. Per-check path gating is where the fidelity bugs actually live -- the sibling PR had to add an entire `contract` filter because nothing here watched packages/loopover-contract/** -- and 19 hand-chosen gates would be exactly the hand-maintained list #9860 rejects. These are sub-second file-reading scripts, so running them unconditionally costs less than choosing 19 gates correctly. Three checks are deliberately NOT in that block. They need git history that validate-code's checkout does not have, and measuring them there showed they degrade in two different and equally bad ways: db:migrations:immutable:check exits 1 with "no orb-v* tags visible", so it would fail every PR, while release-commit-parsing:check returns [] and passes, and releasable-commit-types:check prints "cannot resolve; skipping" and exits 0. The last two would have been permanently green while verifying nothing -- the very failure this change exists to remove, reproduced at the CI layer and then read as coverage. They run in a new drift-checks-history job with fetch-depth: 0 and tags, which asserts up front that the history is actually present rather than trusting it. The aggregator gains a dependency on that job, since a job nothing depends on cannot fail a PR. check-ci-drift-checks-wired.ts is the guard that stops this reopening. It is the sibling of check-checkers-wired.ts on the other axis: that one asks whether a checker runs anywhere (test:ci counts), this one asks whether CI runs it, which is the question that decides whether a merge is gated. Both sides are computed, never listed, so the roster cannot drift the way ci.yml drifted from test:ci in the first place. Includes the contract api-schemas wiring from the sibling branch, because this guard cannot pass without it. Closes #10269
1 parent e1994c3 commit a603424

5 files changed

Lines changed: 418 additions & 4 deletions

File tree

.github/workflows/ci.yml

Lines changed: 130 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ jobs:
4343
mcp: ${{ steps.filter.outputs.mcp }}
4444
mcpCliHarness: ${{ steps.filter.outputs.mcpCliHarness }}
4545
engine: ${{ steps.filter.outputs.engine }}
46+
contract: ${{ steps.filter.outputs.contract }}
4647
discoveryIndex: ${{ steps.filter.outputs.discoveryIndex }}
4748
miner: ${{ steps.filter.outputs.miner }}
4849
minerTestHarness: ${{ steps.filter.outputs.minerTestHarness }}
@@ -188,6 +189,10 @@ jobs:
188189
engine:
189190
- 'packages/loopover-engine/**'
190191
- 'package-lock.json'
192+
contract:
193+
- 'packages/loopover-contract/**'
194+
- 'scripts/gen-contract-api-schemas.ts'
195+
- 'package-lock.json'
191196
miner:
192197
- 'packages/loopover-miner/**'
193198
- 'scripts/check-miner-package.ts'
@@ -478,6 +483,55 @@ jobs:
478483
env:
479484
NODE_OPTIONS: ""
480485
run: node --experimental-strip-types scripts/validate-observability-configs.ts
486+
# ── DRIFT CHECKS THAT CI PREVIOUSLY DID NOT RUN (#10269) ────────────────────────────────────────
487+
# `npm run test:ci` (the local gate) and this file are two independently hand-maintained lists, and
488+
# they had drifted: 22 of the 42 checks reachable from `test:ci` ran in NO workflow at all. They were
489+
# therefore enforced only when a human happened to run the full local aggregate -- which on a repo
490+
# whose gate auto-merges on green CI makes them a convention, not a gate. #10237 is the same class
491+
# having already fired twice on one artifact, found both times by a human and never by CI.
492+
#
493+
# Deliberately ONE ungated block rather than 19 individually-gated steps. Per-check path gating is
494+
# where the fidelity bugs actually live -- #10268 had to add a whole `contract` filter because
495+
# nothing here watched packages/loopover-contract/** -- and 19 hand-chosen gates would be exactly the
496+
# hand-maintained list #9860 exists to reject. These are all sub-second file-reading scripts, so
497+
# running them unconditionally costs far less than choosing 19 gates correctly.
498+
#
499+
# Ordering: this sits after "Build contract package" because the two control-plane generators import
500+
# @loopover/contract/control-plane. Everything else here reads source, package.json or workflow files.
501+
#
502+
# NOT here, on purpose: mcp:tool-reference:check resolves @loopover/engine through node_modules to
503+
# its dist/, so it cannot run before "Build engine package" -- it has its own step below, carrying
504+
# that build's condition rather than a gate chosen from its own inputs. It is the ONLY check in this
505+
# set with a build prerequisite; verified by deleting packages/loopover-engine/dist and re-running
506+
# all of them, which is also how CI caught it here rather than anyone predicting it.
507+
#
508+
# NOT here either: db:migrations:immutable:check, release-commit-parsing:check and
509+
# releasable-commit-types:check all need git history this job does not have (its checkout is shallow
510+
# by design). They run in `drift-checks-history` below -- see that job for why putting them here
511+
# would have been worse than leaving them unwired.
512+
- name: Drift checks (unconditional)
513+
run: |
514+
set -euo pipefail
515+
npm run turbo-inputs:check
516+
npm run workspace-dep-ranges:check
517+
npm run control-plane:contract:check
518+
npm run control-plane:openapi:check
519+
npm run coverage-boltons:check
520+
npm run ui-derived-types:check
521+
npm run server-manifest:check
522+
npm run dead-exports:check
523+
npm run publishable-deps:check
524+
npm run fixture-clock-races:check
525+
npm run typecheck-coverage:check
526+
npm run test-wiring:check
527+
npm run checkers-wired:check
528+
npm run focus-manifest-fields:check
529+
npm run maintainer-associations:check
530+
npm run release-linked-versions:check
531+
npm run mcp:client-config:check
532+
npm run record-mapper-fields:check
533+
npm run dispatch-provenance:check
534+
npm run ci-drift-checks-wired:check
481535
# Runs ahead of Typecheck AND "Test with coverage" (#ci-engine-build-order): src/mcp/find-opportunities.ts
482536
# (root backend, since #2281/#3985) imports packages/loopover-miner/lib/opportunity-fanout.js --
483537
# a .js-suffixed specifier esbuild/Vite resolve straight to the real (gitignored, uncompiled)
@@ -545,6 +599,16 @@ jobs:
545599
- name: Build engine package
546600
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.miner == 'true' }}
547601
run: npx turbo run build --filter=@loopover/engine
602+
# The one drift check with a build prerequisite (#10269): it resolves a bare `@loopover/engine`
603+
# specifier through node_modules to dist/, so it must follow the build above and carries that build's
604+
# condition VERBATIM. The gate is therefore derived ("run iff the thing I need was built"), not
605+
# hand-picked from this check's own inputs -- which is the distinction that keeps it out of the
606+
# hand-maintained-gate class the unconditional block above exists to avoid. If the two conditions ever
607+
# diverge this check silently stops running, so they must move together, exactly as the Typecheck step
608+
# and this build already must.
609+
- name: MCP tool-reference drift check
610+
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.miner == 'true' }}
611+
run: npm run mcp:tool-reference:check
548612
# Mirrors "MCP package check"/"Miner package check" below: the published npm tarball is a
549613
# different surface than the workspace build above (files field, forbidden paths/content,
550614
# stale README wording) and needs its own dry-run validation (#8591).
@@ -822,6 +886,19 @@ jobs:
822886
- name: OpenAPI drift check
823887
if: ${{ github.event_name == 'push' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.uiContract == 'true' }}
824888
run: npm run ui:openapi:check
889+
# Same generated-artifact-drift class as the OpenAPI check above, and it was enforced ONLY by the
890+
# local `npm run test:ci` aggregate -- exactly the local-only gap called out on the selfhost/miner
891+
# env-reference steps in validate-code, which each went stale with zero CI signal until someone
892+
# happened to run the full local script. It has now bitten this file twice (#10237): once when the
893+
# discovery-route schemas were added without teaching the generator, leaving `main` red for every PR
894+
# that ran the local gate, and again when #10160 added `linkedIssueMaintainerExempt` without
895+
# regenerating. Neither surfaced in CI.
896+
# Its inputs are src/openapi/schemas.ts (`backend`), apps/loopover-ui/public/openapi.json (`ui`), and
897+
# the contract package's own modules, which it scans to emit imports -- hence the third filter, added
898+
# with this step because nothing here previously watched packages/loopover-contract/** at all.
899+
- name: Contract api-schemas drift check
900+
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.contract == 'true' }}
901+
run: npm run contract:api-schemas:check
825902
# Checks apps/loopover-ui/src' known-latest MCP version string against the published package, so
826903
# its dependency is `ui` (the file it scans) + `mcp` (the package it checks against) -- NOT the
827904
# OpenAPI contract, which this script never reads.
@@ -956,6 +1033,58 @@ jobs:
9561033
# never serializes with the fast drift/typecheck/build checks -- that split remains). It ran 2023-style
9571034
# as a 3-shard matrix for wall-clock; unsharded again 2026-07-24 (see the job's own header comment) to
9581035
# trade PR latency for 2 fewer runners per PR and the simpler, flake-free single-report pipeline.
1036+
# The three drift checks that need real git history (#10269). They are a SEPARATE job purely because
1037+
# validate-code's checkout is shallow on purpose ("this job no longer uploads to Codecov ... so it has no
1038+
# reason to fetch full history anymore"), and each of these three degrades differently and silently there:
1039+
#
1040+
# - db:migrations:immutable:check exits 1 with "no orb-v* tags visible", so it would fail EVERY PR.
1041+
# - release-commit-parsing:check returns [] and PASSES when it cannot resolve the base.
1042+
# - releasable-commit-types:check prints "cannot resolve ...; skipping" and exits 0.
1043+
#
1044+
# The last two are why this is a job and not a step in validate-code: wired there they would be
1045+
# permanently green while verifying nothing, which is the "guards nothing while looking like a guard"
1046+
# failure (#9860) that #10269 exists to fix. Reproducing that at the CI layer while claiming to fix it
1047+
# would be worse than leaving them unwired, because the green would then be read as coverage.
1048+
drift-checks-history:
1049+
name: drift-checks-history
1050+
needs: changes
1051+
if: ${{ github.event_name == 'push' || github.event.pull_request.draft != true }}
1052+
runs-on: ubuntu-latest
1053+
timeout-minutes: 10
1054+
steps:
1055+
# fetch-depth: 0 + tags is the entire point of this job -- the immutability check compares migration
1056+
# blobs against every orb-v* tag, and both commit checks diff against origin/<base>.
1057+
- name: Checkout
1058+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
1059+
with:
1060+
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
1061+
fetch-depth: 0
1062+
fetch-tags: true
1063+
- name: Setup workspace
1064+
uses: ./.github/actions/setup-workspace
1065+
# A PR checkout leaves no ref for the base branch, and both commit checks diff against
1066+
# `origin/<base>`. Fetch it explicitly rather than assuming fetch-depth: 0 created the remote ref.
1067+
- name: Fetch base ref
1068+
if: ${{ github.event_name == 'pull_request' }}
1069+
run: git fetch --no-tags --quiet origin "+refs/heads/${{ github.base_ref }}:refs/remotes/origin/${{ github.base_ref }}"
1070+
# Fails loudly if the history this job exists to provide is somehow absent, rather than letting the
1071+
# two skip-on-error checks below report a green that means nothing.
1072+
- name: Assert history is present
1073+
run: |
1074+
set -euo pipefail
1075+
tags="$(git tag -l 'orb-v*' | wc -l | tr -d ' ')"
1076+
if [ "$tags" -eq 0 ]; then
1077+
echo "::error::No orb-v* tags after fetch-tags -- the history checks below would be vacuous."
1078+
exit 1
1079+
fi
1080+
echo "orb-v* tags visible: $tags"
1081+
- name: Released-migration immutability check
1082+
run: npm run db:migrations:immutable:check
1083+
- name: Release commit-parsing check
1084+
run: npm run release-commit-parsing:check
1085+
- name: Releasable commit-types check
1086+
run: npm run releasable-commit-types:check
1087+
9591088
validate-tests:
9601089
name: validate-tests
9611090
needs: changes
@@ -1283,7 +1412,7 @@ jobs:
12831412
# Path-filtered jobs report "skipped", which is treated as success.
12841413
validate:
12851414
name: validate
1286-
needs: [changes, validate-code, validate-tests]
1415+
needs: [changes, validate-code, validate-tests, drift-checks-history]
12871416
if: ${{ always() }}
12881417
# Pure result-aggregation (reads needs.*.result, echoes pass/fail) -- no build/test work, so it never
12891418
# needed the self-hosted pool's cached toolchain (#2507).

package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@
9797
"release-commit-parsing:check": "tsx scripts/check-release-commit-parsing.ts",
9898
"releasable-commit-types:check": "tsx scripts/check-releasable-commit-types.ts",
9999
"checkers-wired:check": "node --experimental-strip-types scripts/check-checkers-wired.ts",
100+
"ci-drift-checks-wired:check": "node --experimental-strip-types scripts/check-ci-drift-checks-wired.ts",
100101
"maintainer-associations:check": "node --experimental-strip-types scripts/check-maintainer-association-copies.ts",
101102
"regate-sort-key:check": "node --experimental-strip-types scripts/check-regate-sort-key.ts",
102103
"command-redelivery-guards:check": "node --experimental-strip-types scripts/check-command-redelivery-guards.ts",
@@ -152,7 +153,7 @@
152153
"test:smoke:browser:install": "playwright install chromium",
153154
"test:smoke:browser": "node --experimental-strip-types scripts/smoke-ui-browser.ts",
154155
"pretest:ci": "npm run check-node-version",
155-
"test:ci": "git diff --check && npm run actionlint && npm run lint:composite-actions && npm run db:migrations:check && npm run db:migrations:immutable:check && npm run turbo-inputs:check && npm run workspace-dep-ranges:check && npm run db:schema-drift:check && npm run selfhost:env-reference:check && npm run miner:env-reference:check && npm run selfhost:validate-observability && npm run cf-typegen:check && npm run build --workspace @loopover/engine && npm run build --workspace @loopover/discovery-index && npm run build:mcp && npm run build:miner && npm run build --workspace @loopover/ui-kit && npm run typecheck && npm run test:coverage && npm run test:engine-parity && npm run test:live-gate-parity && npm run test:driver-parity && npm run validate:mcp && npm run test --workspace @loopover/engine && npm run test:workers && npm run test:mcp-pack && npm run test:contract-pack && npm run test:miner-pack && npm run test:engine-pack && npm run test:ui-kit-pack && npm run test:miner-deployment-docs-audit && npm run rees:test && npm run ui:openapi:check && npm run control-plane:contract:check && npm run control-plane:openapi:check && npm run ui:version-audit && npm run docs:drift-check && npm run coverage-boltons:check && npm run import-specifiers:check && npm run ui-derived-types:check && npm run server-manifest:check && npm run dead-source-files:check && npm run dead-exports:check && npm run publishable-deps:check && npm run fixture-clock-races:check && npm run typecheck-coverage:check && npm run test-wiring:check && npm run checkers-wired:check && npm run focus-manifest-fields:check && npm run release-commit-parsing:check && npm run releasable-commit-types:check && npm run regate-sort-key:check && npm run maintainer-associations:check && npm run command-redelivery-guards:check && npm run dispatch-gate-reasons:check && npm run verdict-backoff-chokepoint:check && npm run record-mapper-fields:check && npm run dispatch-provenance:check && npm run validate:no-hand-written-js && npm run replay-runner-manifest:check && npm run coco-dev-versions:check && npm run branding-drift:check && npm run manifest:drift-check && npm run engine-parity:drift-check && npm run engines-nvmrc:check && npm run release-manifest:sync:check && npm run release-linked-versions:check && npm run command-reference:check && npm run mcp:tool-reference:check && npm run mcp:client-config:check && npm run contract:api-schemas:check && npm run ui:lint && npm run ui:typecheck && npm run ui:test && npm run ui:build",
156+
"test:ci": "git diff --check && npm run actionlint && npm run lint:composite-actions && npm run db:migrations:check && npm run db:migrations:immutable:check && npm run turbo-inputs:check && npm run workspace-dep-ranges:check && npm run db:schema-drift:check && npm run selfhost:env-reference:check && npm run miner:env-reference:check && npm run selfhost:validate-observability && npm run cf-typegen:check && npm run build --workspace @loopover/engine && npm run build --workspace @loopover/discovery-index && npm run build:mcp && npm run build:miner && npm run build --workspace @loopover/ui-kit && npm run typecheck && npm run test:coverage && npm run test:engine-parity && npm run test:live-gate-parity && npm run test:driver-parity && npm run validate:mcp && npm run test --workspace @loopover/engine && npm run test:workers && npm run test:mcp-pack && npm run test:contract-pack && npm run test:miner-pack && npm run test:engine-pack && npm run test:ui-kit-pack && npm run test:miner-deployment-docs-audit && npm run rees:test && npm run ui:openapi:check && npm run control-plane:contract:check && npm run control-plane:openapi:check && npm run ui:version-audit && npm run docs:drift-check && npm run coverage-boltons:check && npm run import-specifiers:check && npm run ui-derived-types:check && npm run server-manifest:check && npm run dead-source-files:check && npm run dead-exports:check && npm run publishable-deps:check && npm run fixture-clock-races:check && npm run typecheck-coverage:check && npm run test-wiring:check && npm run checkers-wired:check && npm run ci-drift-checks-wired:check && npm run focus-manifest-fields:check && npm run release-commit-parsing:check && npm run releasable-commit-types:check && npm run regate-sort-key:check && npm run maintainer-associations:check && npm run command-redelivery-guards:check && npm run dispatch-gate-reasons:check && npm run verdict-backoff-chokepoint:check && npm run record-mapper-fields:check && npm run dispatch-provenance:check && npm run validate:no-hand-written-js && npm run replay-runner-manifest:check && npm run coco-dev-versions:check && npm run branding-drift:check && npm run manifest:drift-check && npm run engine-parity:drift-check && npm run engines-nvmrc:check && npm run release-manifest:sync:check && npm run release-linked-versions:check && npm run command-reference:check && npm run mcp:tool-reference:check && npm run mcp:client-config:check && npm run contract:api-schemas:check && npm run ui:lint && npm run ui:typecheck && npm run ui:test && npm run ui:build",
156157
"test:release": "npm run test:ci && npm run changelog:check",
157158
"test:release:mcp": "npm run test:ci",
158159
"test:watch": "vitest",

0 commit comments

Comments
 (0)