RUE-1406: consume prebuilt programs in the CLI corpus (ADR-0070 P2) - #2318
Merged
Conversation
The 64 CLI cases naming a checked-in root each compiled that root inside their own test execution, which never reaches the action cache. The nine roots are `rue_program` build actions now — eight new, plus Phase 1's `examples/meridian/main.rue`, so one artifact serves both the slow-tier large-example scenarios and the six CLI ones — collected by a new `rue_program_staging` rule into `//:cli-staged-programs` and declared by //:cli-tests, //:cli-tests-slow and the four shards through the same `$(location ...)` env contract every other corpus input already uses. The cases keep their TOML form, their corpus actions, their names, labels and tiers; RUE-924's corpus-omission audit is untouched. What changes is that the harness runs the staged executable instead of compiling. It decides structurally rather than from a list: `case_runs_prebuilt_program` destructures `Case` exhaustively and stages only a case that leaves the compile exactly as the `rue_program` action performed it, so a new case field is a compile error rather than a silent assumption. Everything past that point — working directory, argv, stdin, environment, timeouts, expectations — is one shared path, `run_case_program`, which is what keeps the two modes from drifting. With RUE_CLI_STAGED_PROGRAMS unset, as at the `//crates/rue-cli-tests:cli` developer entry point, every case compiles. Staying compile-in-harness, all deliberate: the six cross-target cli-test-fixtures cases, the repo-relative `source_path` fixture whose subject is the TOML mechanism itself, the one differential_opt calculator case, the one-scenario wordfreq root, and the RUE-48 automatic example smokes. //:release-smoke does not carry the staged directory either — it runs only the differential_opt filter, so it would pay nine release-configured compiles for cases that can never consume them. `cli.examples_meridian` is no longer skipped. RUE-1083 disabled it because each of its six cases paid a full 80.7s compile of the same 36k-line root; they share one cached artifact now, so the corpus pays six runtime scenarios. That coverage was running nowhere. Phase 2 also repeats Phase 1's positive warm-cache check for CLI roots: check-rue-program-warm-cache.sh cold-builds the staged programs alongside the canaries and asserts every scan/derive/compile action is cache-served when the staged directory is rebuilt from a relocated checkout root. The checked shard weights still carry the old compile-dominated costs for these cases; they rebalance from the next run's measured timings.
The warm-cache control asserts its first build executes rue_scan, rue_derive_manifest and rue_compile cold. It could not: the probe nonce reaches Rust actions as an otherwise-unused --cfg, which re-keys the Rust actions but leaves the compiler's emitted bytes identical, and these actions are keyed on the compiler artifact. Every canary action was served from the shared cache and the job failed its own cold-namespace assertion the first time it ran, on trunk as well as here. Carry the nonce into the three actions directly. With no nonce set, _probe_env returns its argument untouched and action keys are unchanged -- verified by rebuilding //:jsonfmt across the change as a no-op; a fresh nonce re-executes all three actions locally, and a repeated nonce does not. Also restore the job's 60-minute timeout. The nine staged CLI programs add about 78 seconds of wall time to the cold half, meridian's ~30-40s among it, so the existing headroom already covers them.
A case whose prebuilt executable was missing fell back to compiling its own root, silently. Absence had to be tolerated, because it is also the deliberate exclusion for roots no rue_program owns -- so a renamed root, an edited source_path, or a case that grew a compile-touching field would have left its cases compiling again with nothing red, which is the coverage-shaped failure RUE-924 describes. Have the staging directory declare its inventory in staged-roots.txt, and validate it against the complete unfiltered corpus before libtest2 applies filters, the way the contract metadata is already validated. A root listed in the manifest must exist and must have at least one case that consumes it; the reverse direction stays legal, so the source_path fixture and the wordfreq root keep compiling in the harness. The run reports how many cases take the staged path, so a lane that loses staging is visible in its log rather than only in its wall clock.
The six restored meridian scenarios had no weights at all -- they had never run -- so shard balance and the derived corpus deadlines were sizing them from default_ms. Regenerated platforms["linux-x64"] from the four cli-shard case-timings artifacts of this branch's CI run, which is the first run whose inventory includes them: 11-31ms each, well under the 1059ms placeholder, so the placeholder was conservative rather than dangerous. The same data measures this phase: 36 cases drop by more than three seconds and a linux-x64 corpus pass sheds 1051s, with the lattice scenarios going from about 79s each to hundredths of a second. The automatic RUE-48 smokes still compile their roots and still cost what they cost. common, default_ms, and the linux-arm64 and macos maps are untouched; those platforms keep falling back to common until a run on each supplies timings.
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Aug 12, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes RUE-1406. Part of ADR-0070 (
docs/designs/0070-rue-program-build-actions.md), following RUE-1405.What this does
The 64 CLI cases naming a checked-in root each compiled that root inside their own test execution, which never reaches the action cache. Those roots are
rue_programbuild actions now, and the harness runs the prebuilt executable a case names.rue_programtargets for the CLI-named roots: harbor, jsonfmt, lattice, mosaic, rill, ruelex, first/stats, second/calculator.examples/meridian/main.rueis the ninth and is already a Phase 1 large-example program, so one artifact serves both the slow-tier scenarios and the six CLI ones — "many scenarios, one compile" reaching across two suites.examples/first/holds three sibling roots, sofirst-statsnames its one file instead of globbing the directory; every other root owns its directory and takes the directory-bounded glob.rue_program_stagingrule collecting them into//:cli-staged-programs, a symlinked directory keyed by root path, so the harness's lookup key is the case's ownsource_pathstring with no name mangling on either side.//:cli-tests,//:cli-tests-slowand the four shards declare it through the sameattrs.arg()/$(location ...)contract every other corpus input already uses.rue-cli-tests. Cases keep their TOML form, their corpus actions, their names, labels and tiers.cli.examples_meridianis no longer skipped. RUE-1083 disabled its six scenarios because each paid a full 80.7s compile of the same 36k-line root; they share one cached artifact now, so the corpus pays six runtime scenarios. That coverage was running nowhere.No sharding change, no TOML migration, no decision on the per-file corpus redesign (RUE-1408). RUE-924's corpus-omission audit is untouched — target names, labels and tiers all carry over.
How the two modes stay honest
case_runs_prebuilt_programdestructuresCaseexhaustively and stages a case only when it leaves the compile exactly as therue_programaction performed it — no argument override, no compiler environment beyond the standard library the program already declares, no runner-driven-O, and no assertion about a compile the harness does not run. Because the pattern names every field, a newCasefield is a compile error here rather than a silent assumption that the new knob does not affect the compile.Past that decision point the two modes are one shared path,
run_case_program: same working directory,argv[0], stdin, environment, timeouts and expectations. ADR-0070's Consequences section names drift between the compile path and the staged path as this phase's cost; the exhaustive pattern and the shared runner are the two things holding it.With
RUE_CLI_STAGED_PROGRAMSunset — the//crates/rue-cli-tests:clideveloper entry point, or any hand-run invocation — every case compiles exactly as before.What deliberately keeps compiling in the harness
The 6 cross-target
cli-test-fixturescases (one consumer per(root, target)tuple), the repo-relativesource_pathfixture case (its subject is the TOML resolution mechanism), the onedifferential_optcalculator case (four compiles by design), the one-scenario wordfreq root, the ~4,050 inline-source cases, and the RUE-48 automatic example smokes.//:release-smokedoes not carry the staged directory either: it runs only thedifferential_optfilter, so declaring it there would add nine release-configured program compiles to a deliberately bounded lane for cases that can never consume them.Positive warm-cache control
scripts/check-rue-program-warm-cache.shnow covers both consumer shapes ADR-0070 has. It cold-builds the canaries and the nine staged programs under a fresh nonce, then from a relocated checkout root runs the canaries' four scenarios and rebuilds//:cli-staged-programs, failing unless everyrue_scan/rue_derive_manifest/rue_compileaction was served by the remote cache.The nonce did not create a cold namespace, and this job had never run.
rue-program-warmlanded with Phase 1 and its first-ever execution was dispatched from this branch (run 31615598448) — it failed in 2.6 minutes, and trunk failed the same way in 1.6 (run 31615608639):RUE_CACHE_PROBE_NONCEreaches Rust actions as an otherwise-unused--cfg(toolchains/rust/BUCK, RUE-1034). That re-keys the Rust actions, but an unused cfg does not change the compiler's emitted bytes, andrue_scan/rue_derive_manifest/rue_compileare keyed on the compiler artifact — so every canary action was served from the shared cache. It only looked correct in Phase 1 because nothing had populated those artifacts yet. The nine new programs here were genuinely cold for the same reason: nothing had ever built them.rue_rules.bzlnow carries the nonce into those three actions directly. Verified locally on//:jsonfmt:rue_*actions execute locally, coldThe timeout stays at 60. The earlier 60 → 90 raise is reverted. Run 31615598448 measured the nine staged programs cold at ~78s of wall time (whole cold build 2m24s including the compiler; meridian, the long pole, ~30-40s) — the existing headroom already covers it. Two caveats worth stating: that run never achieved a cold namespace for the canaries, so it under-measures a fully cold build; and meridian at ~30-40s is well under the 80.7s this description quotes elsewhere, which is compiler performance work landing since those numbers were recorded, not a change from this PR.
Verification
The container this was authored in has no working buck2 (
dotslashis unavailable and github.com is blocked), so the suite was not run locally — CI is the authority here. What was checked:Casewas confirmed to produceE0027, so the drift guard fires as intended.rustfmt --checkclean oncrates/rue-cli-tests/src/main.rswith the repository config.bash -non the warm-cache script; syntax-parsedBUCK,rue_rules.bzlandcorpus.bzl.rue_program's scan can read outside its declaredsrcs.Staging is asserted, not assumed
staged_programreturnedNonewhen a case's executable was absent and the case then compiled its own root with nothing reported. Absence had to be tolerated, because it is also the deliberate exclusion for roots norue_programowns — so a renamed root, an editedsource_path, or a case that grew a compile-touching field would have put its cases back to compiling with nothing red. That is the coverage-shaped failure RUE-924 describes, in a PR that cites RUE-924.rue_program_stagingnow writesstaged-roots.txtinto the directory, and the harness validates it against the complete unfiltered corpus beforelibtest2applies filters — the same wayvalidate_contract_metadataalready runs, so a shard checks the whole graph rather than its own slice. A listed root must exist and must have at least one case consuming it. The reverse stays legal: an eligible case whose root is absent from the manifest compiles, which is what the repo-relativesource_pathfixture and the one-scenario wordfreq root need.Verified against the real staging directory and the full 1,850-case inventory:
cli-tests: 64 case(s) run prebuilt rue_program executables— the 64 this description claims, now asserted at runtimestaged program(s) no case consumes: examples/ghost/main.ruestaged program '…' is listed in staged-roots.txt but absent from '…'RUE_CLI_STAGED_PROGRAMSat a directory with no manifestits staged-roots.txt is unreadablecli.examples_mosaic::mosaic_prints_referencepasses both ways — 0.18s staged against 2.09s compiling.Shard weights, refreshed from measurement
The six restored meridian scenarios had no weights at all — they had never run — so shard balance and the derived corpus deadlines sized them from
default_ms(1059 ms). This branch's CI run is the first whose inventory includes them, so its fourcli-shard-*-case-timingsartifacts were fed back throughscripts/generate-cli-shard-weights.py:Meridian measures 11-31 ms per scenario, so the placeholder was conservative rather than dangerous — the risk was over-weighting, not a missed deadline.
The same data measures this phase on linux-x64:
cli.examples_lattice::lattice_prints_referencecli.examples_lattice::lattice_runs_cross_oracle_selftestcli.examples_harbor::harbor_validates_and_simulates_scenariocli.examples::lattice::main(RUE-48 smoke, still compiles)36 cases drop by more than three seconds, and a linux-x64 corpus pass sheds 1,051 seconds.
common,default_ms, and thelinux-arm64andmacosmaps are untouched — the generator replaces a whole map per platform, and required CI only shards the CLI corpus on linux-x64. Those platforms keep falling back tocommon(platforms[p]→common→default_ms) until a run on each supplies timings, which is the remaining half of this follow-up.Known follow-up
linux-arm64andmacosshard weights still carry the old compile-dominated costs, because required CI only shards the CLI corpus on linux-x64 and the generator replaces one platform map at a time. Both fall back tocommon, so nothing breaks; refreshing them needs a full corpus pass on each platform.