Skip to content

ci: let the merge queue take a pull request run of the same tree (needs opt-in) - #255

Merged
maralcbr merged 4 commits into
integration/aurorafrom
ci/merge-queue-reuse
Oct 29, 2026
Merged

maralcbr merged 4 commits into
integration/aurorafrom
ci/merge-queue-reuse

Conversation

@maralcbr

@maralcbr maralcbr commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Policy change. It changes what ci / result attests in the merge queue for some pull requests, and it reverses the README's rule that no pull request run is a source of results. Merged by a maintainer after the end-to-end test below; @chriskearney FYI, and a revert restores full queue builds. Separate from #254 (the safe speed-ups), which it doesn't depend on.

Merge precondition (met 2026-10-28): a scratch-repo probe of GitHub's referenced_workflows encoding (see Evidence) showed that only a local call is recorded as ref: refs/pull/<N>/merge with path: …@<sha>, and always for the run's own PR. Remote, pinned, nested and attack-shaped calls were all rejected. An independent review rates this safe to merge.

Summary

The merge queue builds, KUnit-tests and boots every pull request again, even when that pull request's own run just did all of it for the identical tree. For #249, PR run 40616429413 and queue run 40621870017 had the same build key and the same full tree (2caf83b9…). The queue still spent package 9m24s + boot 1m57s on its critical path, plus KUnit, on Namespace.

 plan (merge_group only)
   package_run, verified_run = trusted runs only          # branch, queue, nightly, manual
+  if no trusted verification for the key:
+    N    = PR number from the queue ref (gh-readonly-queue/integration/aurora/pr-<N>-<base>)
+    head = queue commit's 2nd parent                      # approved after its last push
+    for runs at head with a kernel-verified-<key>, newest first:
+      run is kernel-ci.yml / kernel-ci-label.yml, pull_request event, this repo
+      run passed at attempt 1 (never re-run)
+      GitHub's record of the run: exactly one call of kernel-ci-jobs.yml,
+        ref == refs/pull/<N>/merge, path == …/kernel-ci-jobs.yml@<sha>   # local call
+      tree(<sha>) == queue commit's tree                  # authenticated commit, full tree
+      exactly 1 live kernel-package-<key> and 1 live kernel-verified-<key> in the run
+      verified.json names the key, the run, and <sha>
+        → package_run = verified_run = run; package_artifact = that package's ID
+    any error → warning, usual lookup
 package: download artifact-ids=<package_artifact>        # fails if superseded
 KUnit, boot: skipped (verified_run set)
 result: no kernel-verified record uploaded; no package or metadata either
 push after merge: usual trusted-only lookup → builds/tests what no branch or queue run has

The commit comes from GitHub's own record of the run (referenced_workflows), not from the run's self-reported verified.json. A run of the same head in another pull request (say, onto a branch with tampered workflows) records refs/pull/<other>/merge. A retargeted run records a merge commit with the other base's CI, so its tree differs. Full-tree equality covers .github, tools/aurora-ci and the expected-results files, so a base change in CI files alone means a full build. A later queue entry carries the entries ahead of it, so its tree normally differs and it builds.

What this accepts

  1. Within that run, the record is the run's own word. Nothing authenticates which job wrote kernel-verified-<key>, or which package bytes it verified. Any code at the reviewed head that ran in that run (host suites, kernel build scripts) could have written them.
  2. Fork PR runs count. drm/asahi: g15: t6031: build the J516C board data without a board file #249 is a fork PR; GitHub's record names the PR via refs/pull/<N>/merge for forks too.
  3. Testing moves after the merge for these PRs. Package-version drift and flaky tests surface only when the push builds; if it then fails, the change has already merged. The push may instead reuse a trusted package or verification for the key (inherited behaviour: a 90-day verification can outlive a 14-day package).
  4. Paid build work mostly moves from the queue to the push rather than disappearing (FYI @ryanrhughes, @emirb: same Namespace profile, and no profile or ruleset change).
  5. The binding relies on GitHub's behaviour as observed, not on a documented contract. If GitHub changes the record's shape, lookup most likely falls back to a full build. A change that put one PR's merge ref on another PR's call would reopen the side-PR forgery.

A 72-hour cap on the age of a reused PR run would bound drift; it's a follow-up, not a fix for forgery.

Evidence

  • Lookup against the real drm/asahi: g15: t6031: build the J516C board data without a board file #249 queue commit e3ffe88, read-only: PR drm/asahi: g15: t6031: build the J516C board data without a board file #249 → ('40616429413', '12787597313') in 3.9 s (12787597313 is that run's only kernel-package-3b408bd…). The same evidence for PR soc: apple: sep: publish catacomb files by atomic replacement #250 → ('', '').

  • 145 unit tests pass (7 package-inspection tests skip locally), actionlint clean, check-commits.py clean, CI self-test (incl. zizmor) green.

  • New tests reject:

    • Run shape: earlier head; failed run; re-run (attempt 2); other workflow; pull_request_target; other repository; missing, expired or duplicate packages; an incomplete run listing.
    • Recorded call: another PR of the same head; a call to a pinned commit, a branch, the merge ref by name, another repository or another workflow file; nested or missing calls; malformed entries; a merge of another tree.
    • Record: wrong key, run or commit (including a different commit with the same tree), [], null.
    • Zip: oversized, encrypted or bzip2 archives; not a zip; missing member.
    • Fallback: truncated API responses (IncompleteRead) and every other lookup error fall back to the usual lookup.
  • Mutation check: dropping the ref comparison fails the side-PR and pinned-commit cases.

  • GitHub's run record, probed in a private scratch repo (maralcbr/refwf-probe, 64 runs; available on request):

    • Setup: PR 1 and PR 2 had the same head, onto main and onto an attacker-controlled side branch.
    • Method: this PR's acceptance rule was applied to every run's record.
    call in the run's workflow GitHub records accepted
    local ./…, also when skipped (if: false) ref: refs/pull/<N>/merge, path: …@<merge sha>; the job's GITHUB_SHA is that sha only for the run's own PR
    local → local (nested) two entries no
    remote @<sha>, also when skipped path: …@<sha>, no ref no
    remote @main ref: refs/heads/main, path: …@main no
    remote @refs/heads/main, @refs/pull/1/merge or @pull/1/merge, directly or nested workflow file error, nothing recorded no
    PR 2 calling PR 1's merge sha (once a branch made it resolvable) path: …@<PR 1 merge>, no ref no
  • End to end in a real merge queue: a scratch copy, omacom/linux-aurora-ci-scratch (deleted after the test; run IDs below are from it), with this PR and ci: stop the plan job fetching the kernel's history; split the host tests #254 on integration/aurora. It uses the same ruleset except approvals, the same Namespace profile, and scratch PRs with a comment-only kernel change:

    case queue run plan picked result
    vouched PR, full kernel-ci.yml run on the current base 40807224619 that PR run and its package artifact package downloaded in 38 s; KUnit and boot skipped; 5m55s; merged
    PR with only a ci:build label run (kernel-ci-label.yml) 40811757032 the label run and its package artifact 5m39s; merged
    PR run on a base that has since moved 40807898874 nothing full build, KUnit, boot: 10m39s
    second entry in the queue 40807900176 nothing full build, KUnit, boot: 10m55s
    CI-only PR, key already built on the branch 40809753140 the trusted queue build (pull_request_run empty) trusted reuse as before
    • Reused queue runs uploaded only compile-1 and devicetree-1, with no package, verification or metadata.
    • The push after the reused merge found no trusted build, so it built, ran KUnit and boot, and attested: 40807831373, 11m34s.
    • No plan job warned.
  • Not run: a fork PR end to end. The record shape for a fork PR is the one checked on drm/asahi: g15: t6031: build the J516C board data without a board file #249 above.

Timings, from #249:

merge queue critical path
before (40621870017) plan 6m48s → package 9m24s → boot 1m57s → result: 18m55s
with #254 plan ~10s → package → boot → result: ~12 min
with #254 + this, when it applies plan 11–19 s → longest host test job → result: 5m39s and 5m55s (measured, scratch queue runs above)

When it doesn't apply (later queue entries, CI-only base moves, no full run of the PR at its head), the queue builds as with #254.

Merge Danger

Door: two-way

Revert restores full queue builds. Packages and records taken while it was live were never re-uploaded as trusted, so nothing lingers.

Blast Radius: merge-gate

A bad kernel could pass the queue on PR-run evidence and fail only on the push after merge.

Safety review: an independent review found that a writer could forge evidence through a second PR from the same head onto a branch of their own. That's fixed in 5e17ebc by binding to GitHub's record of the run, and the scratch-repo probe found no way around that binding.

The merge queue built, tested and booted every pull request again even
when a run at the pull request's head had just done so for an identical
tree: run 40616429413 for #249 had the same build key and the same full
tree as its queue commit, and the queue spent another 12 minutes on it.

In a merge queue run only, lookup-build.py now also accepts a pull
request run's packages and verification when the run is kernel CI on a
pull request event in this repository at the head the queue merges (the
queue commit's second parent, approved after its last push), passed at
its first attempt and was never re-run, has exactly one live package and
one live verification artifact for the key, and its verification record
names a merge commit whose tree is the queue commit's tree, CI files and
expected results included. The package job downloads that exact package
artifact by its ID. The queue uploads no reusable package, metadata or
verification record on this path, so the push after the merge still
trusts only the branch and the queue. Anything missing, ambiguous or
unreadable leaves the usual lookup.

This changes what the required check attests for those pull requests:
the record is the pull request run's own word, not authenticated per job
or per package, and the trusted build moves to the push after the merge.

Signed-off-by: Marcelo Alcantara <maralc@gmail.com>
When the merge queue takes a pull request run's results, the push after
the merge takes the usual lookup, so it builds and tests only what no
branch or queue run already has for the key.

Signed-off-by: Marcelo Alcantara <maralc@gmail.com>
A run "at the head being merged" proved less than it seemed. Anyone who
can push a branch can open a second pull request from the same head onto
a branch of their own, whose workflows GitHub then runs on the merge.
That run has the same head, so it passed every check, and it could
upload a verification record claiming any commit with the queue's tree.

Take the pull request number from the queue ref, and use what GitHub
recorded for the run instead of what the run says about itself: exactly
one call of kernel-ci-jobs.yml, made from the run's own workflow file
(ref refs/pull/<number>/merge, path ending in the commit it ran), whose
commit has exactly the queue commit's tree. The record must name that
commit. A call to another ref, commit or repository keeps that target in
its path and does not count.

Also fall back to the usual lookup on a truncated API response
(http.client.HTTPException), which escaped the handler before.

Signed-off-by: Marcelo Alcantara <maralc@gmail.com>
@maralcbr
maralcbr marked this pull request as ready for review October 29, 2026 05:02
@maralcbr
maralcbr merged commit 2f72554 into integration/aurora Oct 29, 2026
17 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant