Repository navigation
Conversation
nvidia-open-dkms (x86_64), v4l2loopback-dkms (the XPS camera depends on it) and broadcom-wl-dkms come from Arch's extra, so the gate never built them: a kernel like 7.2.8-9 could break them and nothing here would say. ci/dkms-packages marks them repo=extra. check-dkms reads them from the channel's own mirror, the snapshot that channel's machines install from, and builds them whenever a kernel changes; they never count as a change of their own. This repository's own copy of the same name wins, as it does for pacman. Build tools now come from the channel's mirror too. Arch updates them on its own schedule, so dkms-check runs daily with --from-repos against edge's kernels, and only when one of those packages or a kernel has a version no green run has built (check-dkms fingerprint, remembered in the Actions cache). A pull request that adds a DKMS recipe is now checked before it publishes, against every kernel, even though master's registry does not list it yet; before, it was left out until edge published it. New: check-dkms needs, for deciding on the CI host whether a build holds anything to check. tests/check-dkms.sh grows from 17 to 33 cases.
DKMS check (x86_64) and DKMS check (aarch64) become required statuses, so every pull request needs both. A small decide job reads the built packages on the host (check-dkms needs): an architecture with no kernel or DKMS package gets "nothing to check" in seconds, and only the others start a container and build. A failed read reports an error instead of passing, and a run superseded by a newer push leaves its result to the newer one.
There were six more ways for it to pass without checking: - build-pr.yml reuses the artifact of a package whose tree already built, so the triggering run can hold none of the pull request's packages. bin/pr-packages now collects the head's packages by their artifact labels, wherever they were built. - A recipe that lost its headers tree or dkms.conf looked like nothing to check. check-dkms needs --channel also counts a package the channel publishes as a kernel or DKMS package, so the check's own "lost its tree" error is reached. - An older run could post success while a newer one was starting. The workflow now runs once per pull request branch and cancels the older run before the newer one posts. - This repository's own copy of a repo= name was left out of promotion changes; only packages really taken from Arch are now. - fingerprint ignored an unreadable repository database; every part is now checked, and a missing package is an error. - --from-repos with --since checked nothing; it now checks the Arch packages whatever else is given. tests/check-dkms.sh: 37 cases.
There were five more ways a required status could go green, or get stuck: - A recipe with no live artifact was skipped, but publish.yml rebuilds what it cannot find. bin/pr-packages now expects an artifact for every architecture where edge publishes a recipe as a kernel or DKMS package, and for at least one architecture for a recipe edge does not publish yet. Anything missing (a build only planned for an author not yet vouched for, an expired artifact) leaves both statuses pending, "Waiting for a build of: ...". check-dkms roles lists what edge publishes. - unpack_packages hid a failed extraction behind `|| die`; each artifact is now extracted with its errors checked and must yield a package. - A failed, cancelled or dispatched build could cancel a running check and then post success. Only successful pull request builds run the check and share the branch's concurrency group; every other completion gets a group of its own and posts nothing. - The fork lookup now passes the branch as an encoded query parameter.
The required statuses dkms-pr.yml posted from a workflow_run could not see what a build would publish, only what artifacts happened to exist: a package that became a DKMS package, a multi-architecture recipe missing one architecture, an arch=any module, and a recipe edge already holds all came out wrong, and an older build could strand a newer head's status pending. The check now runs in build-pr.yml, as jobs the required `result` needs. bin/build-artifacts reads the build's own matrix: each planned package is either this run's artifact or built nothing because edge already holds its version, and each artifact the plan reused must still exist. Every download, extraction and copy is checked. dkms-decide reads those packages on the host in seconds, and only an architecture with a kernel or DKMS package among them starts a container. Both run the base branch's tooling, like the build. No new required check, no status writes, nothing that can race. dkms-pr.yml, bin/pr-packages and check-dkms roles are gone. tests/build-artifacts.sh covers the collector with a stubbed gh.
tests/pr-workflow-approval.cjs renders the result job's expressions as JavaScript, where needs.dkms-decide is a subtraction. dkms_decide reads the same in both. The test now also covers the DKMS jobs: a check that ran must pass, and a skipped one passes.
One job per architecture built every kernel and module pair in a row: the two NVIDIA modules alone took about two minutes per kernel, so a whole channel ran for 20 minutes. Now `plan` reads the channel on the host in seconds (check-dkms matrix) and every pair gets its own leg, named for what it builds: "DKMS (x86_64, linux-omarchy 7.2.8-13, nvidia-open-dkms 615.78.08-3)". A leg that names versions the channel no longer has fails, so a run that raced a promotion is redone rather than passed. build-pr.yml calls dkms-check.yml as a reusable workflow, so a pull request that builds nothing shows one skipped "DKMS" check, and the fan-out exists once. The plan keeps the kernels and DKMS packages the build would publish as one small artifact for the legs. The daily run checks every pair on edge that has not built at its versions yet: each green leg saves its key in the Actions cache, and the plan drops legs whose key exists. That replaces the per-architecture fingerprint. Skipped jobs show their names unevaluated, so no job name uses matrix expressions any more. The build job's runner moves from the matrix to a map, which gives its checks GitHub's default names, the ones they always had.
broadcom-wl failed on every edge kernel with only make's own error lines in the log: a link or modpost failure prints no "error" until make gives up. The leg now prints the compiler's errors and the last 25 lines.
broadcom-wl's build links /usr/lib/broadcom-wl-dkms/wlc_hybrid.o_shipped, which the check left out, so it failed on every kernel while machines build it fine. Still unpacked rather than installed, so DKMS's pacman hook does not build it against every kernel.
linux-aurora 7.1.12.aurora7-2 is built with LLVM=/usr/lib/llvm18/bin/ (CONFIG_CC_IS_CLANG=y, CONFIG_LD_IS_LLD=y). For such a kernel dkms 3.4.4 adds LLVM=1 to make itself (/usr/bin/dkms:799-807), and kbuild then runs clang, ld.lld and the llvm-* tools from PATH. The check's container had none of them, so all four aurora legs that compile failed before any C was compiled: brcmfmac-awdl, v4l2loopback and xpadneo with "clang: command not found", nvidia-open in gen-btf.sh with "llvm-objcopy: command not found". After installing the headers, run reads each kernel's build tree and, when one was built with clang and the tools are missing, installs clang, llvm and lld from the channel's mirror: the toolchain a Mac gets from pacman (clang 23.1.3 on arm-mirror today), which is what its own DKMS would use. Nothing changes for gcc kernels. linux-aurora-headers depends only on pahole, so a Mac without clang cannot build any DKMS module for that kernel. The result says so instead of hiding it behind the install. bin/check-dkms toolchain <build tree>... prints what a kernel needs, so the detection is tested without root.
A package line has seven fields, so reading six put the pkgbase into the kernel release: every leg logged "Installing linux-omarchy-headers (7.2.8-9-omarchy<TAB>linux-omarchy)". The value is only printed, so nothing else changes.
No Apple Silicon Mac has an NVIDIA GPU, so nvidia-open-dkms is never installed next to linux-aurora, yet the check built it there: the leg took 2m20s on edge before it failed for want of llvm-objcopy. kernels= now also takes exclusions: kernels=!<headers>,... builds a module against every kernel but those. nvidia-open-dkms on aarch64 keeps every other kernel (linux-n1x, linux-omarchy-aarch64 today, and any ARM kernel added later) instead of a list that a new one would silently miss.
macbook12-spi-driver-dkms fails on all three x86_64 kernels on edge
(7.2.8), with four errors from four kernel changes:
6.10 'struct acpi_driver' has no member named 'owner' (cc85f9c05bba)
6.11 platform_driver.remove() returns void (0edb555a65d1)
6.12 report_fixup() returns a const pointer (fe73965d0786)
6.12 asm/unaligned.h: No such file or directory (5f60d5f6bbc1)
Its upstream branch last moved on 2024-06-02, so it will not come back
by itself, and it kept the check red on every run. The in-tree
applespi (CONFIG_KEYBOARD_APPLESPI=m in all three kernels) drives the
keyboard and touchpad.
ci/dkms-packages takes known-broken="<why>". run records each pair's
raw outcome and judge decides the verdict: a marked package that fails
is "known broken: <why>" and passes; one that builds fails ("builds
now; remove its known-broken mark"), so a stale mark cannot hide a
working package; an excluded pair is neither. judge is also a mode, so
it is tested without root. run --results <dir> keeps the verdicts for
the CI summary.
The registry is now read once, with the quoted reason taken out first:
a reason may hold spaces, or a word like aarch64 that the old per-field
loops would have read as an architecture. An unknown field is an
error, so a misspelt mark cannot silently stop applying.
The legs must report into one place, a single check on a pull request,
so each leg's result has to be data, not only a job colour.
container now decides the leg's outcome out here, from run's exit
status and the verdicts run leaves in a mounted directory: passed,
excluded, known-broken, fixed, failed, or error when run stopped before
any verdict or its status disagrees with them. --result writes it as
JSON, and the exit status follows it, so a known-broken leg is green.
The container runs a pull request's code (its kernel's and modules'
Makefiles), and anything it printed reached the runner's
workflow-command parser. Its output now sits between
::stop-commands:: and a random token, so it cannot mask values, set
outputs or raise annotations.
report turns a run's legs and their results into one verdict:
{"conclusion", "title": "N/M passed, K known broken, F failed",
"summary"} with a row per leg, failures first. A leg without a result
counts as failed, and so does a plan that did not finish. Names and
details come from package metadata a pull request controls, so they
lose control characters, pipes, angle brackets, brackets and
backticks: no extra cells, HTML or links.
--all keeps every pair of a channel with a build's packages in place
of the published ones, to run the whole check against a pull request's
tooling or build.
GitHub lists every job of a pull_request run as a check, so a kernel pull request carried a line per kernel and module: 45 for the edge check of the check itself (run 39169993011). The matrix now runs after the build, in dkms-check.yml on workflow_run, and the pull request gets one check written through the Checks API: "DKMS check", opened in progress when the build finishes and closed with "N/M passed, K known broken, F failed" and a row per kernel and module, linking the run where every leg is its own job. That run's jobs attach to master's head, not to the pull request. workflow_run runs master's copy with a token that could write, so it follows GitHub's guidance for it: - only `target` and `report` may write, and only checks: write. They check out nothing and download nothing; the event's commit and run ids and the matrix's verdict reach the API through env and jq. - the matrix (dkms-matrix.yml, also what promote.yml calls) holds contents: read and actions: read, no secrets, and runs master's tooling. The build is data: build-pr.yml records what it planned in a dkms-request artifact, and bin/build-artifacts checks its shape before any of it names a path or a URL. Packages are only unpacked in a leg's throwaway container. - tests/dkms-check-workflow.cjs holds these properties. build-pr.yml no longer calls the matrix, and `result` no longer waits for it. The check is "DKMS check" on the pull request's head instead; it blocks a merge once branch protection requires it. dkms-check.yml keeps the daily and by-hand channel runs, and takes -f pr=<n> (or head_sha) for the pull request path, with -f all=true for every pair on the channel: dispatched on a branch, that is how a change to the check is proven before it merges, now that no pull_request trigger runs it.
…links jq's ^ and $ are line anchors, so a request naming "linux-omarchy\n" passed the shape check; the newline then split the package list and the collection read "x86_64" as a package name. The patterns use \A and \z now. An artifact is a pull request's data, and cp follows a link: a packages.tar holding leak-1-1-x86_64.pkg.tar.zst -> /etc/hostname copied the runner's file into the collection. A package that is not a regular file now fails the collection, in both artifact forms.
…he check The DKMS check reads package names, versions and tree names from a pull request's build, and prints and posts them: - The runner still parses the legacy "##[command]" syntax anywhere in a line, and container mode printed the leg's outcome, made of the container's own text, after it had resumed workflow commands. It now prints it before, and the verdict rows reach the result as plain text. - read_local takes only the names makepkg itself allows: pkgname and pkgbase [A-Za-z0-9@._+-], versions [A-Za-z0-9._+~:-], tree names the same, so none can carry "##[", a pipe or markup into logs, job outputs or the summary. - A verdict row with the wrong number of fields or an unknown verdict made a leg "passed" next to a built row; it is an error now. - The summary's kernel, module and detail cells are code spans, so "@omacom/core https://..." neither mentions anyone nor links.
macbook12-spi-driver-dkms is to be retired: #298 packages the freeze-safe T1 Touch Bar driver as apple-ib-drv-dkms, which conflicts with it and replaces it, and the in-tree applespi already drives the keyboard and touchpad. The known-broken reason says so. apple-ib-drv-dkms is listed for x86_64 now. Until #298 publishes it is only noted; without the line, every plan after it publishes would fail with "ships a DKMS module ... but is not in ci/dkms-packages". Built from #298's head (61b712c) as a gate leg against linux-omarchy 7.2.8-9 and linux-omarchy-eevdf/-muqss 7.2.8-8 on edge, it builds apple-ibridge and apple-touchbar on all three.
When a pull request's build shipped nothing, the plan dropped --local and so planned every pair on the channel: a tooling-only pull request would have spent the 43 legs edge has today and could fail on something it never touched. It now plans none and says why, unless all is set, which asks for every pair. The daily plan skipped a leg when any cache entry had its key, but a pull request's run can save any key under its own ref. Only entries of the run's own ref count now.
….yml Concurrency was keyed by the pull request's branch and applied before `target` filtered the event, so a build this workflow ignores (failed, cancelled) or a late build of an older commit could cancel the check of the pull request's head. Pull request builds are keyed by commit now, and ignored ones get a group of their own; a second build of the same commit (a label starts one) still replaces the first. `target` finds the pull request by its head, since the event names none for a fork's, and opens no check for a commit the pull request has moved past. A fork's pull request builds with its own build-pr.yml, which writes the dkms-request this check reads, so a fork that changes that file could describe a build without its kernel and get a green check. Such a pull request is refused: its check fails with the reason and runs no legs. Same-repository branches are pushed by people who can write here, and every other fork is read through master's build-pr.yml.
The fork guard read the pull request's files through a pipeline into grep -q inside an if: an API error made it "not changed", and under pipefail grep -q leaving early could too, so the check opened for a fork that does change build-pr.yml. It ran only when a pull request number was known, so a head_sha dispatch skipped it. target now reads the build run itself in every mode: it must be the pull request build of that commit, from build-pr.yml. For a build from a fork it compares build-pr.yml at the head with the copy at the merge base (the compare API's merge_base_commit), each read by a command of its own: any error fails the job before a check opens, and a missing file counts as changed. Comparing with the merge base, not master's tip, keeps a fork that is only behind from being refused. The concurrency group now also requires the build-pr.yml path, as target does, so a same-named workflow cannot share the group of the real build and cancel its check.
A package's path went into check-dkms's tab-separated lines and its logs unchecked: a file named "tuxedo<TAB>x##[add-mask]y-1-1-x86_64..." shifted the fields and put "##[" into a planning log, where the runner still reads the legacy command syntax. read_local and build-artifacts now take only names of makepkg's shape, and an odd one fails without being repeated.
A fork whose merge base predates build-pr.yml has no such file at either end, and its pull request runs master's copy. Reading it there returned 404, which stopped target before it opened a check, so a pull request that changes nothing about the build got no DKMS check. #152's head (67c20f8, merge base c409cb6) is one. A blob helper now reads the file and says "missing" for a 404 only; any other error still stops the job. Missing at both ends is the same file. With the step's own logic against the API: #298's head is accepted (8825c82 at both ends), #152's is accepted (missing at both), this branch's head would be refused (a189ea1 at its merge base, 4ddc6bc at its head), and an unknown commit stops with exit 1.
A pull request's plan reuses the build artifact of an identical package tree from any earlier run, and build-artifacts took the newest artifact of that name repository-wide. Any run can upload an artifact under any name, so a fork with a workflow of its own could upload one named for another pull request's tree, and that pull request's unchanged build would reuse it: the check vetted the build it was given, not the run that made what it collected. Each reused artifact's run must now be this repository's build-pr.yml, for a pull request or by hand, and a fork's run only when its build-pr.yml is its merge base's: the rule dkms-check.yml applies to the build it checks, with a 404 read as "missing" and any other error failing the collection. Newer artifacts from runs that cannot vouch are skipped and said so; none trusted fails. In the test, a forked run's xpadneo-dkms 6.6.6 artifact under the reused label was collected before and is passed over for the trusted 0.9.8 one now; an artifact from another workflow, an error reading a fork's build-pr.yml, and a fork that predates build-pr.yml are covered too.
matrix tested "-z $(read_local)", which drops read_local's exit status: an archive that does not unpack, or a package name makepkg would not allow, printed [] and exited 0, so a build's kernels and DKMS packages could fall out of the check unseen. read_local now runs on a line of its own, and --save-local reuses its rows rather than a process substitution whose status is lost too.
The lookup read one page of 20 artifacts by name. A fork can upload any number under a pull request's label, so 20 untrusted ones in front hid the trusted one behind them and the collection failed: closed, but a pull request could be denied its check. It now reads every page, 100 at a time, and orders them all newest first. The test stub pages the way GitHub does, and 25 untrusted artifacts in front of a trusted one now leave the trusted one collected.
The pagination test put 25 untrusted artifacts in front of the trusted one, fewer than the 100 a page holds, so it passed without --paginate too. It puts 120 now: green with --paginate, and with it removed the collection fails with "no artifact ... comes from a build this check trusts".
spark-hwmon-dkms (#1587) landed on master after this check was written, and check-dkms fails any channel that publishes a DKMS package missing from ci/dkms-packages: `check-dkms matrix --arch aarch64 --channel edge` stopped with "spark-hwmon-dkms ships a DKMS module for aarch64 but is not in ci/dkms-packages". It drives the GB10's power telemetry, and no Apple Silicon Mac has a GB10, so like nvidia-open-dkms it is built against every aarch64 kernel but linux-aurora. Edge plans two legs for it: linux-n1x 7.0.14 and linux-omarchy-aarch64 7.3.
check-dkms read arm-mirror.omarchy.org for every aarch64 channel, as the builder did when this check was written. Since 5594483 the builder reads arm-rc-mirror and arm-stable-mirror for rc and stable, the snapshots those channels' machines install from, so an rc or stable check on aarch64 built Arch's DKMS packages and the toolchain from edge's live mirror instead of that channel's. mirror_server now names a mirror per channel on both architectures, and tests/check-dkms.sh checks all six against build/Dockerfile's Server lines; it fails on the old function. `check-dkms matrix --arch aarch64` plans rc and stable from arm-rc-mirror and arm-stable-mirror.
[Medium impact] The check reaches pull-request builds, scheduled edge checks, and both channel promotion paths. Fix the merge gate and promotion package selection before merging.
|
| workflow_run: | ||
| workflows: [Build changed packages] | ||
| types: [completed] |
There was a problem hiding this comment.
DKMS check starts after Build changed packages finishes, but the required result check waits only for changes and build. The documented required checks do not include DKMS check. A package-only PR can therefore auto-merge before DKMS starts, or despite its failure. If it merges first, target finds no open PR and skips the check entirely.
Make DKMS part of a required check so broken kernel/module combinations cannot merge and publish unchecked.
| <(printf '%s\n' "$ours") <(printf '%s\n' "$repos")) | ||
| ARCH_SOURCED=$(awk -F'\t' '$1 == "dkms" { print $2 }' <<<"$repos") | ||
| fi | ||
| PACKAGES=$(compose "$(printf '%s\n%s\n' "$published" "$repos" | sed '/^$/d')" "$fresh") |
There was a problem hiding this comment.
Promotions check the wrong packages
With --since, PACKAGES contains only the source channel's packages. The target database only decides which names changed. For a selective kernel promotion, the check uses the source's module versions even though the target keeps its own unpromoted modules. Module promotions have the same problem, and target-only kernels are missed entirely. A promotion can pass without testing the combination users receive, or fail because it tested a different combination.
Build the package set that the target will actually contain. Use the target's Arch mirror for its external modules and build tools.
Nothing here builds a DKMS module against a new kernel. DKMS packages ship source, and the compile only happens on a user's machine during the update. So when linux-omarchy 7.2.8-9 renamed
battery_hook_register(0803-acpi-battery-hook-api-acpi-prefix.patch), the TUXEDO modules stopped building on every 4.0.5 machine that rebooted into it (#1244), and nothing told us first.bin/check-dkmsdoes what a machine does on an update. It installs kernel headers, then runsdkms buildfor each DKMS source tree against each kernel, in a throwaway Arch (or Arch Linux ARM) container. Modules that opt out of a kernel throughBUILD_EXCLUSIVEare reported as excluded, not built.What it checks:
nvidia-open-dkms(x86_64,nvidia.sh),v4l2loopback-dkms(the XPS camera'sintel-ipu7-cameradepends on it, and the Cam Link fix) andbroadcom-wl-dkms.ci/dkms-packagesmarks themrepo=extra.mirror,rc-mirror,stable-mirror; ARM has one), the snapshot that channel's machines install from.dkms.conf), not from names.ci/dkms-packageslists the DKMS packages and their architectures, and a published DKMS package missing from it fails the check. A new recipe is checked against every kernel before it publishes, listed or not.Fanned out, one leg per kernel and module. A small
DKMS planjob reads the channel on the host in seconds (check-dkms matrix). Every kernel and DKMS package pair then gets its own leg, which installs that kernel's headers and builds that package's trees in a throwaway container.DKMS (x86_64, linux-omarchy 7.2.8-13, nvidia-open-dkms 615.78.08-3).${{ matrix.* }}in a name showed raw. Thebuildjob's runner moved from the matrix to a map, which gives its checks GitHub's default names, the ones they always had.It blocks at PR time, through
result.build-pr.ymlcallsdkms-check.ymlas a reusable workflow, and the requiredresultneeds it. No new required check, no branch-protection change, no commit statuses.DKMScheck.bin/build-artifactscollects it from the build's own matrix: each planned package is this run's artifact, or built nothing because edge already holds its version (a README change in a kernel recipe). Each artifact the plan reused from an earlier run must still exist. The plan keeps the kernels and DKMS packages among them as one small artifact for the legs.dkms.conffails the plan.pkgbuilds/, so it can't change the check that judges it. Until this merges, master has nobin/check-dkmsand the plan finds nothing.resultwith it.arch=anypackages, built once on x86_64, are checked on both architectures.Elsewhere:
promote.ymlneeds thedkmsjob): only kernels and DKMS packages whose version differs from the target, narrowed topackageswhen a promotion names some.ship's rc → stable goes through it.dkms-check.ymlschedule): every pair on edge that hasn't built at its versions yet. That catches Arch updating the DKMS packages Omarchy installs from it.gh workflow run dkms-check.yml -f channel=stable(add-f from_repos=truefor only Arch's packages).Merge #1254 and #1244 first. The check is red against stable today, run for real as root in a throwaway 4.0.5 VM:
macbook12-spi-driver-dkmsfails against every kernel, stock-basedlinux-ptlincluded:asm/unaligned.his gone,platform_driver.removenow returns void, andacpi_driver.ownerwas removed. Build the MacBook SPI and Touch Bar modules on 7.x kernels #1254 fixes it.It also found a problem on edge aarch64: nothing builds against
linux-aurora-headers. Every module fails against7.1.12-7-2-ARCHwithError 127, command not found:LLVM=1to every build for a kernel whose.confighasCONFIG_CC_IS_CLANG=y.clangandld.lldonPATH.linux-aurora-headersdepends only onpahole./usr/lib/llvm18/binand isn't onPATH.brcmfmac-awdl-dkmsincluded.linux-aurora's headers package, so it's not part of this PR. Until it lands, an aarch64 PR that ships a DKMS module stays red.tests/check-dkms.shhas 42 cases,tests/build-artifacts.sh11, andtests/pr-workflow-approval.cjscoversresultwith the DKMS job.