Skip to content

config: build udmabuf, the system dma-buf heap, and the AMD and Nouveau GPU drivers into the Aurora kernel - #67

Draft
joshuaswarren wants to merge 1 commit into
omacom:integration/aurorafrom
joshuaswarren:config/udmabuf-zero-copy-host-memory
Draft

joshuaswarren wants to merge 1 commit into
omacom:integration/aurorafrom
joshuaswarren:config/udmabuf-zero-copy-host-memory

Conversation

@joshuaswarren

@joshuaswarren joshuaswarren commented Oct 11, 2026 •

Copy link
Copy Markdown

Turns on CONFIG_UDMABUF=y, CONFIG_DMABUF_HEAPS=y and CONFIG_DMABUF_HEAPS_SYSTEM=y in arch/arm64/configs/asahi.config.

The fragment adds three lines. In the resolved configuration the dependency walk also turns on CONFIG_TEE_DMABUF_HEAPS=y, the TEE-backed heap (TEE itself was already =y and stays =y). Review found no matching platform node on our hardware, so it is inert.

This PR is kernel config only. No Mesa or Honeykrisp change ships with it. A consumer must opt in with a memfd, UDMABUF_CREATE and a dma-buf import.

Why

Honeykrisp can already import a dma-buf (VK_EXT_external_memory_dma_buf, external buffer features 0x6: importable and exportable). The GPU cannot map arbitrary host pages: VM_BIND takes GEM handles only. The missing piece was a way to turn ordinary host pages into a dma-buf. udmabuf does that for memfd pages.

With these options a local AI runtime can keep its weights in one memfd, wrap the memfd with /dev/udmabuf, and import it into Vulkan. There is one resident copy. There is no staging buffer, and the CPU and the GPU see the same pages.

Evidence

A test kernel built from the 12.6 release tree with only these three options added (plus the BTF-mismatch allowance, so the stock modules load) booted on a 14-inch M2 Max MacBook Pro. The test boot was a one-off build of the 12.6 release tree plus only these options. It was not the packaged kernel image. Honeykrisp was a Mesa 26.3 build. The test program ran: memfd, UDMABUF_CREATE, vkImportMemoryFd, one compute dispatch (d[i] = d[i]*2+1), then the CPU read the result through the original memfd mapping.

case result
16 KiB, 64 KiB, 1 MiB, 64 MiB buffers 0 wrong words, same CPU pointer before and after, no copy
64 KiB buffer at memfd offset 16 KiB 0 wrong words
512 MiB buffer (after raising udmabuf.size_limit_mb to 1024) 0 wrong words, 134,217,728 words
with and without DMA_BUF_IOCTL_SYNC around the CPU access 0 wrong words in both
control: Vulkan-exported dma-buf, CPU-mapped, re-imported 0 wrong words, 16 KiB to 64 MiB

The GPU result appearing in the memfd pages shows that the driver maps those pages and does not copy them.

Limits found by the test

  • Size and offset must be multiples of the CPU page size (16 KiB). Sizes 4096 and 20480, and offset 4096, fail with EINVAL.
  • udmabuf.size_limit_mb defaults to 64 MiB per buffer. A 512 MiB buffer fails with EINVAL at 64 and works at 1024. The parameter is writable at run time through sysfs, and that is what the test did. It should also work as udmabuf.size_limit_mb= on the kernel command line, but I have not tested that. This PR does not change the default.
  • udmabuf takes memfd (shmem) pages only: not a file mapping and not anonymous memory. The pages stay resident, so this does not give larger-than-RAM models.
  • The memfd must be shrink-sealed (F_SEAL_SHRINK). The driver rejects a memfd that has a write or future-write seal (drivers/dma-buf/udmabuf.c, check_memfd_seals).
  • /dev/udmabuf is root:kvm mode 0660 with a user ACL on the test machine. Access policy is a userspace matter and is not part of this PR.

How to check after install

  • ls /dev/udmabuf shows the node.
  • zcat /proc/config.gz | grep UDMABUF shows CONFIG_UDMABUF=y.

Not tested

  • This branch head has not been built or booted with the change. The kernel evidence above is from the 12.6 release tree. On this head I only checked that make defconfig asahi.config gives the three options =y, and that the base head gives # CONFIG_UDMABUF is not set and # CONFIG_DMABUF_HEAPS is not set.
  • Buffers above 512 MiB, memory pressure, and many live udmabufs at once.
  • A memfd without F_SEAL_SHRINK. The driver source rejects it, but I did not run it.
  • /dev/dma_heap/system (the system heap was built into the test kernel, but the test did not use it).
  • Other chips. Only a 14-inch M2 Max was booted.
  • Any effect on boot time or kernel size.

I make no claim about merge readiness. The release builds that would carry this need a normal rebuild of modules and image together.

Second change: AMD and Nouveau GPU drivers as modules

A second commit sets CONFIG_DRM_AMDGPU=m and CONFIG_DRM_NOUVEAU=m in the same fragment, for people who want to try a discrete GPU in a Thunderbolt or USB4 enclosure.

Why: PCIe over Thunderbolt already enumerates devices on these laptops (a Thunderbolt 3 dock shows up with its bridges, controllers and network chip behind the PCIe root port). The running kernel has no driver for a discrete GPU: DRM_AMDGPU, DRM_NOUVEAU and DRM_XE are off, and the helpers amdgpu needs (DRM_TTM, DRM_BUDDY, DRM_SUBALLOC_HELPER) are off as well. So an out-of-tree amdgpu module cannot be built against the shipped kernel either.

What I checked, on the Kconfig level only: I merged the fragment into the running 7.1.12 configuration with and without the two new lines and ran olddefconfig on the same source tree (this branch before the second commit) with the same LLVM 22.1.8 toolchain. Only these symbols change: DRM_AMDGPU=m, DRM_NOUVEAU=m, DRM_TTM=m, DRM_TTM_HELPER=m, DRM_BUDDY=m, GPU_BUDDY=y, DRM_SUBALLOC_HELPER=m, DRM_PANEL_BACKLIGHT_QUIRKS=m, DRM_AMD_DC=y, DRM_DISPLAY_DP_HELPER=y, DRM_DISPLAY_DSC_HELPER=y, DRM_DISPLAY_HDCP_HELPER=y, DRM_DISPLAY_HDMI_CEC_NOTIFIER_HELPER=y, CEC_NOTIFIER=y, the Nouveau sub-options (backlight, CH7006, SIL164, debug levels), and the "is not set" entries of the new sub-options. These are built in rather than modules: GPU_BUDDY, CEC_NOTIFIER and the three display helpers, so the kernel image grows a little. ARM64_16K_PAGES, DRM_ASAHI, DRM_APPLE, PCIE_APPLE, USB4, USB4_APPLE_SOC, PHY_APPLE_ATC and APPLE_DART are identical in both results. DRM_AMD_DC_FP does not appear in the resolved configuration, so display output is not expected; this is a compute setup. DRM_XE stays off (it needs 4K pages). HSA_AMD, AMDGPU_SI, AMDGPU_CIK and AMDGPU_USERPTR stay off.

Not done for the second commit:

  • No build, no boot and no card. Module size and the change in package size are not measured.
  • Whether amdgpu works on 16K pages over a Thunderbolt tunnel here is untested. The PCIe driver maps endpoint BARs non-posted on the M1 Pro, M1 Max and M2 Pro/Max laptops, and amdgpu maps its VRAM aperture write-combined, so a fault there is possible; the 13-inch M1 maps BARs posted.
  • The result of the full kernel build with clang on arm64 (DRM_AMD_DC building) is not known.
  • Suspend with an active tunnel is not covered.

If you would rather keep the udmabuf change alone, the second commit can be dropped without touching the first.

Checks for the second change (AMD and Nouveau modules)

Nothing below has run yet. Each check has a pass line and a fail line so a reviewer can tell when it is done.

  • 1. Build: the package builds with both options set to m, and amdgpu.ko and nouveau.ko are in the package for the 16K page kernel. PASS: build exits 0 and modinfo on both files shows vermagic equal to the kernel release. FAIL: build error or a missing module.
  • 2. No change on Apple hardware: boot the new kernel on a 13-inch M1 with no eGPU attached. PASS: Honeykrisp still lists the Apple GPU in vulkaninfo --summary, lsmod shows neither amdgpu nor nouveau, and dmesg has no new error lines compared with the current kernel. FAIL: any new error, or either module auto-loads.
  • 3. BAR assignment (hardware gate, runs on the current kernel before any rebuild): with the dock and card attached, lspci -vv shows BAR0 and BAR5 assigned inside the bridge window and dmesg has no "can't assign" or "no space" line. If it fails on the 13-inch M1, repeat once on a 16-inch M1 Max before calling the card unusable. This gate decides whether the rebuild and the reboot happen at all.
  • 4. Driver bind (new kernel, card attached): modprobe amdgpu (or nouveau for an NVIDIA card) prints a VRAM size line in dmesg and creates /sys/class/drm/cardN. FAIL: SError, oops, firmware error or a hang while mapping VRAM. Stop on the first one, copy dmesg, power the enclosure off. Risk, stated as inference: amdgpu maps VRAM with write-combining, and the PCIe driver says endpoint BARs on the Max laptops fault unless mapped non-posted.
  • 5. Vulkan sees the card: with VK_DRIVER_FILES set to the RADV (or NVK) ICD only, vulkaninfo --summary lists the eGPU with the expected driver name. FAIL: device missing or crash.

Rules for every check: one host, one device, one change at a time; no hot-unplug during compute; no suspend.

This pull request stays a draft until check 3 passes. Check 1 can run on a draft build. Check 2 needs a reboot, so it waits for check 3.

@joshuaswarren

Copy link
Copy Markdown
Author

Checked at head 6984b8c (base ac2e867). The body matches the receipt. Three small points below.

Receipt: run-m2-boot-2bff13d9.log, sha256 checked against SHA256SUMS (all four files OK).

Table rows against the log:

  • 16 KiB, 64 KiB, 1 MiB and 64 MiB udmabuf: 0 wrong words, the CPU pointer is the same before and after. Matches.
  • 64 KiB at memfd offset 16 KiB: 0 wrong words. Matches.
  • 512 MiB after the limit is raised to 1024: 0 wrong words, 134217728 words. Matches.
  • With and without DMA_BUF_IOCTL_SYNC: 0 wrong words in both passes. Matches.
  • Control, Vulkan-exported dma-buf, 16 KiB to 64 MiB: 0 wrong words. Matches.

Limits against the log:

  • Size 4096, size 20480 and offset 4096 fail with EINVAL, in all three passes. Matches.
  • 512 MiB fails with EINVAL at the 64 default and works at 1024. Matches.
  • size_limit_mb reads 64 at the start and is restored to 64 at the end.
  • Kernel GPU timeout and fault lines for the boot: 0.
  • Device is Apple M2 Max (G14C B1), driver Mesa 26.3.0-devel at git 69bde15340, features 0x6 (exportable and importable). The log's own legend for the bits is garbled, but 0x6 is exportable plus importable in the Vulkan flag values, so the body is right.

Config check, rerun just now: make defconfig asahi.config on the PR head gives CONFIG_UDMABUF=y, CONFIG_DMABUF_HEAPS=y and CONFIG_DMABUF_HEAPS_SYSTEM=y. On the base head it gives "# CONFIG_UDMABUF is not set" and "# CONFIG_DMABUF_HEAPS is not set". The diff is the three added lines in asahi.config.

Wording: no hostnames, no paths, and the body says it makes no claim about merge readiness.

Three statements are not backed by the receipt files, so they rest on the author alone:

  • "plus the BTF-mismatch allowance" in the test kernel.
  • "/dev/dma_heap/system exists on the test kernel".
  • "VM_BIND takes GEM handles only".
    The first two are not shown in the log. The third is a statement about the driver, not a test result. Either add the evidence to the folder or mark them as notes.

@chriskearney

Copy link
Copy Markdown

Review is progressing. The diff is exactly the three configuration lines; the resolved configuration also enables the existing TEE heap defaults through their dependencies, so I’m checking those separately rather than assuming only three effective settings change.

The existing udmabuf implementation requires shrink-sealed memfd/hugetlb pages and retains its 64 MiB per-buffer default. The system heap allocates new zeroed pages; it does not wrap an existing memfd. Device access remains governed by the existing userspace rules.

Focused ARM64 builds of udmabuf, the heap framework and system heap are underway. Your M2 Max results remain the hardware evidence; this does not claim a new boot or broader-chip verification.

@chriskearney chriskearney added enhancement New feature or request priority:P2 Recovery, performance or hardware qualification owner:kernel Implementation belongs to kernel area:gpu Affected gpu subsystem status:candidate Candidate exists; review and tests before merge labels Oct 11, 2026
@chriskearney

Copy link
Copy Markdown

Source/configuration review and all three focused ARM64 object builds pass. No HIGH/MEDIUM blocker was found. The release-config seed enables the three requested settings; the separate defconfig+asahi check also records the dependent TEE defaults, which have no matching Apple platform node.

The PR remains draft, so it is not merged. The existing allocation-result cleanup issue has a source-ready upstream repair now tracked in #69; I’m adding failure-path controls before integration. Normal matched image/modules rebuild and broader runtime qualification remain separate from these source checks.

@chriskearney chriskearney added the launch:follow-up Retained improvements or broader retests outside the initial launch label Oct 11, 2026
@joshuaswarren joshuaswarren changed the title config: build udmabuf and the system dma-buf heap into the Aurora kernel config: build udmabuf, the system dma-buf heap, and the AMD and Nouveau GPU drivers into the Aurora kernel Oct 15, 2026
Set CONFIG_DRM_AMDGPU=m and CONFIG_DRM_NOUVEAU=m in the Aurora config
fragment, next to the udmabuf lines of this branch.

Why: PCIe over Thunderbolt already enumerates devices on these laptops, but
the running kernel has no driver for a discrete GPU. DRM_AMDGPU and
DRM_NOUVEAU are off, and the DRM helpers amdgpu needs (TTM, buddy, suballoc)
are off too, so an out-of-tree amdgpu module cannot be built against the
shipped kernel either.

What the resolved configuration does (Kconfig only, nothing built): the
fragment is merged into the running 7.1.12 configuration with and without
the two new lines, then olddefconfig is run on the same tree. Only these
symbols change: DRM_AMDGPU=m, DRM_NOUVEAU=m, and what they pull in or
default: DRM_TTM=m, DRM_TTM_HELPER=m, DRM_BUDDY=m, GPU_BUDDY=y,
DRM_SUBALLOC_HELPER=m, DRM_PANEL_BACKLIGHT_QUIRKS=m, DRM_AMD_DC=y,
DRM_DISPLAY_DP_HELPER=y, DRM_DISPLAY_DSC_HELPER=y,
DRM_DISPLAY_HDCP_HELPER=y, DRM_DISPLAY_HDMI_CEC_NOTIFIER_HELPER=y,
CEC_NOTIFIER=y, the Nouveau sub-options (backlight, CH7006, SIL164, debug
levels), and the "is not set" entries of the new sub-options. The 16K page
size, the Apple GPU, display, PCIe, USB4 and DART symbols are identical.
DRM_AMD_DC_FP does not appear in the resolved configuration, so this is a
compute setup, not display output. DRM_XE stays off (it needs 4K pages),
and HSA_AMD, AMDGPU_SI and AMDGPU_CIK stay off.

Not done: no build, no boot, no card. Module size and the effect on the
package size are not measured. This only prepares the kernel side; whether a
Thunderbolt-attached card works on these laptops is untested.
@joshuaswarren
joshuaswarren force-pushed the config/udmabuf-zero-copy-host-memory branch from d10d360 to a918a87 Compare October 25, 2026 03:31
@joshuaswarren

Copy link
Copy Markdown
Author

Rebased onto integration/aurora. That branch already has the DMA settings. So this just turns on the AMDGPU and Nouveau modules. I'm leaving it a draft. We still need the build and card tests.

@joshuaswarren

Copy link
Copy Markdown
Author

[Joshua's agent] This draft needs an admitted kernel build, boot, and AMD and Nouveau card tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:gpu Affected gpu subsystem enhancement New feature or request launch:follow-up Retained improvements or broader retests outside the initial launch owner:kernel Implementation belongs to kernel priority:P2 Recovery, performance or hardware qualification status:candidate Candidate exists; review and tests before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants