Repository navigation
config: build udmabuf, the system dma-buf heap, and the AMD and Nouveau GPU drivers into the Aurora kernel - #67
Conversation
|
Checked at head 6984b8c (base ac2e867). The body matches the receipt. Three small points below. Receipt: run-m2-boot-2bff13d9.log, sha256 checked against SHA256SUMS (all four files OK). Table rows against the log:
Limits against the log:
Config check, rerun just now: Wording: no hostnames, no paths, and the body says it makes no claim about merge readiness. Three statements are not backed by the receipt files, so they rest on the author alone:
|
|
Review is progressing. The diff is exactly the three configuration lines; the resolved configuration also enables the existing TEE heap defaults through their dependencies, so I’m checking those separately rather than assuming only three effective settings change. The existing udmabuf implementation requires shrink-sealed memfd/hugetlb pages and retains its 64 MiB per-buffer default. The system heap allocates new zeroed pages; it does not wrap an existing memfd. Device access remains governed by the existing userspace rules. Focused ARM64 builds of udmabuf, the heap framework and system heap are underway. Your M2 Max results remain the hardware evidence; this does not claim a new boot or broader-chip verification. |
|
Source/configuration review and all three focused ARM64 object builds pass. No HIGH/MEDIUM blocker was found. The release-config seed enables the three requested settings; the separate defconfig+asahi check also records the dependent TEE defaults, which have no matching Apple platform node. The PR remains draft, so it is not merged. The existing allocation-result cleanup issue has a source-ready upstream repair now tracked in #69; I’m adding failure-path controls before integration. Normal matched image/modules rebuild and broader runtime qualification remain separate from these source checks. |
Set CONFIG_DRM_AMDGPU=m and CONFIG_DRM_NOUVEAU=m in the Aurora config fragment, next to the udmabuf lines of this branch. Why: PCIe over Thunderbolt already enumerates devices on these laptops, but the running kernel has no driver for a discrete GPU. DRM_AMDGPU and DRM_NOUVEAU are off, and the DRM helpers amdgpu needs (TTM, buddy, suballoc) are off too, so an out-of-tree amdgpu module cannot be built against the shipped kernel either. What the resolved configuration does (Kconfig only, nothing built): the fragment is merged into the running 7.1.12 configuration with and without the two new lines, then olddefconfig is run on the same tree. Only these symbols change: DRM_AMDGPU=m, DRM_NOUVEAU=m, and what they pull in or default: DRM_TTM=m, DRM_TTM_HELPER=m, DRM_BUDDY=m, GPU_BUDDY=y, DRM_SUBALLOC_HELPER=m, DRM_PANEL_BACKLIGHT_QUIRKS=m, DRM_AMD_DC=y, DRM_DISPLAY_DP_HELPER=y, DRM_DISPLAY_DSC_HELPER=y, DRM_DISPLAY_HDCP_HELPER=y, DRM_DISPLAY_HDMI_CEC_NOTIFIER_HELPER=y, CEC_NOTIFIER=y, the Nouveau sub-options (backlight, CH7006, SIL164, debug levels), and the "is not set" entries of the new sub-options. The 16K page size, the Apple GPU, display, PCIe, USB4 and DART symbols are identical. DRM_AMD_DC_FP does not appear in the resolved configuration, so this is a compute setup, not display output. DRM_XE stays off (it needs 4K pages), and HSA_AMD, AMDGPU_SI and AMDGPU_CIK stay off. Not done: no build, no boot, no card. Module size and the effect on the package size are not measured. This only prepares the kernel side; whether a Thunderbolt-attached card works on these laptops is untested.
d10d360 to
a918a87
Compare
|
Rebased onto integration/aurora. That branch already has the DMA settings. So this just turns on the AMDGPU and Nouveau modules. I'm leaving it a draft. We still need the build and card tests. |
|
[Joshua's agent] This draft needs an admitted kernel build, boot, and AMD and Nouveau card tests. |
Turns on
CONFIG_UDMABUF=y,CONFIG_DMABUF_HEAPS=yandCONFIG_DMABUF_HEAPS_SYSTEM=yinarch/arm64/configs/asahi.config.The fragment adds three lines. In the resolved configuration the dependency walk also turns on
CONFIG_TEE_DMABUF_HEAPS=y, the TEE-backed heap (TEEitself was already=yand stays=y). Review found no matching platform node on our hardware, so it is inert.This PR is kernel config only. No Mesa or Honeykrisp change ships with it. A consumer must opt in with a memfd,
UDMABUF_CREATEand a dma-buf import.Why
Honeykrisp can already import a dma-buf (
VK_EXT_external_memory_dma_buf, external buffer features 0x6: importable and exportable). The GPU cannot map arbitrary host pages: VM_BIND takes GEM handles only. The missing piece was a way to turn ordinary host pages into a dma-buf.udmabufdoes that for memfd pages.With these options a local AI runtime can keep its weights in one memfd, wrap the memfd with
/dev/udmabuf, and import it into Vulkan. There is one resident copy. There is no staging buffer, and the CPU and the GPU see the same pages.Evidence
A test kernel built from the 12.6 release tree with only these three options added (plus the BTF-mismatch allowance, so the stock modules load) booted on a 14-inch M2 Max MacBook Pro. The test boot was a one-off build of the 12.6 release tree plus only these options. It was not the packaged kernel image. Honeykrisp was a Mesa 26.3 build. The test program ran: memfd,
UDMABUF_CREATE,vkImportMemoryFd, one compute dispatch (d[i] = d[i]*2+1), then the CPU read the result through the original memfd mapping.udmabuf.size_limit_mbto 1024)DMA_BUF_IOCTL_SYNCaround the CPU accessThe GPU result appearing in the memfd pages shows that the driver maps those pages and does not copy them.
Limits found by the test
EINVAL.udmabuf.size_limit_mbdefaults to 64 MiB per buffer. A 512 MiB buffer fails withEINVALat 64 and works at 1024. The parameter is writable at run time through sysfs, and that is what the test did. It should also work asudmabuf.size_limit_mb=on the kernel command line, but I have not tested that. This PR does not change the default.udmabuftakes memfd (shmem) pages only: not a file mapping and not anonymous memory. The pages stay resident, so this does not give larger-than-RAM models.F_SEAL_SHRINK). The driver rejects a memfd that has a write or future-write seal (drivers/dma-buf/udmabuf.c,check_memfd_seals)./dev/udmabufisroot:kvmmode 0660 with a user ACL on the test machine. Access policy is a userspace matter and is not part of this PR.How to check after install
ls /dev/udmabufshows the node.zcat /proc/config.gz | grep UDMABUFshowsCONFIG_UDMABUF=y.Not tested
make defconfig asahi.configgives the three options=y, and that the base head gives# CONFIG_UDMABUF is not setand# CONFIG_DMABUF_HEAPS is not set.F_SEAL_SHRINK. The driver source rejects it, but I did not run it./dev/dma_heap/system(the system heap was built into the test kernel, but the test did not use it).I make no claim about merge readiness. The release builds that would carry this need a normal rebuild of modules and image together.
Second change: AMD and Nouveau GPU drivers as modules
A second commit sets
CONFIG_DRM_AMDGPU=mandCONFIG_DRM_NOUVEAU=min the same fragment, for people who want to try a discrete GPU in a Thunderbolt or USB4 enclosure.Why: PCIe over Thunderbolt already enumerates devices on these laptops (a Thunderbolt 3 dock shows up with its bridges, controllers and network chip behind the PCIe root port). The running kernel has no driver for a discrete GPU:
DRM_AMDGPU,DRM_NOUVEAUandDRM_XEare off, and the helpers amdgpu needs (DRM_TTM,DRM_BUDDY,DRM_SUBALLOC_HELPER) are off as well. So an out-of-tree amdgpu module cannot be built against the shipped kernel either.What I checked, on the Kconfig level only: I merged the fragment into the running 7.1.12 configuration with and without the two new lines and ran
olddefconfigon the same source tree (this branch before the second commit) with the same LLVM 22.1.8 toolchain. Only these symbols change:DRM_AMDGPU=m,DRM_NOUVEAU=m,DRM_TTM=m,DRM_TTM_HELPER=m,DRM_BUDDY=m,GPU_BUDDY=y,DRM_SUBALLOC_HELPER=m,DRM_PANEL_BACKLIGHT_QUIRKS=m,DRM_AMD_DC=y,DRM_DISPLAY_DP_HELPER=y,DRM_DISPLAY_DSC_HELPER=y,DRM_DISPLAY_HDCP_HELPER=y,DRM_DISPLAY_HDMI_CEC_NOTIFIER_HELPER=y,CEC_NOTIFIER=y, the Nouveau sub-options (backlight, CH7006, SIL164, debug levels), and the "is not set" entries of the new sub-options. These are built in rather than modules:GPU_BUDDY,CEC_NOTIFIERand the three display helpers, so the kernel image grows a little.ARM64_16K_PAGES,DRM_ASAHI,DRM_APPLE,PCIE_APPLE,USB4,USB4_APPLE_SOC,PHY_APPLE_ATCandAPPLE_DARTare identical in both results.DRM_AMD_DC_FPdoes not appear in the resolved configuration, so display output is not expected; this is a compute setup.DRM_XEstays off (it needs 4K pages).HSA_AMD,AMDGPU_SI,AMDGPU_CIKandAMDGPU_USERPTRstay off.Not done for the second commit:
DRM_AMD_DCbuilding) is not known.If you would rather keep the udmabuf change alone, the second commit can be dropped without touching the first.
Checks for the second change (AMD and Nouveau modules)
Nothing below has run yet. Each check has a pass line and a fail line so a reviewer can tell when it is done.
m, andamdgpu.koandnouveau.koare in the package for the 16K page kernel. PASS: build exits 0 andmodinfoon both files showsvermagicequal to the kernel release. FAIL: build error or a missing module.vulkaninfo --summary,lsmodshows neitheramdgpunornouveau, anddmesghas no new error lines compared with the current kernel. FAIL: any new error, or either module auto-loads.lspci -vvshows BAR0 and BAR5 assigned inside the bridge window anddmesghas no "can't assign" or "no space" line. If it fails on the 13-inch M1, repeat once on a 16-inch M1 Max before calling the card unusable. This gate decides whether the rebuild and the reboot happen at all.modprobe amdgpu(ornouveaufor an NVIDIA card) prints a VRAM size line indmesgand creates/sys/class/drm/cardN. FAIL: SError, oops, firmware error or a hang while mapping VRAM. Stop on the first one, copydmesg, power the enclosure off. Risk, stated as inference: amdgpu maps VRAM with write-combining, and the PCIe driver says endpoint BARs on the Max laptops fault unless mapped non-posted.VK_DRIVER_FILESset to the RADV (or NVK) ICD only,vulkaninfo --summarylists the eGPU with the expected driver name. FAIL: device missing or crash.Rules for every check: one host, one device, one change at a time; no hot-unplug during compute; no suspend.
This pull request stays a draft until check 3 passes. Check 1 can run on a draft build. Check 2 needs a reboot, so it waits for check 3.