[joshuaswarren]

This PR replaces #184. GitHub closed #184 when its head branch was deleted by mistake, and it would not reopen after the branch came back. The head commit is the same (3f5c88f). The review comments and GREEN receipts are on #184.

Summary

This series (ten commits) makes an AMD Radeon RX 7600 (Navi 33) in a USB4 enclosure work on an Apple Silicon Mac: amdgpu loads, RADV sees the card, and llama.cpp runs a 7B model on it through Vulkan. It fixes bugs that people hit when they run local AI on an eGPU on these Macs. It is not needed for the release candidate, and there is no rush to review or merge it.

The card was measured on a 14-inch M2 Max MacBook Pro (T6021) with 16 KiB pages. Everything below is either measured (a log or a number from that machine) or not tested.

What is wrong without it

1. The 64-bit BAR window of tunnels 1, 2 and 3 is mapped to the wrong place. The device trees map PCI 0x8_00000000 to CPU 0xc, 0x10 or 0x14 in the high word. The firmware's own tables use identity addressing for every tunnel port (PCI address equals CPU address). Linux puts a 64-bit BAR at 0x8_xxxxxxxx, the fabric forwards the CPU window to 0xc_..., and every CPU read of the BAR returns all-ones. 32-bit BARs and config space still work, so the card looks healthy until a driver touches VRAM.
2. Tunnel BARs are non-posted (Device-nGnRnE) mappings, and the amdgpu paths assume write-combined VRAM. Plain memset(), memcpy() and struct stores into a BO that sits behind the BAR take an arm64 alignment fault in the kernel (PSP command, fence and ring buffers) and in user space (Vulkan driver writes to a VRAM buffer).
3. GTT is cached on a host whose PCI DMA is not cache coherent, so the PSP and the GPU can read stale data.
4. A card whose link trains after the one-time bus scan is never enumerated with the tunnel.
5. DCN does not build for arm64 with clang 22, because of an old compiler guard in DRM_AMD_DC_FP.

What the series does

#	Commit	Effect
1	drm/amd: enable arm64 DCN with clang 22	Kconfig only.
2	drm/ttm, drm/amdgpu: honor non-posted BARs for VRAM, doorbell and ROM access	ttm_bus_placement.is_nonposted; ioremap_np() for VRAM, doorbells, registers, VBIOS.
3	drm/amdgpu: access non-posted BO memory with I/O helpers	amdgpu_bo_memset/memcpy_to/memcpy_from use *_io on non-posted resources.
4	drm/amdgpu: report no CPU-visible VRAM on non-posted hosts	Mesa stops offering a host-visible device-local heap.
5	drm/amdgpu: place PSP CPU-visible buffers in GTT behind non-posted BARs	Fixes the alignment faults during PSP init.
6	drm/amdgpu: map GTT write-combined on non-coherent non-posted hosts
7	drm/amdgpu: move a CPU-faulted VRAM BO to GTT behind a non-posted BAR	Fixes the SIGBUS in the Vulkan driver.
8	arm64: dts: apple: map USB4 PCIe tunnel 64-bit windows with identity addressing	Tunnels 1, 2 and 3 of T600x and T602x.
9	PCI: apple: wait for tunneled slot links before scanning	Link wait before the bus scan.
10	PCI: apple: enable the root-port prefetchable window on cold-initialized tunnel ports	One line. Measured by another tester on a 16-inch M1 Max (T6001) with an Intel Arc Pro B60 in a USB4 enclosure: PORT_PREFMEM_ENABLE read 0 and every read through the root port's prefetchable window returned all-ones; one write of 1 made them return data. The hook path itself has not been booted as a kernel. On the 14-inch M2 Max the register also reads 0, but it is a write strobe (it reads 0 after a write), so that read says nothing; the M2 works with identity ranges alone.

The TTM and amdgpu changes are conditional on IORESOURCE_MEM_NONPOSTED on BAR 0 (commit 6 also needs a non-coherent device), with one exception: the GFX v11/v12 queue setup hunks in commit 3 (kgq, kiq, kcq) now compose the MQD in the host backup copy and copy it to the BO whenever a backup exists, on every host. That order is meant to be equivalent. It has only run on the non-posted host above, so it needs a run on a posted host (an x86 AMD GPU) before merge. The MES v12 MQD paths (RDNA 4) are not converted. psp_cpu_buf_domain() has no override: on a non-posted host the PSP buffers are always GTT.

Measured on the M2 Max (RX 7600, Mesa RADV 26.2.5, llama.cpp build 65840ed53)

The first runs used one CPU of the machine's 12. The device tree that made BAR 0 readable was loaded from the boot loader with a saved copy of the live device tree. That copy holds spin-table release addresses that change at every boot, so the other 11 cores failed to start. The numbers below were taken that way unless a line says 12 CPUs. The identity ranges ship properly inside the boot.bin that m1n1 merges at boot; I then rebuilt boot.bin with the same 33-byte change (tunnels 1-3 of the 11 DTBs that have them), booted with all 12 CPUs, and reran the same commands:

model (llama-bench, pp512/tg128, n=3, all layers on the GPU)	1 CPU, fa 0	12 CPUs, fa 0	12 CPUs, fa 1
Qwen2.5-3B Q4_K_M	1747 / 28.54	1880 / 35.34	1996 / 39.76
Qwen2.5-7B Q4_K_M	968 / 31.65	946 / 28.93 (29.5 on repeats)	968 / 28.90 (31.4 to 31.9 on repeats)
Llama-3.1-8B Q4_K_M	862 / 23.04	879 / 25.79	945 / 28.42
Qwen3-8B Q4_K_M	825 / 19.49	856 / 22.88	907 / 23.82

More CPUs help the small model a lot and the 7B not at all (decode at 1, 2, 4, 8 and 12 CPU threads is 29.6 t/s in each case, on P or E cores alike). So decode on the 7B is limited on the GPU side, not by the CPU. The 7B number moved between 28.9 and 31.7 t/s across runs and boots; I do not know why, and the earlier 31.65 is the high end, not a better configuration.

• egpu VRAM self-test: 0 mismatches in 4 KiB through MM_INDEX, 512 qwords through BAR 0, 64 KiB writeq/readq and memset_io/memcpy_toio; PSP, SMU 82.97.0 and DMUB initialise; every ring registers; [drm] Initialized amdgpu 3.64.0. No amdgpu error, fault or alignment line in dmesg after the benchmark runs.
• vulkaninfo --summary: AMD Radeon RX 7600 (RADV NAVI33), conformance 1.4.5.3, 8 GiB device-local heap.
• GGML_VK_VISIBLE_DEVICES=0 llama-bench -m qwen2.5-7b-instruct-q4_k_m.gguf -p 512 -n 128 -r 3 -ngl 99: pp512 967.77 ± 1.09 t/s, tg128 31.65 ± 0.03 t/s (one CPU; see the table below for 12 CPUs). llama-cli --temp 0 -p "The capital of France is" answers "Paris."
• Before the DTS change every read of BAR 0 returned 0xffffffff; after it BAR 0 reads return data.
• Without commit 7 the first llama.cpp run died with SIGBUS (BUS_ADRALN) inside memcpy in the Vulkan driver.

The run above used the same code on the 7.1.12-based tree this was developed on (same logic; the firmware-copy sites differ because this tree already routes them through psp_copy_fw). The series in this PR is rebased on integration/aurora; on this tree I compiled the touched objects with LLVM 22 (W=1) and built the Apple DTBs; I have not booted it on hardware. Compile: one warning, dml_core_mode_support stack frame 2080 > 2056 in display_mode_core.c, which appears because commit 1 turns on DC_FP for arm64. A build with CONFIG_WERROR=y fails on it; I do not know which setting the package build uses.

Decode speed against the llama.cpp Vulkan thread

The llama.cpp Vulkan benchmark thread (ggml-org/llama.cpp discussion 10879) lists an RX 7600 at pp512 1167.20 and tg128 58.03 t/s for Llama 2 7B Q4_0 (commit 91f8c9c, a much older build than the 65840ed53 used here). The same command on this eGPU (RADV_PERFTEST=nogttspill llama-bench -m llama-2-7b.Q4_0.gguf -ngl 100 -fa 0,1 -r 3, 12 CPUs): fa 0 pp512 1005.43 ± 1.80, tg128 29.26 ± 0.03; fa 1 pp512 1226.01 ± 0.38, tg128 32.88 ± 0.01. Against the listed value that is 86 % of the prompt speed and 50 % of the decode speed. I have not run a native RX 7600 with the current llama.cpp, so the real gap today is unknown.

What I tested for the decode gap, with results:

• CPU: 1, 2, 4, 8 and 12 threads, P or E cores: the same 29.4 to 29.6 t/s on the 7B.
• Clocks: forced high DPM, sam, nogttspill: no change (under 2 %).
• Where the tensors live: weights, KV cache and compute buffers are all device VRAM; the only GTT buffers are llama.cpp's own host staging buffers.
• Descriptor memory: a probe that cycles 1 to 5000 distinct descriptor sets shows the same 6.5 µs per dependent dispatch, so there is no cold-descriptor penalty.
• Kernel time: GPU 100 % busy, memory busy 3 %; the large matvecs read 170 to 208 GB/s (peak 288); about 9 ms of the 30 ms token is spent in small ops (norm, rope, set_rows) that cost about 22 µs each, against 4 µs in test-backend-ops. I do not know why.
• One switch helps: GGML_VK_DISABLE_MMVQ=1 gives +8 % decode (29.86 to 32.24 t/s on the Q4_0 model). I have not tried it on a native card.

None of this is a kernel finding, and nothing here changes with this series.

Conditions before merge

1. Run the GFX v11/v12 queue-setup hunks of commit 3 on a posted host (an x86 AMD GPU), because they are not conditional on a non-posted BAR.
2. Boot a kernel with commit 10 on the 14-inch M2 Max (T6021), where the 64-bit window already worked, to show that the extra PORT_PREFMEM_ENABLE write does no harm there.
3. Boot a kernel with commit 10 on a 16-inch M1 Max (T6001); only the register write has been measured there, not the hook path.

Not tested

• Live cable hot-plug (unplug and replug while Linux runs).
• A display on the card. A CPU fault on a pinned VRAM buffer (scanout, cursor) now fails with SIGBUS on non-posted hosts (commit 7).
• Boot cost of commit 9: it waits up to 1 s for each empty slot port of a tunneled switch, one after the other.
• Tunnels 2 and 3 on T600x and T602x, and every Mac other than the M2 Max. Their ranges follow the same rule as tunnel 1; tunnel 0 already used it.
• T6001: a test on a 16-inch M1 Max found that identity addressing alone does not make a 64-bit window work behind the root port there; that is a separate root-port issue and is not changed here.
• 8 GiB resizable BAR (the request fails and PCI restores 256 MiB; the driver continues).
• A display on the card, vkcube, Navi 48 (RDNA 4), and cards other than the RX 7600.

Config and firmware notes

• Build with clang 22 or newer for arm64 DCN (DRM_AMD_DC_FP).
• The card needs the usual amdgpu/ firmware set from linux-firmware for Navi 33 (PSP, SMU, SDMA, GC, VCN, DMCUB).
• Needs the identity device-tree ranges (commit 8) at boot; Linux builds that load a replacement device tree must carry the same ranges.

Update (2026-10-29): second card (RX 9070, RDNA 4) and surprise unplug/replug

Tested on hardware, on a lab build of 7.2.11 (release string ending in -7f91446; tree commit 7f91446be on base 7876c8e83, an ancestor of integration/aurora), carrying the amdgpu/ttm commits of this series (7 of the 10), an unrelated GPU statistics export, and a lab kernel config. This is not a hardware test of the current PR head: the three PCI/DTS commits (62f8a9ea, a43f1e95, 654075e5) were not in the booted tree, and its device trees come from the base. The rebuilds have since run: the ten-commit head and the current eleven-commit head (with the MQD restore-flush commit) both boot and compute on the same machine with identical results and identical output hashes; see the two hardware follow-up comments (issuecomment-6508496398 and issuecomment-6509593634).

Measured on that kernel, one boot, both cards:

• AMD Radeon RX 9070 (Navi 48, DCN 4.0.1) initializes next to the RX 7600: SMU, DMUB and MES up, every ring registered, [drm] Initialized amdgpu 3.64.0 on both minors. Zero SError / alignment-fault / GPU-hang lines across every run of the day, including the hotplug below.
• RADV 26.2.5 enumerates both cards next to the integrated Apple GPU.
• A Vulkan numerics contract passed gates 1-4 on both cards and caught 13 of 13 deliberate-control kernels.
• A 2B-model Vulkan decode (63 steady tokens, two runs per card) returned token ids equal to the on-Apple-GPU reference on both cards, with one identical result hash across all four runs and the reference. Steady median 19.6 ms/token (RX 7600), 27.7 ms (RX 9070). The kernel reports each card's own link at full width and speed; that does not rule out the USB4 tunnel or the enclosure, and the cause of the token-time gap is unmeasured.
• Surprise unplug and replug, one cable at a time, driver loaded, machine idle: lanes retrained to CL0, the enclosure re-enumerated, PCI rescan restored the card, amdgpu rebuilt its rings and re-initialized the minor, RADV re-enumerated, and the same decode still returned the same hash and matching ids. No reset was needed at any point.

So the earlier "hot-plug untested" and Navi 48 gaps in the table above are now measured on the kernel described above (idle machine, one cable at a time). An earlier run on the same machine showed the stock tree initializing neither card.

[joshuaswarren]

This PR replaces #184 (same head 3f5c88fa569e2ea216a8896c06107346068c57a5, same ten commits, same text).

Review record, all on #184:

• GREEN at de83db09e and GREEN at 3f5c88fa5 (read-only reviews by hash).
• Review pass 1 and pass 2 by independent reviewer agents: no code change.

Still open before merge: a second human or maintainer GREEN, a maintainer "Approve and run" for CI, and the conditions listed in the description (MQD hunks on a posted host, commit 10 on a second SoC).

[joshuaswarren]

GREEN at 3f5c88f
Same head as PR 184, where my review is: #184 (comment)

[greptile-apps]

Confidence Score: 5/5

[Medium impact] No new blocking defect was found, and the previous direct-firmware fault is addressed.

Summary

Adds non-posted GPU memory access and Apple USB4 tunnel fixes for external GPUs.

• Apple Silicon tunnel windows and PCI scans now let tunneled GPUs be found and reached.
• AMDGPU uses safe CPU access and memory placement for non-posted GPU memory.
• ARM64 display support now builds with clang 22 and newer.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Non-posted GPU BAR] --> B[Map VRAM with non-posted access]
  B --> C[Kernel copies use I/O helpers]
  A --> D[PSP CPU buffers use GTT]
  A --> E[Report no CPU-visible VRAM]
  E --> F[CPU fault on movable VRAM buffer]
  F --> G[Move buffer to GTT]

Loading

_{Reviews (3) · Last reviewed commit: "PCI: apple: enable the root-port prefetc..." · Reviewed by Greptile}

[joshuaswarren]

GREEN at 3f5c88f.

I read the whole diff of this head (25 files, ten commits) and not only the history of PR 184, which carried the same head. I did not build or boot it. My checks are reading only, plus the receipts named below.

The non-posted changes are all gated on the BAR flag. In TTM, the new is_nonposted field is reset in ttm_resource_init and set only for VRAM, doorbell and register-remap placements. In amdgpu, the memset and memcpy helpers choose the I/O variants from the same flag. The PSP buffers, the DMUB buffer and the CPU-faulted VRAM move all take the GTT path only behind that flag. The CPU-fault path returns SIGBUS for a pinned buffer and for a failed move, and it retries on -EBUSY and -ERESTARTSYS. The device-tree change swaps the PCI-side window address for tunnels 1 to 3 to match the CPU address in both t600x and t602x, with the same three lines each. The link wait in apple_pcie_add_bus is bounded, skips hot-plug capable ports and returns 0 on every path. The DCN Kconfig line only changes the arm64 clang case.

Notes, none of them blocking:

1. The description of commit 10 says the hook path has not been booted as a kernel. That is out of date. The one-line change in apple_pcie_config_write (kernel_init || tunnel_cold_init) was built into a full kernel and booted on a 16-inch M1 Max MacBook Pro with an Intel Arc Pro B60. I compared the booted patch with the hunk in this diff, and the changed line is identical. On a fresh boot the card's registers and VRAM read correctly with no manual register write. Please update that sentence.
2. The GFX v11 and v12 queue hunks already note that the compose-in-host-copy order needs a run on a posted host. One more difference there: on a host without the non-posted flag, several paths that used memcpy_toio or memcpy_fromio now call plain memcpy through amdgpu_bo_memcpy_to and amdgpu_bo_memcpy_from. That is the gfx11 reset and restore branches and the backup copies. It is probably harmless on x86, but it is a behaviour change on posted hosts, so please say so in the description when you ask for that test.
3. The MES v12 MQD paths are still not converted, as the description says. I did not look for other plain-pointer writers into VRAM buffers outside the files in this diff.

Nothing here was run by me, so this is a read review. The measured results in the description are the author's.

[joshuaswarren]

New head 11fd43ba18a4b3a093cd06dc209007094fa05bc3 (two commits changed; both Greptile threads of the previous head have a reply with the commit and are resolved): the DMUB buffer goes to GTT behind a non-posted BAR only when the firmware is loaded by the PSP (drm/amdgpu: access non-posted BO memory with I/O helpers), and the ROM read through the BAR copies only the image, using the 0x55 0xAA signature and the length byte of the ROM header (drm/ttm, drm/amdgpu: honor non-posted BARs for VRAM, doorbell and ROM access).

Receipt at this head (CT-local, arm64 clang W=1, 16K pages, DRM_AMDGPU=y, DRM_AMD_DC=y): drivers/gpu/drm/amd/amdgpu/, drivers/gpu/drm/ttm/ and drivers/gpu/drm/amd/display/ rc 0; one warning, display_mode_core.c:6894 stack frame size 2080 > 2056 (existing code, not in this PR). The patches apply on current integration/aurora. Hardware: the earlier form of this logic ran on the M2 Max with the RX 7600 (amdgpu loads, DMUB hardware initialized: version=0x07003300 with PSP load); the ROM change was compiled, not run on hardware. Not run: a boot of this head.

[joshuaswarren]

New head 654075e59744da60d1cdf876048dbf38b588ff3d (the DMUB commit changed again, see the thread above). Receipt (CT-local, arm64 clang W=1, 16K pages, DRM_AMDGPU=y, DRM_AMD_DC=y): amdgpu/, ttm/ and display/ rc 0; the only warning of the full build, display_mode_core.c:6894 (stack frame 2080 > 2056), is existing code. Not run: a boot of this head.

[joshuaswarren]

GREEN at 654075e for the delta from 11fd43b and the two areas w7P named; the rest of the series was not re-read in this pass.

Read: the ten commits of this head against the ones at 11fd43b, compared per commit as diffs with the hash lines removed. Nine commits are identical. The one that differs is "drm/amdgpu: access non-posted BO memory with I/O helpers", in dm_dmub_sw_init:

• Direct (non-PSP) DMUB firmware loading behind a non-posted BAR is now refused up front with -EOPNOTSUPP and an error message, before any allocation. The reason in the comment is right: the backdoor loader converts addresses as VRAM addresses and fills the buffer with plain CPU copies.
• The buffer placement is then simply GTT whenever BAR 0 is non-posted (the PSP-only condition moved into the early refusal). PSP loading is unchanged, and the author reports DMUB initialised on the RX 7600.
  Consequence to know: on a non-posted host with a direct-load ASIC, DM init now fails instead of continuing in a mode that could not work.
  ROM copy (in "honor non-posted BARs ..."): for a non-posted ROM resource the code assigns the ROM BAR if needed, enables it, maps with ioremap_np, checks the 0xaa55 signature (unmapping and disabling on failure) and copies only the image length from byte 2 (units of 512 bytes), falling back to the window size when that byte is zero or larger than the window. Reads of the header use readw/readb on the I/O mapping. I did not check the matching release path.
  Not run: build or the card; the author's compile and hardware receipts are not reproduced. Merge still needs CI and a second non-author GREEN.

[joshuaswarren]

GREEN at 654075e

Read the whole diff of the ten commits (25 files, +447/-111), including the DMUB hunk that changed since 11fd43ba. Everything outside the PCI and device tree files is gated on pci_resource_flags(pdev, <BAR>) & IORESOURCE_MEM_NONPOSTED, which only the Apple host bridge sets, so other hosts take the old code.

Checked

• DMUB (changed hunk). Behind a non-posted BAR the DMUB framebuffer is allocated in GTT only, and direct firmware loading returns -EOPNOTSUPP with an error message (the backdoor loader converts addresses as VRAM addresses and fills the buffer with plain CPU copies). adev->firmware.load_type is fixed in the early init, before dm_dmub_sw_init(). PSP loading is unchanged.
• TTM and resource plumbing. bus.is_nonposted is cleared in ttm_resource_init() and set in amdgpu_ttm_io_mem_reserve() from BAR 0 (VRAM), BAR 2 (doorbell) and BAR 5 or 2 (register remap). ttm_kmap_iter_linear_io_init(), ttm_bo_ioremap() and ttm_bo_vmap() use ioremap_np() (no write-combine and no cached fallback for such resources); ttm_io_prot() returns pgprot_noncached. ioremap_np() returns NULL on architectures that do not define it, and none of this runs there.
• Accessors. amdgpu_bo_memset, amdgpu_bo_memcpy_to and amdgpu_bo_memcpy_from switch to memset_io, memcpy_toio and memcpy_fromio when the BO's resource is non-posted and keep the plain call otherwise. The converted sites (PSP firmware and RAS buffers, SMU tables, MES, SDMA7 and GFX11/12 firmware, CSA, MES event log) pass the BO that owns the pointer.
• MQD paths (gfx11, gfx12, MES11). The MQD is composed in the host backup copy, copied to the BO with the accessor, flushed through the HDP, and ring->mqd_ptr is restored. I checked that gfx_v11_0_kiq_init_register() only reads the MQD, so the order (compose, copy, init registers) leaves the same device and backup contents as the old order (compose, init registers, copy back).
• GTT and faults. A CPU fault on a VRAM BO moves it to GTT permanently (pinned BOs get SIGBUS, a failed move SIGBUS, -EBUSY and -ERESTARTSYS retry); GEM refuses CPU_ACCESS_REQUIRED VRAM there; the info ioctl reports no CPU-visible VRAM; GTT pages are mapped write-combined when the device is not DMA-coherent; PSP buffers go to GTT.
• PCI and device tree. The add_bus wait applies only to a downstream port with a slot, no hot-plug capability, link-active reporting and a tunneled controller; it waits up to the tunnel ready time for the data-link-layer bit, sleeps PCIE_RESET_CONFIG_WAIT_MS after the link comes up, and returns 0 either way. The PORT_PREFMEM hook now also runs for tunnel_cold_init. The tunnel 1, 2 and 3 windows become identity (0x43000000 0xc 0x0 0xc 0x0 ...), the same form the T8152 ADT has for its tunnel (PCI address equals CPU address).

Findings (not blocking)

1. The measured coverage is one card (RX 7600, gfx11 on T6021). The gfx12, MES12, MES12.1 and SDMA7 edits use the same pattern and are not exercised.
2. The conversion is by site, not by audit: plain memset/memcpy on BO pointers in other IP blocks (VCN, JPEG, others) are not touched. A grep over amdgpu/ for BO CPU pointers that are not routed through the helpers would show whether another block faults the same way.
3. The two dtsi hunks for T600x and T602x are checked by reading (the form matches the ADT of the other machine), not by a schema run.

Not checked

Compile and checkpatch receipts (the author's), hardware, and the DCN build with clang 22 (first commit, Kconfig only). Merge still needs CI and the owner's word.

[joshuaswarren]

Status at head 654075e59744da60d1cdf876048dbf38b588ff3d: two independent reviews GREEN (one scoped to the delta since the earlier head), all Greptile threads resolved, amdgpu/ttm/display objects compile rc 0 (arm64 clang W=1). Measured only on the RX 7600 (gfx11); gfx12, MES12 and SDMA7 edits are not exercised. Not done: CI. Needs a maintainer's CI approval and merge. Not needed for the release candidate.

[joshuaswarren]

Measured proof on hardware (2026-10-29), with the exact kernel provenance up front: this ran on a lab build of 7.2.11 (release string ending in -7f91446; tree commit 7f91446be on base 7876c8e83, an ancestor of integration/aurora), carrying the amdgpu/ttm commits of this series (7 of the 10, plus an unrelated GPU statistics export and a lab kernel config). This is explicitly NOT a hardware test of the current PR head 654075e5: the three PCI/DTS commits (62f8a9ea, a43f1e95, 654075e5) were not in the booted tree, and its device trees come from the base. A rebuild of the integration tip with the full ten commits will be tested separately. Everything below was measured on that booted kernel.

Machine: 14-inch M2 Max MacBook Pro (T6021). Two cards in USB4 enclosures, both enumerated at boot:

• AMD Radeon RX 7600 (Navi 33, DCN 3.2.1)
• AMD Radeon RX 9070 (Navi 48, DCN 4.0.1), which extends the tested matrix past Navi 33 to RDNA 4

Measured, per card, on the same boot:

1. Init: SMU, DMUB and MES come up (CP_MES_CNTL programmed), every ring registers, [drm] Initialized amdgpu 3.64.0 on both minors. Zero SError / alignment-fault / GPU-hang lines in dmesg across every run of the day, including the hotplug below.
2. RADV 26.2.5 enumerates both cards next to the integrated Apple GPU (three Vulkan devices).
3. Numerics contract (TensorFold Vulkan contract binary): both kernels pass gates 1-4 on both cards; 13 of 13 deliberate-control kernels caught (the contract's own fault-injection checks). Gate time 69.1 s (7600) / 75.6 s (9070).
4. 2B model decode through Vulkan (63 steady tokens, two runs per card): token ids equal the on-Apple-GPU reference on both cards in both runs, and the result hash is identical across all four runs and the reference (6bfbb1a5...). Steady median 19.6 ms/token (7600), 27.7 ms (9070). The kernel reports each card's own link at full width and speed; that does not rule out the USB4 tunnel or the enclosure, and the cause of the token-time gap is unmeasured.
5. Surprise unplug + replug, driver loaded (one cable at a time, queue idle): Thunderbolt lanes retrain to CL0, the enclosure re-enumerates, PCI rescan picks the card up, amdgpu rebuilds all rings and re-initializes the minor, RADV re-enumerates, and the same decode still returns the same hash and matching ids. No reset was needed at any point.
6. RDNA 4 note: the RX 9070 went through the gfx v12 queue paths with the folded MQD ordering (compose in the host backup, copy to the BO, HDP flush, then program registers). The "MES v12 MQD paths not converted" caveat in the description did not block init, compute, or hotplug on this card; I am not claiming more than that observation.

Baseline: an earlier run on the same machine showed the stock integration tree initializing neither card (BAR 0 reads all-ones; PSP alignment faults), which is the "before" state this series removes.

A stock user needs this series merged, amdgpu enabled in the config (#67), the vulkan-radeon userspace package, and an explicit modprobe amdgpu (the default distro config blacklists it on these Macs).

[joshuaswarren]

Follow-up, as promised at the top of the earlier comment: the full series at this PR's exact head 654075e59744 (all ten commits) has now been booted and tested on hardware.

Kernel: integration/aurora tip 2f725546d plus the ten commits of this PR (cherry-picked clean) plus one unrelated GPU statistics commit (a copy of PR #281's content); release string ending in -e54bd23. First boot, no reset. Identity check on the booted machine: 151 of 151 loaded modules from this kernel's own module tree, uniform vermagic, zero module BTF rejections, Wi-Fi up at about +60 s.

Measured, same machine, same boot, both cards:

• modprobe amdgpu returns 0; both cards bind, SMU initializes on both; zero SError / alignment-fault / GPU-hang lines.
• vulkaninfo lists both cards next to the integrated Apple GPU.
• The same numerics contract as before: 2 kernels pass gates 1-4 on both cards; 13 of 13 deliberate-control kernels caught. Gate time 68.4 s (RX 7600), 75.6 s (RX 9070).
• The same 2B decode (63 steady tokens, two runs per card): token ids equal the on-Apple-GPU reference on both cards and runs, and the result hash is identical to the earlier run and the reference (6bfbb1a5...).

So the ten-commit series, exactly as it stands in this PR on the current integration tip, boots and computes end to end, and reproduces the earlier results byte for byte. The physical unplug/replug coverage remains the run described in the earlier comment (idle machine, one cable at a time, seven-commit kernel); it was not repeated on this boot.

[joshuaswarren]

GREEN at 83994dd for the new commit 83994dd3c226 (my earlier PASS covered 654075e5; the new head adds only this commit).

Checked from the tree at that head:

• The diff touches gfx_v11_0.c and gfx_v12_0.c only. Each of the six restore copies (kgq, kiq, kcq in each file) gets braces and amdgpu_device_flush_hdp(adev, NULL) directly after amdgpu_bo_memcpy_to. That is the same call the init copies use. No other line changes.
• A search for amdgpu_bo_memcpy_to at this head finds no MQD copy in the gfx files without a flush after it. The remaining unflushed copies are firmware and data uploads, outside this change.
• The commit message claim about MES holds. In mes_v12_0.c and mes_v12_1.c the MQD BO is allocated with AMDGPU_GEM_DOMAIN_GTT only. In mes_v11_0.c it is VRAM | GTT. The gfx MQD BOs use VRAM | GTT (amdgpu_gfx.c, amdgpu_gfx_mqd_sw_init), so the restore flush applies to them.
• The commit message claims parity with the init paths and nothing stronger.

CI at the time of this comment: approve and CI self-test pass; the host test jobs were still running.

Not checked: I did not compile the objects (the author reports a clean build with 0 warnings), and the reset and suspend restore paths were not run on hardware. The PR text says so.

[joshuaswarren]

Third hardware data point, at this PR's current head 83994dd3c226 (the eleven-commit series including the MQD restore-flush commit):

Kernel: integration/aurora tip 2f725546d plus the eleven commits plus one unrelated GPU statistics commit; release string ending in -965d3d6. First boot, no reset. Identity check: 151 of 151 modules from the kernel's own tree, uniform vermagic, zero module BTF rejections, Wi-Fi up.

Same measurements as before, same boot, both cards:

• modprobe amdgpu returns 0; both cards bind, SMU initializes on both; zero SError / alignment-fault lines.
• vulkaninfo lists both cards next to the integrated Apple GPU.
• Numerics contract: 2 kernels pass gates 1-4 on both cards; 13 of 13 deliberate-control kernels caught (gate time 68.1 s and 75.9 s).
• 2B decode: token ids equal the on-Apple-GPU reference on both cards; result hash 6bfbb1a5... again — identical across all three kernels this series has now booted on (the seven-commit tree, the ten-commit head, this eleven-commit head) and the Apple-GPU reference.

Restore-path coverage (reset and suspend resume): NOT TESTED on hardware. The debugfs recovery trigger is not writable on this build (mode 0444 for root, module parameter gpu_recovery=-1), and no safe alternative trigger was available in the test window. The flush commit is compile-verified (clang 22, the exact build commands, zero warnings) and mirrors the init-path pattern, but its code path has not run.

[joshuaswarren]

The series is now rebased onto integration/aurora tip 2f725546d; head f3ea195773f9. git range-diff against the previous head shows all eleven patches unchanged (= on every commit): same content, current base.

The base is the one the hardware runs used: the kernel booted on the 14-inch M2 Max tonight (release string ending in -965d3d6) was integration/aurora 2f725546d plus exactly these eleven commits plus one unrelated GPU statistics commit. So the hardware results in the three comments above — both cards, three boots, identical output hashes, physical unplug/replug) were measured on this tree modulo that unrelated commit, and they carry over to this head unchanged. A no-eGPU regression run (boot plus the test suite on Macs without an eGPU, against a control at the base) is in progress; its comparison will be posted here.

[joshuaswarren]

GREEN at f3ea195 (rebase of 83994dd onto integration/aurora 2f72554; my earlier GREENs covered 654075e5 and 83994dd3 on the old base).

Checked:

• The PR has 11 commits before and after. The patch-id of the whole PR diff (merge base to head) is the same on both bases: 7b01f645eadaac583c1d2b5c3e264d47f26314c3. The changed-file list is the same 25 files. So the rebase moved the 11 commits without changing their content.
• The only differences between the old and new heads over those files are in amdgpu_kms.c and amdgpu_dm.c. Both come from the base (3050 commits moved between the two merge bases), not from the PR. The patch-id match shows the PR's own changes in them are unchanged.
• The commit list is the series I reviewed: the TTM and amdgpu non-posted BAR changes, the three PCI and device-tree commits, and the HDP flush commit last.

CI at the time of this comment: compile, device trees, the three host-test jobs, commits, CI self-test and approve pass. KUnit and package were still running. GitHub reports the merge state as blocked, which is a branch rule beyond these checks.

Not checked: the hardware run on the rebased tree (the author reports the same tree minus a stats commit passed on the 14-inch M2 Max), and the two checks still running.

[joshuaswarren]

No-eGPU regression check, maclab: PASS.

The PR head f3ea195773f9 (the eleven commits on integration/aurora 2f725546d) was booted and run through the test suite on three maclab Macs (M3, Neo, and a third unit), against a control build of the plain base 2f725546d on the same machines. The diff of failure ids between the PR build and the control is empty on all three: the series changes nothing on Macs without an eGPU. An earlier maclab attempt against the pre-rebase head ran on a stale merge base without the current Apple GPU and display stack and produced failures attributable to that base; those runs are not evidence about this series and are excluded.

[chriskearney]

Approve for integration/aurora at f3ea195773f9508f1aa4c8ff79f8336ec074d45d.

I reviewed the 25-file diff and the critical MQD, BIOS/ROM and TTM paths. An additional independent source pass found no concrete blocker in the unconditional GFX v11/v12 changes: composing in the host backup preserves the initialized MQD contents; both KIQ register-setup functions only read them; the restore copies gain the HDP flush. I also independently checked that the pre-rebase eleven-commit series and this PR have the same stable patch-id (7b01f645eadaac583c1d2b5c3e264d47f26314c3).

Code-owner disposition of the description's condition 1: I explicitly waive the posted-x86 hardware run for this ARM64-only integration merge. That test has not passed or been supplied. Posted-VRAM x86 MQDs are outside linux-aurora's targets; ARM64 GFX/compute MQDs use GTT. This is a scoped source-review decision, not an upstream/x86 qualification or a waiver of GitHub checks. A broader port still needs its own validation.

Evidence and limits:

• Exact-head CI is green. The package configuration does not build AMDGPU/TTM, so that CI does not compile-cover those changes or enable eGPU support by itself.
• The linked RX 7600/RX 9070 boot-and-compute results are author-reported, on the patch-identical series plus a statistics change and lab configuration. I did not independently reproduce those runs. Physical unplug/replug was on the earlier seven-commit kernel at idle, not repeated on the final head.
• GPU reset/suspend restore remains untested. The bounded PCI link-wait boot cost is unmeasured. The three-Mac no-eGPU comparison is also author-reported. None is a full release or cross-platform qualification.

These documented residuals are acceptable for this integration change. Merge remains through the required code-owner/CI/merge-queue rules.