queue.depth: G17P stalls under deep mixed render+compute on one queue (driver trigger not yet found) #5

Activity

1. DjDeveloperr commented on Oct 29, 2026

   

   DjDeveloperr

   on Oct 29, 2026

   ContributorAuthor

   More actions

   Fixed by aurora-silicon/linux#571 (merged as 3a83be6e on integration/aurora; omacom mirror omacom/linux-aurora#276, stacked on #275).

   Native comparison (Neo, macOS 26.6.2, same firmware)

   A Metal copy of the c128+r64 phase. One MTLCommandQueue keeps 128 compute command buffers in flight (a 4×64 chain dispatch each) and 64 render command buffers. Each render clears a 64×64 target, draws a scissored triangle and copies the target to a buffer. Every result is verified.

   Run	Result
   blit copy, 15 s	pass, 733k command buffers (48.9k/s)
   compute-kernel copy (hk's render-then-compute shape), 15 s	pass, 48.0k/s
   untracked resources, 15 s	pass, 46.9k/s
   default queue limit, 15 s	pass, 45.4k/s
   blit copy, 60 s	pass, 2.76M command buffers (46.1k/s)
   two processes at once, 15 s	both pass (23.5k/s each)

   Every run had 0 errors, and AGX recoveryCount stayed 0.

   How Metal schedules, from scheduled/completed handler times, kernel and GPU timestamps, and a long head command buffer with 191 behind it:

   • 36 command buffers per queue between kernel scheduling and completion. That holds for compute-only, render-only and mixed loads, and behind compute or render heads. Two queues hold 72.
     • commit() never blocks; the backlog waits in the driver.
     • In steady state the window stays full (scheduled-not-completed p50 32), and commit→scheduled is ~3 ms.
   • Scheduling is in commit order across engines, and completions arrive in order.
   • No other render/compute ordering. With a light head command buffer, later compute runs during an earlier render and vice versa; renders stay in order.
   • One queue's compute command buffers are spread round-robin over three compute channels. Behind a long compute head, every third waits for it.

   Linux

   Each engine admitted a context's ioctls in submission order, but render and compute advanced independently. Render admission took work-state nodes for ioctls far ahead of an older ioctl still waiting on the compute queue.

   Each rule was tested as a diagnostic on the c128+r64 repro:

   Rule	Result
   native depth alone (36 over unretired ioctls)	firmware crash at 0.25 s
   36 over node-holding ioctls	firmware crash at 7 s
   node caps 32/48/56 (from #570)	stall
   ordered admission across engines, strict or with run-ahead up to 16	30 s passes
   ordered admission, run-ahead 32	stall at 6.6 s

   #571 admits an ioctl only while at most 8 older ioctls of its queue are still waiting for their node. Strict order also passes, but there it costs a scheduler round trip per refusal, which showed up in a windowed Vulkan client. Native avoids that cost because it schedules at commit on one thread.

   Qualification on a fresh boot:

   • regression gate PASS;
   • targeted CTS 4687/0;
   • full synchronization2 36516 pass / 0 non-pass;
   • repro 4 × 30 s plus a 60 s run, 0 errors and no GPU kernel lines;
   • KUnit 153/153;
   • same-boot perf A/B within noise or better on every row: gpubench light/heavy/copy/queues/concurrent/sync, rt.scene Solar Bay, vkmark through Hyprland;
   • the old unordered admission still stalls on the same boot.

   queue.depth should pass on kernels with #571. The separate CTS-only stampless compute retirements (~9.7k per synchronization2 pass) are unchanged and remain a separate item.
2. ludagoo

   added a commit that references this issue on Oct 31, 2026

   drm/asahi: g17: admit a context's ioctls to work states in submission…

   ...

   1d3726b