xe: jit: gemm: tile scrambling for improved cold-TLB performance #2607
Conversation
github-actions
bot
added
the
platform:gpu-intel
|
make test |
|
make test perf-gpu |
petercad
force-pushed
the
petercad/scramble
branch
from
9d556d4 to
9455179
Compare
|
make test |
|
make test perf-gpu |
|
Closed in favor of #2631. |
Addresses MFDNN-12523 -- lower than expected performance of memory-bound TN compressed weights kernels on MTL/ARL when TLB is cold. Note that cold TLB can't be benchmarked directly in benchdnn -- even with
--cold-cache=allthe TLB will be warm. However, the driver knobDirectSubmissionNewResourceTlbFlush=1can be used to flush TLB prior to each kernel launch.This PR introduces tile "scrambling" -- reordering tiles in the m and/or n dimensions so that adjacent workgroups are not too close. Specifically, we want to avoid cases where two different workgroups are simultaneously accessing chunks of the weights matrix that belong to the same 64KB page. If this page is not found in the STLB, then both workgroups will sit idle waiting for the page walk to complete. If we must pay the penalty of a page walk to fill the STLB, we would be better off having each workgroup on a different page -- this will fill the STLB faster and reduce the total amount of "dead time" waiting for duplicate page walks.
In more detail, the scrambling algorithm replaces the original group ID (in m or n dimensions) with
(ID * stride) mod #groups, where the stride is chosen to be coprime to #groups (so we don't miss any groups) as well as large enough that adjacent groups don't start on the same 64KB page. Currently, the scrambling is only applied to XeHPG/XeLPG compressed weights TN GEMV kernels, though it may be useful in other cases.