You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 5bfbb59
Browse filesBrowse the repository at this point in the historyBrowse files
[MLX] Respect each DFlash query's sliding attention window (#25335)
DFlash attention currently exposes the retained context and entire draft
block to every query. Later queries can therefore attend to keys outside
their configured window. Pass the window to MLX attention and make the
eager Muse Glimmer mask use the absolute distance between each query and
key, excluding unfilled ring slots.
Use independent absolute-position oracles for sequence and packed cache
tests, including wrapped windows, partial context, and blocks wider than
the window. Add an asset-independent Python mask regression to MLX CI;
it runs without DFlash2 model additions.
The native window regression fails before the correction and passes
afterward. Cache lifecycle tests pass with strict FP32, and the isolated
Python window test passes all 54 parameter subtests. The official
PyTorch noncausal mask also agrees across 72 independent cases,
including wrapped caches and windows smaller than the block. Both
downloaded original/DFlash2 checkpoints use block 16 and window 2048,
where official DFlash, MLX, sglang, and llama.cpp agree on the strict
past-window bound. Transformers uses an inclusive boundary for the
original draft, a known one-key difference. For synthetic blocks wider
than the window, this change follows official PyTorch symmetric masking
rather than the MLX/llama.cpp whole-block convention. This correction
intentionally changes original DFlash draft numerics, including existing
off-graph artifacts without re-export, while preserving their ABI. It
depends on the MLX pin/API update below it.
The eager mask is also used by CUDA DFlash, so CUDA SWA draft numerics
change. CUDA execution was not tested on this Apple Silicon host; run
the CUDA DFlash checks in CI or on a CUDA host before landing. Native
builds of each staged layer remain for CI; local native results cover
the final implementation.
Tracked in #23011.
Authored with assistance from OpenAI Codex.
0 commit comments