Enable compact sliding-window attention for Gemma 3 and Gemma 4 variants

[ani300]

Summary

• route supported sliding layers in Gemma 3 and Gemma 4 through spyre::sliding_window_attention
• use the single runtime-mask API for both prefill and decode; no Python position, origin, or valid-start values enter compiled blocks
• allocate prompt-sized sliding caches, then compact them at first decode into bounded window + 64 anchored buffers
• share one fixed-shape runtime mask across all sliding layers, so every decode position reuses the same graph
• preserve chunked prefill, left padding, Gemma 4 PLE, and KV-sharing behavior
• enable the custom SWA path for Gemma 4 VLM masks using the generic non-causal access mode
• match current Transformers VLM mask composition exactly: global layers remain causal and sliding layers use window AND (causal OR vision-blockwise)
• enable compact causal SWA for the text-only Gemma 4 MoE adapter and route its writes through the compact cache index
• handle short prompts whose minimum sliding-cache allocation is wider than the generic prefill mask
• roll compact caches every 64 decode tokens using offset-aware device-to-device copies

The SWA mode remains explicitly disableable for debugging. The existing Gemma 4 multimodal adapter still does not support the MoE VLM checkpoint; the MoE change here applies to the causal-LM adapter.

Dependency

Draft until torch-spyre/torch-spyre#4423 lands. That follow-up to merged #3405 provides the mandatory runtime-mask API, generic non-causal access mode, and kernel/compiler fixes exposed by Gemma 4 D=256 decode.

Compilation behavior

The removed API supplied Python cache_seqlen, buffer_origin, and valid_start values and could compile up to 64 decode-position variants per block. Position, unwritten rows, window placement, padding, and VLM blockwise regions now travel only in tensor data. Runtime-mask tests observe one graph per fixed shape/access mode; prefill and decode therefore retain the expected two shape variants per block.

is_causal is static per prepared model. Text-only Gemma 3/4 and Gemma 4 MoE use the narrow causal square-prefill plan. Gemma 4 VLM uses the generic plan because its sliding mask contains future same-image pairs. Decode still scans only the compact 1088-row allocation.

Correctness

• focused hf-adapters CPU SWA suites: 36 passed
• Gemma 4 Spyre layer A/B suite: 5 passed
• anchored 70-step decode across a cache roll: passed
• upstream mask-order regressions cover future same-image allowance and old same-image window rejection
• companion torch-spyre non-causal correctness and graph-reuse tests: 2 passed
• Gemma 4 E2B CPU-vs-Spyre greedy-token comparison from the preceding revision: 5/5 top-1 agreement, no NaNs
• Black, Ruff, and diff checks pass

The branch is based on upstream main at 1b3c1a6; all commits are DCO-signed and GPG-signed.

Performance

Local causal Gemma-like shape from the preceding revision: B=1, Hq=4, Hkv=2, D=256, W=1024.

path	Lk	warm median
runtime-mask SWA	1088	0.562 ms
masked SDPA	1088	0.677 ms
full-cache masked SDPA	8192	2.692 ms

The compact 1088-column mask build plus transfer measured 146.2 us median, paid once per decode step and shared across all sliding layers. Non-causal VLM prefill intentionally scans the full physical cache because future vision-block endpoints are runtime mask data.