Skip to content

Enable compact sliding-window attention for Gemma 3 and Gemma 4 variants - #496

Draft
ani300 wants to merge 5 commits into
torch-spyre:mainfrom
ani300:gemma4-swa-op
Draft

Enable compact sliding-window attention for Gemma 3 and Gemma 4 variants#496
ani300 wants to merge 5 commits into
torch-spyre:mainfrom
ani300:gemma4-swa-op

feat(gemma4): enable SWA for multimodal and MoE models

bd5b6f2
Select commit
Loading
Failed to load commit list.
DCO / DCO succeeded Sep 10, 2026 in 1s

DCO

All commits are signed off!

View more details on DCO