feat: add Gemma4 dense and MoE custom implementations#2190

Open

hallerite wants to merge 7 commits intomainfrom

Commits on Apr 3, 2026

feat: add Gemma4 dense and MoE custom implementations

hallerite
and
claude
committed
Merge remote-tracking branch 'origin/main' into feat/gemma4
hallerite
committed
fix: Gemma4 VLM support, SDPA fallback for head_dim>256, chat template kwargs

hallerite
and
claude
committed
chore: add Gemma4 test configs for SFT and RL

hallerite
and
claude
committed
feat: add Gemma4 tool call parser entries

hallerite
and
claude
committed
style: ruff format modeling_gemma4.py

hallerite
and
claude
committed
refactor: use shared quack RMSNorm for Gemma4

hallerite
and
claude
committed