[Model] Gemma 4 M3 31B

[FeathBow]

Description

Parent: #758. Crate: pegainfer-gemma4 · feature: gemma4.

M3 validates the shared text stack at 31B: 60 layers, the same five-to-one local-to-global pattern, a larger hidden size, twice the attention heads, and a different global KV head count. The parent is explicit that this is not checkpoint loading with a bigger file — the global KV head count, the hidden size, the 262144-entry vocabulary and its lm-head cost, workspace sizing and long-context admission each need evidence at this scale rather than extrapolation from 12B.

Proposed Plan

• Serve the published BF16 checkpoint at tensor parallel size 1 on the platform the parent commits this line to, and confirm the manifest test covers the 31B row of the parent's architecture table and fails closed on the variants this line does not serve.
• Re-derive the memory envelope at this scale rather than scaling the 12B one: weights, scales and metadata, allocator reserve, graph and GEMM workspaces, activations, both KV families, and peak observed device memory at the declared context and concurrency.
• Confirm the decode graph bucket set and the split-KV chunk policy still hold once the global family's head count changes, since both were tuned against the smaller geometry.
• Measure the 262144-entry lm-head cost at this hidden size — GEMM, logits buffer, sampler scratch and softcap — and fold it into the envelope rather than treating it as a rounding term.
• Run the same correctness set as the 12B line at 31B, and take a same-host comparison against a pinned vLLM version.

Acceptance Criteria

• Tokenizer and chat-template identifier parity, including special tokens, byte fallback and digit behaviour.
• HF logits and intermediate probes across single token, short prefill, prefill plus decode and mixed batch; sliding-window boundaries at 1023, 1024 and 1025 with multiple-window continuation; local and global alternation including the final-global-layer assertion.
• Batch invariance, KV release and reuse, admission failure and post-failure recovery, and a declared long-context gate matching this line's documented envelope.
• Zero failed requests and output sanity in the retained HTTP concurrency and QPS cells, with the pinned vLLM comparison on the same host and peak observed device memory recorded.
• A retained snapshot with regression thresholds, plus a recipe and hardware-matrix entry with unsupported options failing explicitly.