feat(gemma): qualify gemma-3-270m and refuse fp16 for Gemma 3

[zhenshanx-nv]

Background

#1304 added the Gemma 3 text path and qualified the 1B and 4B. Google publishes three more
text-capable widths; this qualifies two of them on that same path, with no code change to the
builder, and closes off a precision that is not safe for the generation.

gemma-3-270m is 268M — the smallest useful model this repository builds. gemma-3-12b is
structurally identical to the already-qualified 4B: same Gemma3ForConditionalGeneration,
nested text_config, sliding_window 1024, pattern 6, rope_local_base_freq 10000,
rope_theta 1e6, linear rope scaling factor 8, query_pre_attn_scalar 256 — only wider
(48 layers, hidden 3840, 16 heads, 8 KV heads).

Exit Criteria

• Both widths match the Hugging Face reference, on the existing code path.
• A precision that is wrong for this generation cannot be selected.
• Gemma and Gemma 2 are unaffected.

Implementation

Manifests only for the two widths. The one code change is the fp16 refusal.

Change categories

• Model or runtime behavior

Validation

Commands and Results

Both built at bf16 and driven through the native CLI against the Hugging Face reference,
using the prompt rendering and metric of families/gemma/tests/test_e2e.py:

width	engine	reference	NED
gemma-3-270m	[50429, 107, 106] Paris	identical	0.0000
gemma-3-12b	[50429, 106] Paris	identical	0.0000

Why fp16 is refused. Largest absolute value leaving a decoder layer, reference in fp32,
against the fp16 maximum of 65,504:

width	peak	layers over	fp16 result
gemma-2-2b	4,059	0 of 26	works, 16x headroom
gemma-3-270m	102,956	11 of 18	token 0 repeatedly
gemma-3-1b	61,040	0 of 26	works, by 7%
gemma-3-4b	298,680	29 of 34	token 0 repeatedly

The 1B fits on one prompt by 7%, which is luck rather than headroom, so the refusal covers the
generation rather than individual widths. Gemma 2 keeps fp16 on its measured margin.

Other suites:

• families/gemma/tests/ — 35 passed, 2 of them new
• apps/benchmark/trtmc_benchmark/tests — 1107 passed
• ruff check clean

Hardware, Environment, and Revisions

• NVIDIA H100 80GB HBM3, x86 dev container
• Manifests name the official google/gemma-3-270m-it and google/gemma-3-12b-it. Those repos
  are gated and this container has no HF token, so local verification used the unsloth
  mirrors of the same weights; CI fetches the official ones, as it already does for
  gemma-2-2b-it and the widths in feat(gemma): add Gemma 3 text support (1B, 4B) and fix the Gemma stop tokens #1304.
• Rebased on 29facea2

Not Run / Remaining Gaps

• gemma-3-27b is deliberately not in this PR. It is the last text width and needs a
  large-memory host to build — the comparable Qwen3.6-27B build peaked at 291 GB — so it is
  being qualified separately rather than shipped unverified here.
• Gemma 3n and Gemma 4 remain refused by the model-type gate.
• Both widths are in excluded_profiles, not benchmarked: the release-performance receipt was
  collected only for gemma-2-2b, which exercises the same builder and runtime path.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

The fp16 numbers are the part worth keeping. Gemma 3 was qualified at bf16 in #1304 after fp16
produced a single repeated token, but the reason was not established at the time. Measuring the
activations settles it: this is range, not a graph defect, and the 1B "working" at fp16 is an
accident of one prompt landing 7% inside the limit. Refusing per width would have shipped that
accident as a supported configuration.

Third-party provenance: Gemma is published by Google; the checkpoints are the
google/gemma-3-270m-it and google/gemma-3-12b-it Hugging Face repositories.

Risk level

• Low

Two manifests on an existing code path, plus a refusal that only narrows what can be built.
No checkpoint that builds correctly today is affected.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d97f8db2-9a00-49c8-b36f-c1fa86265b05

📥 Commits

Reviewing files that changed from the base of the PR and between 5d5680c and b28d154.

📒 Files selected for processing (1)

• apps/benchmark/performance/release.yaml

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

📝 Summary

Summary

Adds gemma-3-270m qualification to the existing Gemma 3 text path.

• Uses BF16, bundled artifacts, disabled remote code, 256-token sequences, and single-device tensor parallelism.
• Adds an FP32 premerge reference test with chat-template support and an 8-token limit.
• Excludes the model from release performance profiles because no matching workload receipt exists.
• Rejects FP16 for gemma3 and gemma3_text because measured activations can exceed the FP16 range.
• Preserves Gemma 2 FP16 behavior.
• Keeps Gemma 3 27B, Gemma 3n, and Gemma 4 out of scope.
• Removes the failed gemma-3-12b qualification.

Architecture impact

• Family-owned files: families/gemma/model.py and Gemma test manifests and gate tests.
• Changed application surface: apps/benchmark/performance/release.yaml changes release profile selection.
• Dependency direction: No new dependencies. The benchmark configuration consumes Gemma’s public qualification path.
• Affected consumers: Gemma 3 build requests, Gemma qualification tests, and release performance profile selection.
• Blast-radius question: The official gated-checkpoint behavior for larger Gemma 3 variants remains unresolved. Local success with an unsloth mirror did not reproduce the official gemma-3-12b premerge result.

Review status

HUMAN REVIEW REQUIRED

No evidence-backed blocking violation is supplied. Review finding counts are unavailable. Human review is required for the unresolved official-checkpoint compatibility question.

Walkthrough

The change adds a Gemma 3 270M test manifest, excludes it from release-performance profiles, and rejects fp16 builds for Gemma 3 model types while preserving Gemma 2 fp16 gate behavior.

Changes

Gemma 3 support

Layer / File(s)	Summary
Gemma 3 manifest and performance configuration families/gemma/tests/manifests/gemma-3-270m.json, apps/benchmark/performance/release.yaml	Adds the Gemma 3 270M BF16 test manifest and excludes the model from release-performance profiles.
Gemma 3 fp16 build gate families/gemma/model.py, families/gemma/tests/test_model_type_gate.py	Marks gemma3 and gemma3_text as unsafe for fp16, rejects those builds, and tests continued gate behavior for Gemma 2.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Merge Risk: ⚪ Minimal · up to b28d1

The Gemma 3 qualification and fp16 safety guard introduce no evidenced merge-blocking risk.

🚥 Pre-merge checks | ✅ 8 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description uses the required sections and records substantial validation evidence, but it is materially stale. It still states that gemma-3-12b is included and passed validation, while the curren…	Update the description to cover only gemma-3-270m and the FP16 refusal. Remove the gemma-3-12b implementation and validation claims, update the exit criteria and affected models, and record the failed gemma-3-12b premerge result and reason …

✅ Passed checks (8 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files. (1 skipped: 1 u…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Family Ownership Boundary	✅ Passed	This pull request introduces no cross-family dependencies or family-boundary violations. All modified code is contained within the gemma family: families/gemma/model.py (adds fp16 refusal for Gemma …
Shared Semantic Neutrality	✅ Passed	The custom check requires evaluation of changed shared code outside family-owned directories (model Python, runtime, C++ tests, E2E, family-tools) for model-specific configuration or semantics expansi…
Benchmark Validation Integrity	✅ Passed	No benchmark-validation integrity failure is introduced. The release change only adds gemma-3-270m to excluded_profiles; it does not change timing scope, measurement iterations, baseline configura…
Shared Change Blast Radius	✅ Passed	PASS. The PR changes the shared release-performance catalog, but it documents and supports the required blast radius. The new exclusion addresses the model-agnostic coverage rule: a ready manifest mus…
Title check	✅ Passed	The title clearly summarizes the two active changes: qualifying gemma-3-270m and refusing FP16 for Gemma 3.

Full details: Description check

Explanation

The description uses the required sections and records substantial validation evidence, but it is materially stale. It still states that gemma-3-12b is included and passed validation, while the current objectives state that its manifest was removed after an NED 0.8387 failure.

Resolution

Update the description to cover only gemma-3-270m and the FP16 refusal. Remove the gemma-3-12b implementation and validation claims, update the exit criteria and affected models, and record the failed gemma-3-12b premerge result and reason for exclusion under Not Run / Remaining Gaps or Notes For Future Readers.

---

_{Comment @coderabbitai help to get the list of available commands.}

[zhenshanx-nv]

Dropped gemma-3-12b from this PR.

The internal premerge run on 5d5680c8 failed only on that case:

E  AssertionError: assert 0.8387096774193549 <= 0.15
E   +  where 0.8387096774193549 = _normalized_edit_distance('The capital of France is Paris.', 'Paris')
families/gemma/tests/test_e2e.py:480

The HF reference produced Paris; the engine produced a different continuation. My local
qualification of the same manifest passed exactly (engine ids=[50429, 106] text='Paris',
reference ids=[50429, 106] text='Paris', NED=0.0000), but it ran against the
unsloth/gemma-3-12b-it mirror because google/gemma-3-12b-it is gated. Divergence between
the official and mirrored config.json is the leading explanation — the same class of issue
produced three earlier rounds of failures on #1304 — but I have not been able to confirm it,
so the manifest is withheld rather than retried blind.

gemma-3-270m and the fp16 refusal are unchanged and independent of this.