perf(gemma4): optimize prefill and decode - #485

Open

AdnanHoque wants to merge 34 commits into

torch-spyre:mainfrom

AdnanHoque:perf/gemma4-prefill-logit-row

Commits on Sep 8, 2026

perf(gemma4): pair decode route scheduling with indexed-selection layouts
AdnanHoque
committed
b1accd5
perf(gemma4): copy only the prefill logit row used by generation
AdnanHoque
committed
c8a2e20
perf(gemma4): compute only the final prefill vocabulary row for generation
AdnanHoque
committed
f2100df
refactor(gemma4): share the scoped decode call and cleanup path
AdnanHoque
committed
6cf881d
perf(gemma4): select decode down weights in output blocks
AdnanHoque
committed
7fddbd5
perf(gemma4): stage decode gate and up weights in reduction blocks
AdnanHoque
committed
5802791
test(gemma4): guard every full down gather when blocks are enabled
AdnanHoque
committed
e462c2b
test(gemma4): allow independent down blocking to compose
AdnanHoque
committed
7bd5bf3
perf(gemma4): specialize persistent prefill matmul divisions
AdnanHoque
committed
b2547ec
fix(gemma4): validate prefill schedule before cache mutation
AdnanHoque
committed
387c2af
perf(gemma4): enable paired route scheduling automatically
AdnanHoque
committed
c04fa16
perf(gemma4): enable supported decode blocks and inherit automatic route scheduling
AdnanHoque
committed
4e05058
perf(gemma4): enable supported decode blocks and inherit automatic route scheduling
AdnanHoque
committed
16aa372
perf(gemma4): enable bounded prefill logit transfers by default
AdnanHoque
committed
f4ae6ae
perf(gemma4): automatically select supported prefill divisions
AdnanHoque
committed
3328765
perf(gemma4): select the generation head row automatically
AdnanHoque
committed
ca7b141
fix(gemma4): include checkpoint BF16 in automatic optimization selection
AdnanHoque
committed
6dbe226
fix(gemma4): include checkpoint BF16 in automatic optimization selection
AdnanHoque
committed
ecdd49c
fix(gemma4): include checkpoint BF16 in automatic optimization selection
AdnanHoque
committed
e871abd
test(gemma4): reject mixed 16-bit optimization inputs
AdnanHoque
committed
f437993
test(gemma4): reject mixed 16-bit optimization inputs
AdnanHoque
committed
d7de86d
test(gemma4): reject mixed 16-bit optimization inputs
AdnanHoque
committed
b170a20
fix(generation): retain final-logit-row copies across updated prefill paths
AdnanHoque
committed
74eb61a
fix(gemma4): select the active prefill driver for final-row generation
AdnanHoque
committed
e497ba2
Merge commit 'f437993' into perf/gemma4-consolidated-review
AdnanHoque
committed
ce9bb50
Merge commit 'e497ba2' into perf/gemma4-consolidated-review
AdnanHoque
committed
70e9194
Merge commit 'd7de86d' into perf/gemma4-consolidated-review
AdnanHoque
committed
ed651f7
perf(gemma4): consolidate the published prefill and decode features
AdnanHoque
committed
d641667
test(gemma4): consolidate performance coverage into existing model suites
AdnanHoque
committed
37f402b
docs(gemma4): clarify final-row projection during chunked prefill
AdnanHoque
committed
6b01e51

Navigation Menu

Uh oh!

perf(gemma4): optimize prefill and decode - #485

perf(gemma4): optimize prefill and decode#485
AdnanHoque wants to merge 34 commits into
torch-spyre:mainfrom
AdnanHoque:perf/gemma4-prefill-logit-row

Commits on Sep 8, 2026

perf(gemma4): pair decode route scheduling with indexed-selection layouts

perf(gemma4): copy only the prefill logit row used by generation

perf(gemma4): compute only the final prefill vocabulary row for generation

refactor(gemma4): share the scoped decode call and cleanup path

perf(gemma4): select decode down weights in output blocks

perf(gemma4): stage decode gate and up weights in reduction blocks

test(gemma4): guard every full down gather when blocks are enabled

test(gemma4): allow independent down blocking to compose

perf(gemma4): specialize persistent prefill matmul divisions

fix(gemma4): validate prefill schedule before cache mutation

perf(gemma4): enable paired route scheduling automatically

perf(gemma4): enable supported decode blocks and inherit automatic route scheduling

perf(gemma4): enable supported decode blocks and inherit automatic route scheduling

perf(gemma4): enable bounded prefill logit transfers by default

perf(gemma4): automatically select supported prefill divisions

perf(gemma4): select the generation head row automatically

fix(gemma4): include checkpoint BF16 in automatic optimization selection

fix(gemma4): include checkpoint BF16 in automatic optimization selection

fix(gemma4): include checkpoint BF16 in automatic optimization selection

test(gemma4): reject mixed 16-bit optimization inputs

test(gemma4): reject mixed 16-bit optimization inputs

test(gemma4): reject mixed 16-bit optimization inputs

fix(generation): retain final-logit-row copies across updated prefill paths

fix(gemma4): select the active prefill driver for final-row generation

Merge commit 'f437993' into perf/gemma4-consolidated-review

Merge commit 'e497ba2' into perf/gemma4-consolidated-review

Merge commit 'd7de86d' into perf/gemma4-consolidated-review

perf(gemma4): consolidate the published prefill and decode features

test(gemma4): consolidate performance coverage into existing model suites

docs(gemma4): clarify final-row projection during chunked prefill

Commits on Sep 10, 2026

Merge current main into Gemma performance PR

Clarify elision capability and measured prefill dependencies

ci: install test dependencies into the active image environment

ci: keep module-test callers in the prepared environment