Skip to content

feat(ntt): proved batched coset LDE over packed KoalaBear matrices - #401

Draft
mitschabaude-bot wants to merge 157 commits into
Verified-zkEVM:mainfrom
mitschabaude-bot:coset-lde
Draft

mitschabaude-bot wants to merge 157 commits into
Verified-zkEVM:mainfrom
mitschabaude-bot:coset-lde

Conversation

@mitschabaude-bot

Copy link
Copy Markdown
Contributor

Depends on #399 (and through it #395, #390). GitHub cannot use a fork branch as an upstream base, so this draft targets main and its diff includes the earlier PRs. Review only the new changes.

Why

This is the next step toward STARKs in Lean. The prover's main cost is committing to the trace: each column is extended from its trace domain to a blown-up coset, and the rows are hashed into a Merkle tree. Plonky3 does this with coset_lde_batch, followed by a row bit-reversal in TwoAdicFriPcs::commit. This PR adds that operation to CompPoly with a correctness proof, and it runs as fast as Plonky3 on mid-size and large matrices.

What

NTTFast.Packed.Native.cosetLde takes a row-major KoalaBear matrix packed as Montgomery words (like Plonky3's RowMajorMatrix) and returns the extension in commitment order, as row chunks a commitment can hash in parallel. joinRows copies the chunks into one buffer.

  • Algorithm. In bit-reversed row order the output is $2^{bits}$ blocks of $h$ rows. Block $t$ is the coset $\text{shift} \cdot \omega^{\mathrm{bitrev}(t)} \cdot \langle \omega^{2^{bits}} \rangle$ in DIF order. So each column needs:

    • one unnormalized inverse transform;
    • per block, one DIF transform of the scaled coefficients.

    No padding or reversal pass is needed. The steps run as parallel tasks:

    1. transpose row ranges of the input into columns, with $16 \times 16$ tiles;
    2. transform each column;
    3. transpose each block's rows back with $8 \times 8$ tiles, as soon as that block is done.
  • Proof. Native.joinRows_cosetLde proves the joined output equal to ldeSpec: entry $(r, j)$ is column $j$'s Lagrange interpolant (CLagrange.interpolatePow) at $\text{shift} \cdot \omega^{\mathrm{bitrev}(r)}$.

    • The transposes are proved by covering: every write stores the word's final value, and the writes together cover the output (CosetLdeTransposes.lean).
    • The transforms reuse the packed NTT leaf proofs.
  • Arithmetic. Packed add/sub/mul now reduce with an unsigned minimum (vpminud). This makes the transform kernels about 10% faster.

  • Benchmark. --suite lde compares against Plonky3 0.4.2's coset_lde_batch with three DFTs: Radix2DitParallel, RecursiveDft and Radix2DFTSmallBatch. Full output digests agree before timing.

Performance

Eight physical cores of a Ryzen 7 3700X, shared host, three alternating rounds. Times are in milliseconds per call. The Plonky3 column is its fastest DFT for each shape.

Rows Columns Blowup Lean Plonky3 Lean / Plonky3
$2^{14}$ 64 2 4.23 2.48 1.71×
$2^{16}$ 8 2 2.84 1.65 1.72×
$2^{16}$ 64 2 17.7 16.8 1.05×
$2^{16}$ 256 2 84.7 86.7 0.98×
$2^{18}$ 64 2 81.9 88.2 0.93×
$2^{20}$ 8 2 68.3 57.3 1.19×
$2^{20}$ 64 2 451 364 1.24×
$2^{16}$ 64 4 29.7 30.6 0.97×

Lean returns row chunks, while Plonky3 returns one buffer; joining the chunks costs one extra copy of the output. Small matrices are limited by task overhead. On a single thread, Plonky3's hand-written AVX2 butterflies are 2–3× faster per element; we make up the difference with parallelism. The benchmarking wiki logs the variants that were tried and rejected.

graikos and others added 30 commits September 29, 2026 07:33
…ocks

Fold the 64x64 product's middle terms without carry bookkeeping so clang emits
one widening multiply for squarings too; `mul x x` inlined into `square` used
to compile to a four-imul chain. Run the Fermat inverse and the pow ladder on
unreduced words (`mulLazy`, `squareNLazy`, `invLazy`, `powLazy`) with a single
canonicalization at the end, and make `squareN` tail recursive. Correctness
rests on word-level cast lemmas (`foldUInt128Lazy_cast`, `mulLazy_cast`,
`squareNLazy_cast`, `invLazy_cast`, `powLazy_cast`).

A/B against master on the same core: inv 311 -> 167 ns, pow 99 -> 74 ns,
mul and add unchanged.
`conditionalSubtract` was a plain def, so every KoalaBear and BabyBear `add`
paid an out-of-line call. A/B: add latency 844 -> 606 ps, throughput
876 -> 166 ps.
…nreduced words

`univariate-dense-goldilocks` now runs the native-word carrier beside ZMod like
the other fast fields, with rows named in the shared `-fast` scheme. Add tests
that unreduced `ofUInt64` words compare and multiply as their residues.
Compute the canonical sum as `x - (p - y)`: the subtraction borrows exactly
when `x + y < p`, so one borrow-selected correction by `2^32 - 1` yields the
reduced result, three dependent instructions instead of a carry check plus a
separate compare against `p`. `addRaw_toNat` carries the bound and the cast
lemma; the carry-based reducer and its lemmas are removed.

A/B against master on the same core: add latency 1021 -> 616 ps, throughput
381 -> 230 ps; mul, inv and pow unchanged.
# Conflicts:
#	bench/CompPolyBench/Fields/Arith.lean
… the Goldilocks fold

Two changes to the 128-bit fold behind every fast Goldilocks multiply. The borrow
of the low word against the top limb has probability about 2^-32, but a Lean `if`
compiled to a select on the critical path; moving that arm into a `@[noinline]`
function forces a predicted branch. The middle term `hi_lo * (2^32 - 1)` compiled
to a three-cycle `imul` inside loops (clang holds the constant in a register); it
is now `(hi <<< 32) - hi_lo`, two one-cycle operations. `shiftLeft32_sub_low` and
`foldTail_cast` carry the proofs; the fold lemma cases on the borrow.

A/B against master on the same core: mul latency 3077 -> 2485 ps, throughput
815 -> 739 ps, inv 314 -> 136 ns, pow 101 -> 64 ns; add unchanged.
One-sentence docstrings, `2 ^ k` literals instead of decimals, a `_toNat`
bridge for the product's high word, implicit bounds on the `addRaw` lemmas,
camelCase limb names, and README rows for the Goldilocks files.
`Native.joinRows_cosetLde` proves that the joined row chunks of
`cosetLde` are the packed `ldeSpec`: row `r`, column `j` is the column's
Lagrange interpolant (`CLagrange.interpolatePow`) on the subgroup of
order `2 ^ logH`, evaluated at `shift * omega ^ bitrev r`.
`Native.cosetLde_eq` states each chunk as a row range of the spec.

The transposes are proved by covering (`CosetLdeTransposes.lean`):
`Fill T S N g` says `g` overwrites words only with their targets `T`,
among them every word of `S`, and fills compose along folds. The strided
16 x 16 and eight-source 8 x 8 tile kernels, the row-tile loop and the
scalar edges each fill their words, and together cover the output.

The column transforms reuse the packed leaf proofs: `leafReversed` is
the radix-four DIF stages (restoring `leaf16_encode`), `mulWords` the
coordinate products. `getD_block` reduces a block entry to the
interpolant's coefficient sum through `bitRevNat_concat`. The task
graph reduces with `Tasks.get_bind`/`get_spawn`, now in `Data/Tasks`.

Docs: the `lde` suite in the bench guide, the LDE pass in the
benchmarking log, and a roadmap entry.
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

🤖 PR Summary

ℹ️ Some sections were omitted to fit GitHub's comment size limit.

feat(ntt): proved batched coset LDE over packed KoalaBear matrices


Statistics

Metric Count
📝 Files Changed 157
✅ Lines Added 24074
❌ Lines Removed 1200

Lean Declarations

✏️ Removed: 35 declaration(s)

CompPoly/Fields/BLS12_377/Fast.lean (1)

  • instance instMont64x8Field : Mont64x8Field BLS12_377.scalarFieldSize where

CompPoly/Fields/BLS12_381/Fast.lean (1)

  • instance instMont64x8Field : Mont64x8Field BLS12_381.scalarFieldSize where

CompPoly/Fields/BN254/Fast.lean (1)

  • instance instMont64x8Field : Mont64x8Field BN254.scalarFieldSize where

CompPoly/Fields/Goldilocks/Fast.lean (2)

  • private theorem mul_assoc_field (x y z : Field) : (x * y) * z = x * (y * z)
  • private theorem pow_succ (x : Field) (n : Nat) : pow x (n + 1) = pow x n * x

CompPoly/Fields/Goldilocks/FastDefs.lean (1)

  • def reduceAddWithCarryRaw (lo : UInt64) (carry : Bool) : UInt64

CompPoly/Fields/Goldilocks/FastReduction.lean (6)

  • theorem addWithCarry_bound (x y : UInt64)
  • theorem addWithCarry_value (x y : UInt64) :
  • theorem reduceAddWithCarryRaw_cast (lo : UInt64) (carry : Bool)
  • theorem reduceAddWithCarryRaw_lt (lo : UInt64) (carry : Bool)
  • theorem wideMul_high_nat
  • theorem wideMul_high_toNat

CompPoly/Fields/Montgomery/Native64x8.lean (1)

  • private theorem carry_top_zero {S A B Q c : ℕ} (h : S + 2 ^ 256 * c = A + B)

CompPoly/Fields/Montgomery/Native64x8Field.lean (1)

  • theorem two_mul_q_lt : 2 * P.modulusLimbs.toNat < 2 ^ 256

CompPoly/Fields/Pasta/Fast.lean (1)

  • instance instMont64x8Field : Mont64x8Field Vesta.baseFieldSize where

CompPoly/Fields/Secp256k1.lean (7)

  • abbrev BaseField
  • abbrev ScalarField
  • def BASE_FIELD_CARD : Nat
  • def SCALAR_FIELD_CARD : Nat
  • example :
  • theorem BaseField_is_prime : Nat.Prime BASE_FIELD_CARD
  • theorem ScalarField_is_prime : Nat.Prime SCALAR_FIELD_CARD

bench/CompPolyBench/Fields/Binary/Tower.lean (7)

  • @[inline] private def towerSampler {E : Type} (xs : Array E) (one : E) : Nat → E × E
  • @[specialize] private def runTowerGroup (groupKey title method : String)
  • def checksumConcreteBt128 (x : ConcreteBTField 7) : Nat
  • def checksumFastBT128 (x : Fast.FastBT128) : Nat
  • private def runTowerInv (preset : BenchPreset) (gen : StdGen) :
  • private def runTowerMul (preset : BenchPreset) (gen : StdGen) :
  • private def towerShape : String

bench/CompPolyBench/Fields/Binary/Tower/Scalar.lean (5)

  • private def runTowerInv64 (preset : BenchPreset) (gen : StdGen) :
  • private def runTowerMul64 (preset : BenchPreset) (gen : StdGen) :
  • private def runTowerMul8 (preset : BenchPreset) (gen : StdGen) :
  • private def towerWordPool (bits : Nat) (gen : StdGen) : Array UInt64 × StdGen
  • private def towerWordRep (suffix representation field : String) (pool : Array UInt64) :

bench/CompPolyBench/Fields/Inputs.lean (1)

  • @[inline] def sinkMont64x8 {modulus : Nat} [Montgomery.Native64x8.Mont64x8Field modulus]
✏️ Added: 1339 declaration(s)

CompPoly/Data/Array/Involution.lean (9)

  • @[inline] def permuteInvolution (order : Array Nat) (a : Array α) : Array α
  • @[inline] private def involutionStep (f : Nat → Nat) (a : Array α) (i : Nat) : Array α
  • @[simp] private theorem size_involutionStep (f : Nat → Nat) (a : Array α) (i : Nat) :
  • @[simp] theorem size_permuteInvolution (order : Array Nat) (a : Array α) :
  • private theorem getD_swap (a : Array α) (i j k : Nat) (d : α)
  • private theorem permuteInvolution_eq_fold (order : Array Nat) (a : Array α) :
  • private theorem prefix_invariant (f : Nat → Nat) (a : Array α) (d : α)
  • private theorem size_fold (f : Nat → Nat) (is : List Nat) (a : Array α) :
  • theorem permuteInvolution_eq_map (order : Array Nat) (a : Array α) (d : α)

CompPoly/Data/ByteArray/Pack.lean (12)

  • @[simp] theorem size_setUInt32LE! (a : ByteArray) (off : Nat) (v : UInt32) :
  • @[simp] theorem size_usetUInt32LE (a : ByteArray) (i : USize) (v : UInt32) (h) :
  • def getUInt32LE : (a : @& ByteArray) → (i : @& Nat) → (h : i + 4 ≤ a.size
  • def getUInt32LE! (a : @& ByteArray) (off : @& Nat) : UInt32
  • def setUInt32LE : (a : ByteArray) → (i : @& Nat) → (v : UInt32) →
  • def setUInt32LE! (a : ByteArray) (off : @& Nat) (v : UInt32) : ByteArray
  • def ugetUInt32LE : (a : @& ByteArray) → (i : USize) →
  • def usetUInt32LE : (a : ByteArray) → (i : USize) → (v : UInt32) →
  • theorem get!_eq_getElem! (a : ByteArray) (i : Nat) : a.get! i = a[i]!
  • theorem getElem!_setUInt32LE!_of_outside (a : ByteArray) (o j : Nat) (v : UInt32)
  • theorem getUInt32LE!_setUInt32LE!_of_disjoint (a : ByteArray) (o₁ o₂ : Nat) (v : UInt32)
  • theorem getUInt32LE!_setUInt32LE!_self (a : ByteArray) (off : Nat) (v : UInt32)

CompPoly/Data/ByteArray/Packed.lean (68)

  • @[inline, reducible] def ofWord32 {R : Type u} (toWord : R → UInt32) (ofWord : UInt32 → R)
  • @[inline] def getAt (b : ByteArray) (k : USize) (hk : k.toNat < b.size / width R)
  • @[inline] def setAt (b : ByteArray) (k : USize) (x : R) (hk : k.toNat < b.size / width R)
  • @[inline] def setIf (n : Nat) (hn : n < USize.size) (b : ByteArray) (k : USize) (x : R) :
  • @[simp] theorem decode_encode (a : Array R) : decode (encode a) = a
  • @[simp] theorem encode_empty : encode (#[] : Array R) = ByteArray.empty
  • @[simp] theorem size_alloc (n : Nat) : (alloc R n).size = width R * n
  • @[simp] theorem size_alloc_div (n : Nat) : (alloc R n).size / width R = n
  • @[simp] theorem size_encode (a : Array R) : (encode a).size = width R * a.size
  • @[simp] theorem size_encode_uint32 (W : Array UInt32) : (encode W).size = 4 * W.size
  • @[simp] theorem size_setAt (b : ByteArray) (k : USize) (x : R) (hk) (hu) :
  • @[simp] theorem size_zeros (n : Nat) : (zeros n).size = n
  • @[simp] theorem width_uint32 : width UInt32 = 4
  • def alloc (R : Type u) [PackedRepr R] (n : Nat) : ByteArray
  • def bytesAt (b : ByteArray) (off w : Nat) : Vector UInt8 w
  • def decode (b : ByteArray) : Array R
  • def encode (a : Array R) : ByteArray
  • def fillZeros (target : Nat) : Nat → ByteArray → ByteArray
  • def get (b : ByteArray) (j : Nat) : R
  • def ofWordBytes (v : Vector UInt8 4) : UInt32
  • def replaceAt (b : ByteArray) (off : Nat) (values : ByteArray) : ByteArray
  • def replicate (n : Nat) (x : R) : ByteArray
  • def replicateGo (n : Nat) (block acc : ByteArray) : ByteArray
  • def wordBytes (v : UInt32) : Vector UInt8 4
  • def zeroBlock : ByteArray
  • def zeros (n : Nat) : ByteArray
  • private theorem drop_flatMap (l : List R) (i : Nat) :
  • private theorem length_flatMap (l : List R) :
  • private theorem take_flatMap (l : List R) (i : Nat) :
  • theorem bytesAt_encode (a : Array R) (i : Nat) (hi : i < a.size) :
  • theorem copySlice_encode (a o : Array R) (i c : Nat) :
  • theorem copySlice_replicate (k m len : Nat) (hl : len ≤ k) :
  • theorem encode_append (a b : Array R) : encode (a ++ b) = encode a ++ encode b
  • theorem encode_extract (a : Array R) (first last : Nat) :
  • theorem encode_ofWord32 {R : Type u} (toWord : R → UInt32) (ofWord : UInt32 → R)
  • theorem fillZeros_replicate (target : Nat) :
  • theorem get!_replaceAt (b v : ByteArray) (off i : Nat) (h : off + v.size ≤ b.size) :
  • theorem getAt_eq (b : ByteArray) (k : USize) (hk) (hu) :
  • theorem getElem_decode (b : ByteArray) (i : Nat) (hi : i < (decode (R
  • theorem getElem_decode_eq_get (b : ByteArray) (j : Nat) (hj : j < (decode (R
  • theorem getUInt32LE!_encode (W : Array UInt32) (k : Nat) :
  • theorem getUInt32LE!_eq_ofWordBytes (b : ByteArray) (off : Nat) (h : off + 4 ≤ b.size) :
  • theorem get_encode (a : Array R) (j : Nat) (hj : j < a.size) : get (encode a) j = a[j]
  • theorem get_setAt (b : ByteArray) (k : USize) (x : R) (hk) (hu) (j : Nat) :
  • theorem ofWordBytes_wordBytes (v : UInt32) : ofWordBytes (wordBytes v) = v
  • theorem read_encode (a : Array R) (off : USize) (i : Nat) (ho : off.toNat = width R * i)
  • theorem replaceAt_empty (b : ByteArray) (off : Nat) (h : off ≤ b.size) :
  • theorem replaceAt_encode (a v : Array R) (i : Nat) :
  • theorem replaceAt_encode_uint32 (W V : Array UInt32) (i : Nat) :
  • theorem replaceAt_replaceAt (b p q : ByteArray) (off : Nat)
  • theorem replaceAt_self (b : ByteArray) (off w : Nat) (h : off + w ≤ b.size) :
  • theorem replicateGo_encode (x : R) : ∀ (n m a : Nat),
  • theorem replicate_eq (n : Nat) (x : R) : replicate n x = encode (Array.replicate n x)
  • theorem setAt_encode (a : Array R) (k : USize) (x : R) (hk) (hu) :
  • theorem setIf_encode (a : Array R) (k : USize) (x : R) (hn : width R * a.size < USize.size) :
  • theorem setIf_encode_of_eq (a : Array R) (k : USize) (x : R) (m : Nat) (hm : m < USize.size)
  • theorem setUInt32LE!_encode (W : Array UInt32) (k : Nat) (v : UInt32) :
  • theorem setUInt32LE!_eq_replaceAt (b : ByteArray) (off : Nat) (v : UInt32)
  • theorem setUInt32LE!_eq_replaceAt_encode (b : ByteArray) (off : Nat) (v : UInt32)
  • theorem setUInt32LE!_getUInt32LE! (b : ByteArray) (off : Nat) (h : off + 4 ≤ b.size) :
  • theorem set_eq_splice (a : Array R) (i : Nat) (x : R) (hi : i < a.size) :
  • theorem size_decode (b : ByteArray) : (decode (R
  • theorem size_replaceAt (b values : ByteArray) (off : Nat) (h : off + values.size ≤ b.size) :
  • theorem slot_toNat (k : USize) (n : Nat) (hk : k.toNat < n / width R) (hu : n < USize.size) :
  • theorem toUInt8_shiftRight (v : UInt32) (s : UInt32) (k : Nat) (hs : s.toNat = 8 * k)
  • theorem wordBytes_ofWordBytes (v : Vector UInt8 4) : wordBytes (ofWordBytes v) = v
  • theorem write_encode (a : Array R) (off : USize) (i : Nat) (x : R)
  • theorem zeros_eq (n : Nat) : zeros n = ⟨Array.replicate n 0⟩

CompPoly/Data/Bytes/UInt32.lean (2)

  • @[simp] theorem ByteCodec.width_uint32 : ByteCodec.width UInt32 = 4
  • theorem ByteCodec.toBytes_uint32 (x : UInt32) :

CompPoly/Data/List/BatchInv.lean (15)

  • @[simp] theorem length_batchInv [Inv K] [One K] (xs : List K) :
  • def batchInv [Inv K] [One K] (xs : List K) : List K
  • def batchInvGo : List K → List K → K → List K → List K
  • def prefixProducts (q : K) : List K → List K
  • def productFrom (q : K) : List K → K
  • theorem batchInvGo_map (hmul : ∀ a b, f (a * b) = f a * f b) (xs ps : List K) (acc : K)
  • theorem batchInvGo_spec_reverse (r : List K) (hl : ∀ x ∈ r, x ≠ 0) (q : K) (hq : q ≠ 0)
  • theorem batchInv_eq (xs : List K) (hxs : ∀ x ∈ xs, x ≠ 0) : batchInv xs = xs.map (·⁻¹)
  • theorem batchInv_map [Inv K] [One K] [Inv L] [One L] (hmul : ∀ a b, f (a * b) = f a * f b)
  • theorem length_batchInvGo (xs ps : List K) (hl : xs.length = ps.length) (acc : K)
  • theorem length_prefixProducts (q : K) (l : List K) :
  • theorem prefixProducts_append_singleton (q : K) (l : List K) (x : K) :
  • theorem prefixProducts_map (hmul : ∀ a b, f (a * b) = f a * f b) (q : K) (l : List K) :
  • theorem productFrom_eq (q : K) (l : List K) : productFrom q l = q * l.prod
  • theorem productFrom_map (hmul : ∀ a b, f (a * b) = f a * f b) (q : K) (l : List K) :

CompPoly/Data/Tasks.lean (10)

  • @[inline] def spawnAfter (deps : Array (Task α)) (n : Nat) (f : Array α → Nat → β) :
  • @[inline] def spawnAll (n : Nat) (f : Nat → α) : Array (Task α)
  • @[simp] theorem get_bind (t : Task α) (f : α → Task β) (prio : Task.Priority) (sync : Bool) :
  • @[simp] theorem get_spawn (fn : Unit → α) (prio : Task.Priority) :
  • @[simp] theorem map_get_spawnAfter (deps : Array (Task α)) (n : Nat) (f : Array α → Nat → β) :
  • @[simp] theorem map_get_spawnAll (n : Nat) (f : Nat → α) :
  • def join (ts : Array (Task α)) : Task (Array α)
  • def joinGo : List (Task α) → List α → Task (Array α)
  • theorem get_join (ts : Array (Task α)) : (join ts).get = ts.map Task.get
  • theorem get_joinGo (ts : List (Task α)) (as : List α) :

CompPoly/Fields/BLS12_377/Fast.lean (1)

  • instance instMont64x4Field : Mont64x4Field BLS12_377.scalarFieldSize where

CompPoly/Fields/BLS12_381/Fast.lean (1)

  • instance instMont64x4Field : Mont64x4Field BLS12_381.scalarFieldSize where

CompPoly/Fields/BN254/Fast.lean (1)

  • instance instMont64x4Field : Mont64x4Field BN254.scalarFieldSize where

CompPoly/Fields/BabyBear/FastExt4.lean (3)

  • @[inline] instance instBinomial4 : Binomial4 BabyBear.fieldSize where
  • abbrev Ext4 : Type
  • theorem irreducible_X_pow_four_sub_eleven : Irreducible (X ^ 4 - C 11 : Field[X])

CompPoly/Fields/BabyBear/InterpolateCoset.lean (1)

  • def interpolateCosetRun (logN : ℕ) (ω s : Field) (width : ℕ) (evals : ByteArray) (z : Ext4)

CompPoly/Fields/Extension/Map.lean (6)

  • @[simp] theorem map_d (e : F →+* G) (P : ExtensionParams F) : (P.map e).d = P.d
  • @[simp] theorem map_q (e : F →+* G) (P : ExtensionParams F) : (P.map e).q = P.q
  • def map (e : F →+* G) (P : ExtensionParams F) : ExtensionParams G where
  • theorem irreducible_map_poly (e : F ≃+* G) (P : ExtensionParams F) (h : Irreducible P.poly) :
  • theorem irreducible_map_ringEquiv (e : F ≃+* G) {f : F[X]} (h : Irreducible f) :
  • theorem map_poly (e : F →+* G) (P : ExtensionParams F) : (P.map e).poly = P.poly.map e

CompPoly/Fields/Goldilocks/FastDefs.lean (7)

  • def addRaw (x y : UInt64) : UInt64
  • def foldUInt128Borrow (lo hiHi hiLo : UInt64) : UInt64
  • def foldUInt128Lazy (lo hi : UInt64) : UInt64
  • def invLazy (x : UInt64) : UInt64
  • def mulLazy (x y : UInt64) : UInt64
  • def powLazy (acc x : UInt64) (n : Nat) : UInt64
  • def squareNLazy (x : UInt64) : Nat → UInt64

CompPoly/Fields/Goldilocks/FastReduction.lean (12)

  • private theorem foldTail_cast (lo hiHi hiLo t0 : UInt64) (hhiLo : hiLo.toNat < 2 ^ 32)
  • private theorem wideMul_hi_nat (p00 p01 p10 p11 : Nat) :
  • theorem addRaw_cast {x y : UInt64} (hx : x.toNat < Goldilocks.fieldSize)
  • theorem addRaw_lt {x y : UInt64} (hx : x.toNat < Goldilocks.fieldSize)
  • theorem addRaw_toNat {x y : UInt64} (hx : x.toNat < Goldilocks.fieldSize)
  • theorem foldUInt128Lazy_cast (lo hi : UInt64) :
  • theorem invLazy_cast (x : UInt64) :
  • theorem mulLazy_cast (x y : UInt64) :
  • theorem powLazy_cast (acc x : UInt64) (n : Nat) :
  • theorem shiftLeft32_sub_low (hi : UInt64) :
  • theorem squareNLazy_cast (x : UInt64) (n : Nat) :
  • theorem wideMul_snd_toNat (x y : UInt64) :

CompPoly/Fields/KoalaBear/FastExt4.lean (3)

  • @[inline] instance instBinomial4 : Binomial4 KoalaBear.fieldSize where
  • abbrev Ext4 : Type

…and 1189 more not listed.

✏️ Affected: 21 declaration(s) (line number changed)
  • theorem addLimbs_toNat (a b : Limbs8) (ha : a.Bounded) (hb : b.Bounded) : in CompPoly/Fields/Montgomery/Native64x8.lean moved from L278 to L319
  • theorem add_bounded (q a b : Limbs8) : (add q a b).Bounded in CompPoly/Fields/Montgomery/Native64x8.lean moved from L372 to L422
  • @[inline] def add (q a b : Limbs8) : Limbs8 in CompPoly/Fields/Montgomery/Native64x8Defs.lean moved from L149 to L178
  • @[specialize] def gcdFinalChunks (q : Limbs8) (negInv : UInt64) (finalRounds : Nat) in CompPoly/Fields/Montgomery/Native64x8InvDefs.lean moved from L194 to L215
  • @[specialize] def gcdInvCandidate (modulus : Nat) [P : GcdData modulus] (q : Limbs8) in CompPoly/Fields/Montgomery/Native64x8InvDefs.lean moved from L207 to L230
  • @[specialize] def gcdMainLoop (q : Limbs8) (negInv : UInt64) (rounds : Nat) (a u b v : Limbs8) in CompPoly/Fields/Montgomery/Native64x8InvDefs.lean moved from L174 to L193
  • def towerScalarTasks : List BenchTask in bench/CompPolyBench/Fields/Binary/Tower/Scalar.lean moved from L93 to L101
  • private def runScalarInv (modulus : Nat) [Mont64x4Field modulus] [GcdData modulus] in bench/CompPolyBench/Fields/Montgomery.lean moved from L32 to L33
  • private def a : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L31 to L31
  • private def b : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L35 to L34
  • private def montA : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L39 to L37
  • private def montAB : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L47 to L43
  • private def montB : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L43 to L40
  • private def negInv : UInt64 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L25 to L25
  • private def pMontA : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L84 to L79
  • private def pMontAB : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L92 to L85
  • private def pMontB : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L88 to L82
  • private def pNegInv : UInt64 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L82 to L77
  • private def pq : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L80 to L75
  • private def q : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L23 to L23
  • private def rMod : Limbs4 in tests/CompPolyTests/Fields/Pasta/Fast.lean moved from L27 to L27

sorry Tracking

  • No sorrys were added, removed, or affected.

Coverage Notes

  • AI file summarization partially analyzed 3 file(s) because their individual diffs exceeded the per-file size budget. Statistics and Lean signal tracking still cover the full PR.
Partially Analyzed Files
  • CompPoly/Univariate/NTTFast/ColumnsCorrectness.lean (+1474/-0)
  • CompPoly/Univariate/NTTFast/Packed/CosetLdeCorrectness.lean (+1101/-0)
  • docs/wiki/benchmarking.md (+569/-4)
* Additional-instructions analysis was skipped because the full diff exceeded the analysis size budget, and partial results would be misleading.

Last updated: 2026-10-28 23:01 UTC.

Montgomery.Native32.Ext4 p is FastField p [X] / (X^4 - W) for any
Mont32Field p with a Binomial4 p instance, which supplies W, a fast
multiplication by W, and the irreducibility of X^4 - W over FastField p.
Its spec field Ext4.Spec p is Ext over FastField p itself, so consumers
never meet ZMod; specEquiv, the toSpec_* agreement lemmas and the byte
codec carry over. The norm inverse gets its non-square W from the
irreducibility.

KoalaBear.Fast.Ext4 becomes the X^4 - 3 instance (mulW x = x + x + x),
and BabyBear.Fast.Ext4 the X^4 - 11 instance, with Plonky3 known answers.
Both transfer their ZMod certificates along ringEquiv with the new
Extension.Map (ExtensionParams.map, irreducible_map_ringEquiv).
FastField gets Fintype and card lemmas. Coset interpolation is respecced
to the new spec field. Extension benchmarks and packed interpolation
time the same as before.
The packed kernels, plan and refinement proofs work on FastField p for any
Mont32Field p. The raw word operations take the modulus and
montgomeryNegInv from the instance, ofWord checks against p, and every
kernel takes p and is @[specialize]d; the unrolled generators emit add p.

The plan calls the class PackedNTT p, whose run is Native.run p
(run_eq), so a field's kernels are specialized and compiled once, in the
CPU-targeted native library: Packed/Instances.lean provides KoalaBear and
BabyBear, and their hot kernels disassemble to the instructions of the
KoalaBear-only kernels. CompPolyNativeSmoke runs both instances against
the DFT specification and the parallel path against the sequential one.
interpolateCoset, interpolateCosetPacked and their correctness proofs move
to Fields/Montgomery and work over FastField p with any Binomial4 quartic
extension; the weights' constants use Binomial4.mulW, and stored words are
checked against the instance's modulus.

The packed loops are @[specialize]d, and interpolateCosetPacked calls the
class PackedInterpolation p, whose instances in KoalaBear/InterpolateCoset
and BabyBear/InterpolateCoset (the roots of CompPolyInterpolateNative)
compile them at the field's modulus. CompPolyNativeSmoke checks both
instances against the reference.
# Conflicts:
#	CompPoly/Univariate/NTTFast/Packed/Arithmetic.lean
#	docs/wiki/benchmarking.md
#	lakefile.lean
LdePlan p, Native.cosetLde and its correctness proofs work on FastField p
for any Mont32Field p; the minimum-form packed arithmetic takes the
modulus from the instance, and leaf16, mulWords and the coset kernels take
p and are @[specialize]d.

LdePlan.cosetLde calls the class PackedCosetLde p, whose KoalaBear and
BabyBear instances join the PackedNTT ones in Packed/Instances.lean;
LdePlan.joinRows_cosetLde restates the correctness theorem for it, and
CompPolyNativeSmoke checks both instances against ldeSpec.
# Conflicts:
#	CompPoly/Data/Tasks.lean
…r radix-four pass

`lastHalf` checks its output once per call, writes the output's last word back unchanged
(`touchLast`), and then stores its sixteen half lines with proved bounds (`store8`): the
exclusivity check of the first store no longer holds the 128 lane values across a copy, so they
stay in registers. `radix4Blocks` checks the whole pass once for quarters of whole sixteen-cell
batches and runs the blocks in a machine-word loop (`blocks16Loop`), without the per-block
natural-number checks.
`leaf16x2` runs the last four DIF layers of two sixteen-word blocks at once and multiplies every
butterfly of the first three layers by its twiddle, the first one being one, so that each
block's layers are uniform and the compiler vectorizes them. `tile8` checks its eight lines and
its output tile once on machine words, writes the output's last word back unchanged and then
loads and stores without checks, one output line at a time.
…s once per transform

`leaf16Blocks` checks all its block pairs once and runs `leaf16x2` in a machine-word loop
(`leaf16x2Loop`); `mulWords` does the same for its sixteen-word products (`mul16Loop`). Neither
loop compares natural numbers per block any more.
The packed multiplication subtracts the high word of the quotient times the modulus from the high
word of the product and corrects the sign by an unsigned minimum, the quotient a product of words.
The compiler multiplies a lane group with four 32-bit products and two word products, without
64-bit comparisons: the coset LDE of 2^15 × 1024 words is 24% faster on one thread.
…ps of four blocks

Each of the four last DIF layers of a forward leaf runs in one pass over the leaf, a kernel call
per four sixteen-word blocks: the kernel touches the buffer, reads its blocks, pairs each block's
words in eight vector lanes and stores every block as two halves with proved stores. A layer's
lanes are then one vector operation each, where the two-block kernel of all four layers left the
compiler shuffling scalars. The passes are the layers block by block (fold_leaf16Stages), each
kernel the layer's model on its blocks (leafStage8_encode and siblings, LeafStages.lean).
transposeRows moved its tiles across all columns for every sixteen rows, so each row of tiles
stored one line into every column of a piece of up to 21 MB: a page per line. Down a block of up to
1024 rows before across, each column receives four kilobytes per block. The transpose of 2^15 rows
of 2633 words takes 150 ms instead of 440 on one thread, 560 ms instead of 1080 for 2^16 rows.
A coset block of an odd leaf runs its first DIF layer out of place on the coefficients, each word
multiplied by its coset table word as it is read (pair16Scaled), instead of storing the products in
a pass of their own. pairsScaled_encode proves the pass the first layer of the products, and
leafReversed_odd splits an odd leaf after that layer. An even leaf's first radix-four pass would
read four quarters of the coefficients and of the table at a power-of-two stride, too many lines
of one cache set, so its products still come first. The coset LDE of 2^15 × 2633 words runs 3%
fewer instructions on one thread.
…nput transposes and fused coset products, with the Keccak LDE timings
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants