Repository navigation
feat(ntt): proved batched coset LDE over packed KoalaBear matrices - #401
mitschabaude-bot wants to merge 157 commits into
Conversation
…ocks Fold the 64x64 product's middle terms without carry bookkeeping so clang emits one widening multiply for squarings too; `mul x x` inlined into `square` used to compile to a four-imul chain. Run the Fermat inverse and the pow ladder on unreduced words (`mulLazy`, `squareNLazy`, `invLazy`, `powLazy`) with a single canonicalization at the end, and make `squareN` tail recursive. Correctness rests on word-level cast lemmas (`foldUInt128Lazy_cast`, `mulLazy_cast`, `squareNLazy_cast`, `invLazy_cast`, `powLazy_cast`). A/B against master on the same core: inv 311 -> 167 ns, pow 99 -> 74 ns, mul and add unchanged.
`conditionalSubtract` was a plain def, so every KoalaBear and BabyBear `add` paid an out-of-line call. A/B: add latency 844 -> 606 ps, throughput 876 -> 166 ps.
…nreduced words `univariate-dense-goldilocks` now runs the native-word carrier beside ZMod like the other fast fields, with rows named in the shared `-fast` scheme. Add tests that unreduced `ofUInt64` words compare and multiply as their residues.
Compute the canonical sum as `x - (p - y)`: the subtraction borrows exactly when `x + y < p`, so one borrow-selected correction by `2^32 - 1` yields the reduced result, three dependent instructions instead of a carry check plus a separate compare against `p`. `addRaw_toNat` carries the bound and the cast lemma; the carry-based reducer and its lemmas are removed. A/B against master on the same core: add latency 1021 -> 616 ps, throughput 381 -> 230 ps; mul, inv and pow unchanged.
# Conflicts: # bench/CompPolyBench/Fields/Arith.lean
… the Goldilocks fold Two changes to the 128-bit fold behind every fast Goldilocks multiply. The borrow of the low word against the top limb has probability about 2^-32, but a Lean `if` compiled to a select on the critical path; moving that arm into a `@[noinline]` function forces a predicted branch. The middle term `hi_lo * (2^32 - 1)` compiled to a three-cycle `imul` inside loops (clang holds the constant in a register); it is now `(hi <<< 32) - hi_lo`, two one-cycle operations. `shiftLeft32_sub_low` and `foldTail_cast` carry the proofs; the fold lemma cases on the borrow. A/B against master on the same core: mul latency 3077 -> 2485 ps, throughput 815 -> 739 ps, inv 314 -> 136 ns, pow 101 -> 64 ns; add unchanged.
One-sentence docstrings, `2 ^ k` literals instead of decimals, a `_toNat` bridge for the product's high word, implicit bounds on the `addRaw` lemmas, camelCase limb names, and README rows for the Goldilocks files.
# Conflicts: # CompPoly/Fields/README.md
`Native.joinRows_cosetLde` proves that the joined row chunks of `cosetLde` are the packed `ldeSpec`: row `r`, column `j` is the column's Lagrange interpolant (`CLagrange.interpolatePow`) on the subgroup of order `2 ^ logH`, evaluated at `shift * omega ^ bitrev r`. `Native.cosetLde_eq` states each chunk as a row range of the spec. The transposes are proved by covering (`CosetLdeTransposes.lean`): `Fill T S N g` says `g` overwrites words only with their targets `T`, among them every word of `S`, and fills compose along folds. The strided 16 x 16 and eight-source 8 x 8 tile kernels, the row-tile loop and the scalar edges each fill their words, and together cover the output. The column transforms reuse the packed leaf proofs: `leafReversed` is the radix-four DIF stages (restoring `leaf16_encode`), `mulWords` the coordinate products. `getD_block` reduces a block entry to the interpolant's coefficient sum through `bitRevNat_concat`. The task graph reduces with `Tasks.get_bind`/`get_spawn`, now in `Data/Tasks`. Docs: the `lde` suite in the bench guide, the LDE pass in the benchmarking log, and a roadmap entry.
🤖 PR Summary
feat(ntt): proved batched coset LDE over packed KoalaBear matrices Statistics
Lean Declarations ✏️ Removed: 35 declaration(s)
✏️ Added: 1339 declaration(s)
…and 1189 more not listed. ✏️ Affected: 21 declaration(s) (line number changed)
Coverage Notes
Partially Analyzed Files
Last updated: 2026-10-28 23:01 UTC. |
Montgomery.Native32.Ext4 p is FastField p [X] / (X^4 - W) for any Mont32Field p with a Binomial4 p instance, which supplies W, a fast multiplication by W, and the irreducibility of X^4 - W over FastField p. Its spec field Ext4.Spec p is Ext over FastField p itself, so consumers never meet ZMod; specEquiv, the toSpec_* agreement lemmas and the byte codec carry over. The norm inverse gets its non-square W from the irreducibility. KoalaBear.Fast.Ext4 becomes the X^4 - 3 instance (mulW x = x + x + x), and BabyBear.Fast.Ext4 the X^4 - 11 instance, with Plonky3 known answers. Both transfer their ZMod certificates along ringEquiv with the new Extension.Map (ExtensionParams.map, irreducible_map_ringEquiv). FastField gets Fintype and card lemmas. Coset interpolation is respecced to the new spec field. Extension benchmarks and packed interpolation time the same as before.
The packed kernels, plan and refinement proofs work on FastField p for any Mont32Field p. The raw word operations take the modulus and montgomeryNegInv from the instance, ofWord checks against p, and every kernel takes p and is @[specialize]d; the unrolled generators emit add p. The plan calls the class PackedNTT p, whose run is Native.run p (run_eq), so a field's kernels are specialized and compiled once, in the CPU-targeted native library: Packed/Instances.lean provides KoalaBear and BabyBear, and their hot kernels disassemble to the instructions of the KoalaBear-only kernels. CompPolyNativeSmoke runs both instances against the DFT specification and the parallel path against the sequential one.
interpolateCoset, interpolateCosetPacked and their correctness proofs move to Fields/Montgomery and work over FastField p with any Binomial4 quartic extension; the weights' constants use Binomial4.mulW, and stored words are checked against the instance's modulus. The packed loops are @[specialize]d, and interpolateCosetPacked calls the class PackedInterpolation p, whose instances in KoalaBear/InterpolateCoset and BabyBear/InterpolateCoset (the roots of CompPolyInterpolateNative) compile them at the field's modulus. CompPolyNativeSmoke checks both instances against the reference.
# Conflicts: # CompPoly/Univariate/NTTFast/Packed/Arithmetic.lean # docs/wiki/benchmarking.md # lakefile.lean
LdePlan p, Native.cosetLde and its correctness proofs work on FastField p for any Mont32Field p; the minimum-form packed arithmetic takes the modulus from the instance, and leaf16, mulWords and the coset kernels take p and are @[specialize]d. LdePlan.cosetLde calls the class PackedCosetLde p, whose KoalaBear and BabyBear instances join the PackedNTT ones in Packed/Instances.lean; LdePlan.joinRows_cosetLde restates the correctness theorem for it, and CompPolyNativeSmoke checks both instances against ldeSpec.
# Conflicts: # CompPoly/Data/Tasks.lean
…r radix-four pass `lastHalf` checks its output once per call, writes the output's last word back unchanged (`touchLast`), and then stores its sixteen half lines with proved bounds (`store8`): the exclusivity check of the first store no longer holds the 128 lane values across a copy, so they stay in registers. `radix4Blocks` checks the whole pass once for quarters of whole sixteen-cell batches and runs the blocks in a machine-word loop (`blocks16Loop`), without the per-block natural-number checks.
`leaf16x2` runs the last four DIF layers of two sixteen-word blocks at once and multiplies every butterfly of the first three layers by its twiddle, the first one being one, so that each block's layers are uniform and the compiler vectorizes them. `tile8` checks its eight lines and its output tile once on machine words, writes the output's last word back unchanged and then loads and stores without checks, one output line at a time.
…s once per transform `leaf16Blocks` checks all its block pairs once and runs `leaf16x2` in a machine-word loop (`leaf16x2Loop`); `mulWords` does the same for its sixteen-word products (`mul16Loop`). Neither loop compares natural numbers per block any more.
…els, with the Keccak LDE timings
The packed multiplication subtracts the high word of the quotient times the modulus from the high word of the product and corrects the sign by an unsigned minimum, the quotient a product of words. The compiler multiplies a lane group with four 32-bit products and two word products, without 64-bit comparisons: the coset LDE of 2^15 × 1024 words is 24% faster on one thread.
…ps of four blocks Each of the four last DIF layers of a forward leaf runs in one pass over the leaf, a kernel call per four sixteen-word blocks: the kernel touches the buffer, reads its blocks, pairs each block's words in eight vector lanes and stores every block as two halves with proved stores. A layer's lanes are then one vector operation each, where the two-block kernel of all four layers left the compiler shuffling scalars. The passes are the layers block by block (fold_leaf16Stages), each kernel the layer's model on its blocks (leafStage8_encode and siblings, LeafStages.lean).
transposeRows moved its tiles across all columns for every sixteen rows, so each row of tiles stored one line into every column of a piece of up to 21 MB: a page per line. Down a block of up to 1024 rows before across, each column receives four kilobytes per block. The transpose of 2^15 rows of 2633 words takes 150 ms instead of 440 on one thread, 560 ms instead of 1080 for 2^16 rows.
A coset block of an odd leaf runs its first DIF layer out of place on the coefficients, each word multiplied by its coset table word as it is read (pair16Scaled), instead of storing the products in a pass of their own. pairsScaled_encode proves the pass the first layer of the products, and leafReversed_odd splits an odd leaf after that layer. An even leaf's first radix-four pass would read four quarters of the coefficients and of the table at a power-of-two stride, too many lines of one cache set, so its products still come first. The coset LDE of 2^15 × 2633 words runs 3% fewer instructions on one thread.
…nput transposes and fused coset products, with the Keccak LDE timings
Depends on #399 (and through it #395, #390). GitHub cannot use a fork branch as an upstream base, so this draft targets
mainand its diff includes the earlier PRs. Review only the new changes.Why
This is the next step toward STARKs in Lean. The prover's main cost is committing to the trace: each column is extended from its trace domain to a blown-up coset, and the rows are hashed into a Merkle tree. Plonky3 does this with
coset_lde_batch, followed by a row bit-reversal inTwoAdicFriPcs::commit. This PR adds that operation to CompPoly with a correctness proof, and it runs as fast as Plonky3 on mid-size and large matrices.What
NTTFast.Packed.Native.cosetLdetakes a row-major KoalaBear matrix packed as Montgomery words (like Plonky3'sRowMajorMatrix) and returns the extension in commitment order, as row chunks a commitment can hash in parallel.joinRowscopies the chunks into one buffer.Algorithm. In bit-reversed row order the output is$2^{bits}$ blocks of $h$ rows. Block $t$ is the coset $\text{shift} \cdot \omega^{\mathrm{bitrev}(t)} \cdot \langle \omega^{2^{bits}} \rangle$ in DIF order. So each column needs:
No padding or reversal pass is needed. The steps run as parallel tasks:
Proof.$(r, j)$ is column $j$ 's Lagrange interpolant ($\text{shift} \cdot \omega^{\mathrm{bitrev}(r)}$ .
Native.joinRows_cosetLdeproves the joined output equal toldeSpec: entryCLagrange.interpolatePow) atCosetLdeTransposes.lean).Arithmetic. Packed
add/sub/mulnow reduce with an unsigned minimum (vpminud). This makes the transform kernels about 10% faster.Benchmark.
--suite ldecompares against Plonky3 0.4.2'scoset_lde_batchwith three DFTs:Radix2DitParallel,RecursiveDftandRadix2DFTSmallBatch. Full output digests agree before timing.Performance
Eight physical cores of a Ryzen 7 3700X, shared host, three alternating rounds. Times are in milliseconds per call. The Plonky3 column is its fastest DFT for each shape.
Lean returns row chunks, while Plonky3 returns one buffer; joining the chunks costs one extra copy of the output. Small matrices are limited by task overhead. On a single thread, Plonky3's hand-written AVX2 butterflies are 2–3× faster per element; we make up the difference with parallelism. The benchmarking wiki logs the variants that were tried and rejected.