Repository navigation
perf(fields): faster Goldilocks, inlined 32-bit Montgomery add - #392
Conversation
…ocks Fold the 64x64 product's middle terms without carry bookkeeping so clang emits one widening multiply for squarings too; `mul x x` inlined into `square` used to compile to a four-imul chain. Run the Fermat inverse and the pow ladder on unreduced words (`mulLazy`, `squareNLazy`, `invLazy`, `powLazy`) with a single canonicalization at the end, and make `squareN` tail recursive. Correctness rests on word-level cast lemmas (`foldUInt128Lazy_cast`, `mulLazy_cast`, `squareNLazy_cast`, `invLazy_cast`, `powLazy_cast`). A/B against master on the same core: inv 311 -> 167 ns, pow 99 -> 74 ns, mul and add unchanged.
`conditionalSubtract` was a plain def, so every KoalaBear and BabyBear `add` paid an out-of-line call. A/B: add latency 844 -> 606 ps, throughput 876 -> 166 ps.
…nreduced words `univariate-dense-goldilocks` now runs the native-word carrier beside ZMod like the other fast fields, with rows named in the shared `-fast` scheme. Add tests that unreduced `ofUInt64` words compare and multiply as their residues.
Compute the canonical sum as `x - (p - y)`: the subtraction borrows exactly when `x + y < p`, so one borrow-selected correction by `2^32 - 1` yields the reduced result, three dependent instructions instead of a carry check plus a separate compare against `p`. `addRaw_toNat` carries the bound and the cast lemma; the carry-based reducer and its lemmas are removed. A/B against master on the same core: add latency 1021 -> 616 ps, throughput 381 -> 230 ps; mul, inv and pow unchanged.
… the Goldilocks fold Two changes to the 128-bit fold behind every fast Goldilocks multiply. The borrow of the low word against the top limb has probability about 2^-32, but a Lean `if` compiled to a select on the critical path; moving that arm into a `@[noinline]` function forces a predicted branch. The middle term `hi_lo * (2^32 - 1)` compiled to a three-cycle `imul` inside loops (clang holds the constant in a register); it is now `(hi <<< 32) - hi_lo`, two one-cycle operations. `shiftLeft32_sub_low` and `foldTail_cast` carry the proofs; the fold lemma cases on the borrow. A/B against master on the same core: mul latency 3077 -> 2485 ps, throughput 815 -> 739 ps, inv 314 -> 136 ns, pow 101 -> 64 ns; add unchanged.
One-sentence docstrings, `2 ^ k` literals instead of decimals, a `_toNat` bridge for the product's high word, implicit bounds on the `addRaw` lemmas, camelCase limb names, and README rows for the Goldilocks files.
🤖 PR SummaryThis PR accelerates the native-word Goldilocks field and KoalaBear/BabyBear addition, entirely in Lean (no externs, no Performance Change
Statistics
Lean Declarations ✏️ Removed: 9 declaration(s)
✏️ Added: 19 declaration(s)
📋 **Additional Analysis**The diff is a large performance-oriented refactor of the Goldilocks fast field implementation, introducing lazy arithmetic kernels (mulLazy, powLazy, invLazy, foldUInt128Lazy) that defer canonicalization, replacing the previous add/mul/inv kernels and their proofs. It also adds the missing FastReduction correctness module, updates docs and the benchmark driver, and adds test coverage. All parts of the diff are directly relevant to the provided instructions (contributing guidelines, performance PR requirements, new types/codec rules, style guide, etc.). No part of the diff is irrelevant to the supplied instructions. I will structure the review around the sections the instructions define: Pull Request Guidelines, Performance Contributions, Style and Naming Guidelines, and Documentation/References. I will also check for any violations of other miscellaneous rules in the instructions (e.g., the Answer: 📄 **Per-File Summaries**
Last updated: 2026-10-20 10:56 UTC. |
# Conflicts: # CompPoly/Fields/README.md
|
Measured after merging this PR into the benchmark branch (#390). Goldilocks improved substantially:
Five interleaved A/B pairs, comparing Multiplication throughput was the one regression; a second five-pair run confirmed it (1.180 → 1.239 ns, 5% more time). Addition throughput received A fresh, separate five-pair comparison against Plonky3 0.4.2 puts Goldilocks add latency at parity, mul latency at 1.28× Rust's time, inversion at 0.44×, and exponentiation at 0.87×. Full Lean/Rust results and method. Full build, tests, style/documentation checks, and the axiom sweep passed; all 24 cross-language result checks matched. |
dhsorens
left a comment
There was a problem hiding this comment.
I'm happy with this, thanks!
Resolve CompPoly/Fields/README.md by taking Verified-zkEVM#392's Goldilocks rows.
Speeds up the native-word Goldilocks field and the KoalaBear/BabyBear addition. Pure Lean, no externs, no
native_decide. The carrier stays{ x : UInt64 // x.toNat < p }, so theFunction.Injective.fieldtransfer and the codec are untouched.wideMuluses the schoolbook shape clang folds into one wideningmul, forx * xas well asx * y; once Lean inlinedmul x xintosquarethe old shape compiled to fourimuls, which everyinv/powsquaring paid@[noinline]cold function, so it becomes a predicted branch instead of acmovon the critical path; the middle term is(hi <<< 32) - hiLoinstead of a multiply by2^32 - 1, which clang turned into a three-cycle registerimulinside loopsaddisx - (p - y)with one borrow-selected correction:sub; lea; cmovaeinv(Fermat chain) andpow(runtime square-and-multiply) run on unreduced words (mulLazy,squareNLazy,invLazy,powLazy) with a single canonicalization at the end;squareNis tail recursivewideMul_snd_toNatbridges the product's high word to ℕ, a cast lemma per lazy kernel (foldUInt128Lazy_cast,mulLazy_cast,squareNLazy_cast,invLazy_cast,powLazy_cast),shiftLeft32_sub_lowfor the middle term, bound and cast foraddRaw; the carry-based reducer and its lemmas are gone@[inline] conditionalSubtractinMontgomery/Native32.lean: KoalaBear and BabyBearaddpaid an out-of-line call per operationunivariate-dense-goldilocksgets fast rows beside ZMod like the other fast fields, rows named in the shared-fastscheme; tests guard unreducedofUInt64wordsPinned A/B against master, same core,
--medium: Goldilocks mul latency 3.08 → 2.49 ns, throughput 815 → 739 ps, add 1.02 → 0.62 ns and 381 → 230 ps, inv 314 → 136 ns, pow 101 → 64 ns; KoalaBear/BabyBear add 0.84 → 0.61 ns and 876 → 167 ps. Every other rowsame. Against Plonky3 (rev 069855b,--release, same chains, same core): Goldilocks add and pow at parity, inv 2.4x ahead (Fermat vs binary GCD), mul 1.35x behind.The remaining mul gap is the final canonical subtract, about 0.7 ns, plus Plonky3's inline asm
sbbcarry trick. AQuotientcarrier that drops the canonical invariant was built and proved but is not viable: Lean boxes everyQuotvalue, one allocation per multiply, and a plain structure cannot be aField(x + -x = 0fails on unreduced words). Lazy words are therefore confined to chains. Binary GCD inversion, plonky2's shorter chain, amulHiextern and Montgomery form were all measured and lost.