Skip to content

perf(BF64): check the Rabin certificate on GF(2) bitmasks - #412

Merged
alexanderlhicks merged 4 commits into
Verified-zkEVM:mainfrom
parithosh:gf2-rabin-certificate
Nov 18, 2026
Merged

alexanderlhicks merged 4 commits into
Verified-zkEVM:mainfrom
parithosh:gf2-rabin-certificate

Conversation

@parithosh

Copy link
Copy Markdown
Contributor

Stacked on #410; the diff shrinks to one commit once #410 merges.

Over GF(2) a polynomial is a bitmask: addition is xor and multiplication is carry-less. The new CompPoly.RabinCert.Binary (Data/Polynomial/RabinCertificateBinary.lean) checks a square-and-multiply chain that way. It proves toPoly_clmul (carry-less product is polynomial multiplication) and provides dvd_X_pow_sub_X_of_runChain / isCoprime_X_pow_sub_X_of_runChain with the same shape as the list versions, so the Rabin wrappers are unchanged. gen_rabin_certificate.py --binary (p = 2 only) emits the data, and BF64.BaseCert is regenerated with it; list mode stays byte-identical.

File (alternating runs, one 4-core machine) #410 version This PR
BF64/Basic.lean 27.4 s, 27.1 s 2.9 s, 2.6 s
BF64/BaseCertificate.lean 5.1 s, 5.3 s 2.2 s, 2.3 s

The file-level maxHeartbeats 400000 / maxRecDepth 4000 in BF64/Basic.lean are no longer needed. A deliberately wrong claimed result is rejected.

Checks. lake build --wfail, lake build --wfail CompPolyTests, lake exe axiomsweep --check. Per-declaration diff: basePoly_irreducible keeps its statement and axioms (propext, Classical.choice, Quot.sound); no existing axiom set changes.

Tradeoffs.

  • A second chain checker: the bitmask module mirrors the list checker's soundness proofs (~300 lines), so changes to the chain format need making twice.
  • Removed public declarations that existed only for the list encoding: BF64.baseCoeffs, BF64.toPoly_baseCoeffs, BF64.toPoly_replicate_zero. A downstream user of these would break.
  • The six data definitions in BF64.BaseCert change type from List ℕ / List Step to ℕ / List Binary.Step, and the data reads as large decimals rather than coefficient lists.
  • AES's GF(2) certificate still uses the list checker (about 1 s to gain; its file already defines a modulus that would clash with the generated one).
  • Same failure-message tradeoff as perf(BF64): check the Rabin chains with decide +kernel #410: a wrong certificate fails with an opaque kernel type mismatch.

ampagent and others added 2 commits November 5, 2026 10:44
`by rfl` evaluated each runChain certificate twice: once in the
elaborator's isDefEq, then again in the kernel. `decide +kernel` skips
the elaborator pass. BF64/Basic.lean goes from 51.2 s and 46.0 s to
26.4 s and 26.0 s (alternating runs, same orb). Statement and axioms
of basePoly_irreducible are unchanged.

Co-authored-by: Parithosh Jayanthi <parithosh@forkchoice.ai>
Amp-Thread-ID: https://ampcode.com/threads/T-01a1a66d-72db-7532-b204-db0dff8754a6
Over GF(2) a polynomial is a bitmask, addition is xor and multiplication
is carry-less. CompPoly.RabinCert.Binary checks a square-and-multiply
chain that way, with the same soundness statements as the list checker
(dvd_X_pow_sub_X_of_runChain, isCoprime_X_pow_sub_X_of_runChain), so the
Rabin wrappers are unchanged. gen_rabin_certificate.py --binary emits the
data; BF64.BaseCert is regenerated with it.

BF64/Basic.lean: 27.4 s, 27.1 s -> 2.9 s, 2.6 s; BF64/BaseCertificate.lean:
5.1 s, 5.3 s -> 2.2 s, 2.3 s (alternating runs, same orb, against the
decide +kernel version). The file-level maxHeartbeats/maxRecDepth options
are no longer needed. basePoly_irreducible keeps its statement and axioms;
the list-encoding helpers baseCoeffs, toPoly_baseCoeffs and
toPoly_replicate_zero are removed.

Co-authored-by: Parithosh Jayanthi <parithosh@forkchoice.ai>
Amp-Thread-ID: https://ampcode.com/threads/T-01a1a66d-72db-7532-b204-db0dff8754a6
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

🤖 PR Summary

Mathematical Formalization

  • Binary GF(2) bitmask representation of polynomials. Adds CompPoly/Data/Polynomial/RabinCertificateBinary.lean, which defines polynomials over GF(2) as bitmasks: addition is xor, multiplication is carry-less via new clmulBits / clmul. The toPoly bridge converts a bitmask to a polynomial in (ZMod 2)[X].
  • Key theorems. toPoly_clmul equates carry-less multiplication with polynomial multiplication. dvd_X_pow_sub_X_of_runChain and isCoprime_X_pow_sub_X_of_runChain prove the two Rabin-test conditions (trace and coprimality) from a Step-based runChain, mirroring the existing list-based checker.
  • No change to the public Rabin wrappers. The theorems share the same shape as their list counterparts, so irreducible_of_rabin_prime_power (used in basePoly_irreducible) is unaffected.

Proof Completion (sorries removed)

  • No sorry or admit placeholders are introduced; the new theorems are proved.

Protocols / Soundness

  • Core correctness. The binary checker is proved sound: a successfully checked chain implies the polynomial satisfies the Rabin irreducibility criterion.
  • Deliberately wrong certificate is rejected, as stated in the PR body.

Infrastructure / CI

  • Python script updated. scripts/gen_rabin_certificate.py gains a --binary flag (valid only for p=2) that emits the certificate data as ℕ bitmasks instead of List ℕ coefficient lists. Generated code conditionally imports RabinCertificateBinary and opens CompPoly.RabinCert.Binary.
  • New modulus definition. basePoly (x^64 + x^4 + x^3 + x + 1 over GF(2)) is now stored as the bitmask 18446744073709551643 in BaseCertificate.lean, replacing the baseCoeffs list.
  • BF64/BaseCertificate.lean completely rewritten. All six certificate data definitions (traceSteps, cop32Steps, cop32Rp, cop32W, cop32U, cop32V) change type from List ℕ / List Step to ℕ / List Binary.Step. Documentation updated to reflect the binary format.

Refactoring

  • BF64/Basic.lean
    • Replaces BF64.baseCoeffs and toPoly_baseCoeffs with BF64.modulus and toPoly_modulus (equating toPoly modulus to basePoly). The proof of basePoly_irreducible is updated to use toPoly_modulus.
    • Removes baseCoeffs, toPoly_baseCoeffs, and toPoly_replicate_zero from the public API. Downstream users of these would break.
  • CompPoly.lean now publicly imports CompPoly.Data.Polynomial.RabinCertificateBinary.
  • File-level maxHeartbeats / maxRecDepth in BF64/Basic.lean are no longer needed — performance gains make them unnecessary.

Documentation

  • docs/wiki/binary-fields-and-ntt.md: States that BaseCertificate.lean data consists of GF(2) bitmasks checked by CompPoly.RabinCert.Binary, and regeneration uses --binary.
  • docs/wiki/field-extensions.md: Adds paragraphs on the binary bitmask representation, the decide +kernel optimization for chain validation, and the performance impact (BF64 basic drops from ~27s to ~3s; certificate data from ~5s to ~2s). Usage instructions updated.
  • docs/wiki/generated-files.md: Updated regeneration command and consumer reference.

Performance

  • Major speedup. BF64/Basic.lean compiles in ~2.9s vs ~27s; BF64/BaseCertificate.lean in ~2.2s vs ~5s (per given benchmarks).

Notable side effects

  • Two chain-check codebases. The new bitmask module (~300 lines) mirrors the list checker; any future change to the chain format must be applied to both.
  • AES still uses the list checker; its GF(2) certificate is not converted (estimated ~1s gain).
  • Failure message tradeoff. A wrong certificate fails with an opaque kernel type mismatch (same issue as PR perf(BF64): check the Rabin chains with decide +kernel #410).

Statistics

Metric Count
📝 Files Changed 8
✅ Lines Added 505
❌ Lines Removed 556

Lean Declarations

✏️ Removed: 3 declaration(s)

CompPoly/Fields/Binary/BF64/Basic.lean (3)

  • def baseCoeffs : List ℕ
  • theorem toPoly_baseCoeffs : toPoly 2 baseCoeffs = basePoly
  • theorem toPoly_replicate_zero {p : ℕ} (n : ℕ) (rest : List ℕ) :
✏️ Added: 26 declaration(s)

CompPoly/Data/Polynomial/RabinCertificateBinary.lean (24)

  • @[simp] theorem toPoly_zero : toPoly 0 = 0
  • def chainExp : ℕ → List Step → ℕ
  • def checkStep (f cur : ℕ) (s : Step) : Bool
  • def clmul (a b : ℕ) : ℕ
  • def clmulBits : ℕ → ℕ → ℕ → ℕ
  • def runChain (f : ℕ) : ℕ → List Step → Option ℕ
  • noncomputable def toPoly (n : ℕ) : (ZMod 2)[X]
  • private theorem mod_add_mul_cancel {f q r : (ZMod 2)[X]} (hf0 : f ≠ 0) :
  • theorem checkStep_sound {f cur : ℕ} {s : Step} (h : checkStep f cur s = true) :
  • theorem coeff_toPoly (n i : ℕ) :
  • theorem dvd_X_pow_sub_X_of_runChain {fN : ℕ} {f : (ZMod 2)[X]} (hfN : toPoly fN = f)
  • theorem isCoprime_X_pow_sub_X_of_runChain {fN : ℕ} {f : (ZMod 2)[X]} (hfN : toPoly fN = f)
  • theorem runChain_sound {fN : ℕ} {f : (ZMod 2)[X]} (hfN : toPoly fN = f) (hf0 : f ≠ 0) :
  • theorem step_sound {fN cur : ℕ} {f : (ZMod 2)[X]} (hfN : toPoly fN = f) (hf0 : f ≠ 0)
  • theorem toPoly_clmul (a b : ℕ) : toPoly (clmul a b) = toPoly a * toPoly b
  • theorem toPoly_clmulBits (n a b : ℕ) :
  • theorem toPoly_congr {a b : ℕ} (h : ∀ i, a.testBit i = b.testBit i) : toPoly a = toPoly b
  • theorem toPoly_mod_two_pow_succ (a n : ℕ) :
  • theorem toPoly_one : toPoly 1 = 1
  • theorem toPoly_shiftLeft_one (a : ℕ) : toPoly (a <<< 1) = X * toPoly a
  • theorem toPoly_two : toPoly 2 = X
  • theorem toPoly_two_pow (k : ℕ) : toPoly (2 ^ k) = X ^ k
  • theorem toPoly_xor (a b : ℕ) : toPoly (a ^^^ b) = toPoly a + toPoly b
  • theorem xpow_mod_of_runChain {fN : ℕ} {f : (ZMod 2)[X]} (hfN : toPoly fN = f) (hf0 : f ≠ 0)

CompPoly/Fields/Binary/BF64/BaseCertificate.lean (1)

  • def modulus : ℕ

CompPoly/Fields/Binary/BF64/Basic.lean (1)

  • theorem toPoly_modulus : toPoly modulus = basePoly
✏️ Affected: 4 declaration(s) (line number changed)
  • def cop32Rp : ℕ in CompPoly/Fields/Binary/BF64/BaseCertificate.lean moved from L533 to L154
  • def cop32U : ℕ in CompPoly/Fields/Binary/BF64/BaseCertificate.lean moved from L543 to L160
  • def cop32V : ℕ in CompPoly/Fields/Binary/BF64/BaseCertificate.lean moved from L548 to L163
  • def cop32W : ℕ in CompPoly/Fields/Binary/BF64/BaseCertificate.lean moved from L538 to L157

sorry Tracking

  • No sorrys were added, removed, or affected.

📋 **Additional Analysis**

Findings

Performance contribution requirements

  • The diff claims a performance improvement ("from about 26 s to about 2 s") in the new file's docstring and in documentation updates, making it a performance contribution per the project's rules. The PR does not include any of the required elements for a performance contribution: no perf(...) title (not visible in the diff, but the PR description should follow the format), no A/B evidence table, no statement of which lake commands were run and their results, and no mention of lake exe axiomsweep --check passing.
  • The rules require that a performance PR contain concrete-input tests for the implementation. The new RabinCertificateBinary.lean has no separate test file (e.g. under tests/). The theorems in the file serve as proofs but are not the required test harness.

Syntax errors

  • Basic.lean uses (by decide +kernel) in three places. Lean does not recognize +kernel as part of a tactic term. Valid syntax would be by decide. This will cause a compilation failure.

Style and naming

  • The new file follows the project's naming conventions (camelCase for functions, snake_case for theorems, UpperCamelCase for types). No violations.

Documentation

  • The three wiki markdown files are updated to reflect the binary checker. This satisfies the requirement to update wiki pages when the PR changes commands or structure.
  • The generator script gen_rabin_certificate.py is updated to add the --binary flag; the changes are documented in the wiki.

Other checks

  • The PR adds a new file under CompPoly/ and modifies CompPoly.lean – the author should have run ./scripts/update-lib.sh and ./scripts/check-imports.sh; the diff shows only the expected single public import line, so this is likely done.
  • The axiom baseline (scripts/axiom_baseline.json) is not updated; the PR does not appear to add or close any sorry entries.

Summary
The PR is on the right track but needs to fix the syntax error, provide performance evidence, and add tests for the new checker to comply with the project's performance contribution rules.


📄 **Per-File Summaries**
  • CompPoly.lean: The file CompPoly.lean now publicly imports CompPoly.Data.Polynomial.RabinCertificateBinary, a new module that was not previously available. This adds the module's definitions (likely a binary variant of Rabin certificates for polynomials) to the project's public API, extending the existing Rabin certificate infrastructure.
  • CompPoly/Data/Polynomial/RabinCertificateBinary.lean: This new file introduces a binary bitmask representation for Rabin certificates over GF(2). It defines carry-less multiplication via clmulBits and clmul, a toPoly bridge mapping bitmasks to polynomials in (ZMod 2)[X], and a Step structure with checkStep, runChain, and chainExp for checking square-and-multiply chains. Key theorems include toPoly_clmul equating clmul with polynomial multiplication, dvd_X_pow_sub_X_of_runChain for the trace condition, and isCoprime_X_pow_sub_X_of_runChain for the coprimality condition, both derived from a checked chain.
  • CompPoly/Fields/Binary/BF64/BaseCertificate.lean: The file switches from the generic RabinCertificate module to the binary-specific RabinCertificateBinary module, and all Rabin-certificate data (traceSteps, cop32Steps, cop32Rp, cop32W, cop32U, cop32V) is rewritten to represent polynomials over GF(2) as bitmasks (ℕ values) instead of lists of coefficients. A new modulus definition (18446744073709551643) is added, and the documentation is updated to clarify that polynomials are stored as bitmasks and that consumers should use CompPoly.RabinCert.Binary.runChain for validation.
  • CompPoly/Fields/Binary/BF64/Basic.lean: Changed BF64/Basic.lean to replace the baseCoeffs list representation of the polynomial's coefficients with a bitmask modulus for the same polynomial x^64 + x^4 + x^3 + x + 1 over GF(2). The theorem toPoly_baseCoeffs (which related toPoly on baseCoeffs to basePoly) is removed, and a new theorem toPoly_modulus (asserting toPoly modulus = basePoly) is added. The proof of basePoly_irreducible, which uses irreducible_of_rabin_prime_power and the Rabin certification chain functions dvd_X_pow_sub_X_of_runChain and isCoprime_X_pow_sub_X_of_runChain, is updated to use toPoly_modulus instead of toPoly_baseCoeffs. The open imports are updated to include CompPoly.RabinCert.Binary and CompPoly.RabinCert (for irreducible_of_rabin_prime_power).
  • docs/wiki/binary-fields-and-ntt.md: Updated the documentation in docs/wiki/binary-fields-and-ntt.md to clarify that the Rabin certificate data in BaseCertificate.lean consists of GF(2) bitmasks and is checked by CompPoly.RabinCert.Binary, and to specify that the regeneration script should be invoked with the --binary flag. No other changes were made to this file.
  • docs/wiki/field-extensions.md: The documentation in field-extensions.md was updated with two notable additions. First, a new paragraph describes an optimization for the Rabin test chain validation: using by decide +kernel instead of by rfl to discharge a large chain's runChain hypothesis cuts the elaborator's redundant evaluation, reducing BF64/Basic.lean compilation from about 48 s to 26 s. Second, a new section explains the binary (bitmask) representation of polynomials over GF(2), referencing Data/Polynomial/RabinCertificateBinary.lean—which uses xor for addition and carry-less clmul for multiplication—and noting that the same chain-checking theorems feed the same wrappers, with scripts/gen_rabin_certificate.py --binary generating the data; for BF64's degree-64 modulus this dropped BF64/Basic.lean from about 27 s to 3 s and the certificate data module from about 5 s to 2 s. Additionally, the usage instructions for the irreducibility wrapper step were revised to reflect the switch to decide +kernel for chain checks while keeping Bézout checks with rfl, and to clarify the assembly through irreducible_of_rabin_prime_degree (prime degree) or irreducible_of_rabin_degree_six (composite degree).
  • docs/wiki/generated-files.md: Updated the documentation for CompPoly/Fields/Binary/BF64/BaseCertificate.lean: the regeneration command now includes --binary, and the runtime verification uses CompPoly.RabinCert.Binary.runChain (instead of the generic runChain), emitting polynomial coefficients as ℕ bitmasks for GF(2) rather than coefficient lists. The prose paragraph explaining that BaseCertificate.lean lives beside its field was also amended to describe the --binary output format and the Binary consumer.
  • scripts/gen_rabin_certificate.py: The script gains a --binary flag (valid only for p=2) that switches polynomial representation from List ℕ coefficient lists to ℕ bitmasks via new helper functions to_bits and steps_to_lean_binary. The generated Lean code conditionally imports RabinCertificateBinary instead of RabinCertificate, opens CompPoly.RabinCert.Binary, adds a modulus definition as a bitmask, and uses poly_def to produce either def ... : ℕ := ... or def ... : List ℕ := ... depending on the flag. The regeneration command wrapper _wrap_command also accepts the --binary argument to include it in the emitted shell lines.

Last updated: 2026-11-18 01:20 UTC.

@parithosh parithosh changed the title Gf2 rabin certificate perf(BF64): check the Rabin certificate on GF(2) bitmasks Nov 5, 2026
@alexanderlhicks
alexanderlhicks enabled auto-merge (squash) November 18, 2026 00:40
@alexanderlhicks
alexanderlhicks merged commit 79e3f93 into Verified-zkEVM:main Nov 18, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants