polygon/sync, stagedsync: add forkchoice finality changeset optimization

[madumas]

Summary

On Polygon PoS, Heimdall milestones provide deterministic finality: once a milestone is validated, the covered blocks can never be reorged. Today, erigon still generates changesets (undo data for reorgs) for these finalized blocks during syncToTip and event loop processing. This is wasted work. On a typical sync cycle of 200+ blocks, ~95% are already finalized by milestones.

This PR adds an opt-in --experimental.use-forkchoice-finality flag that skips changeset generation for blocks at or below the finalized milestone, and enables aggressive pruning for those blocks (matching initial-sync behavior).

How it works

Sync layer (polygon/sync/sync.go):

• Track lastFinalizedBlockNum as milestones are validated, both at-tip and ahead-of-tip. The value is updated monotonically via max() to handle out-of-order processing.
• In commitExecution(), resolve the best finalized header post-flush and pass it to UpdateForkChoice().
• Initialize lastFinalizedBlockNum from the latest waypoint in initialiseCcb().
• If syncToTip yields no waypoint (already at tip), fetch the latest milestone from Heimdall.

Execution layer (execution/stagedsync/exec3.go):

• getFinalizedBlockNum() reads the finalized hash from DB once before the execution loop (1 DB read, not per-block).
• shouldGenerateChangeSets() returns false for blocks <= finalized when the flag is enabled.
• Blocks without changesets get aggressive pruning (10h timeout + GreedyPruneHistory) and don't trigger the "batch can be pruned" early break, allowing larger batches like initialCycle.

Forkchoice (execution/eth1/forkchoice.go):

• Write ForkchoiceFinalized hash before the execution stage runs, so shouldGenerateChangeSets() can read it.

Design decisions

• Opt-in via --experimental.use-forkchoice-finality. This is an aggressive optimization with an irreversible trade-off (see warning below). The experimental prefix signals this.

• Finalized block cached once before execution loop. The finalized hash doesn't change mid-cycle (written in forkchoice.go before execution starts), so one DB read is sufficient. The parallel executor (exec3_parallel.go) still reads per-call since it runs concurrently.

• Monotonic lastFinalizedBlockNum via max(). Milestones can arrive out of order (an at-tip milestone for block 1000 may be processed after an ahead-of-tip milestone for block 1012). Using max() prevents regressing the finalized mark.

• Aggressive pruning matches initialCycle behavior. When we skip changesets, commitment history accumulates faster. Matching the initial-cycle pruning strategy (longer timeout, greedy history prune) prevents long-term commit degradation.

• "No unwindable block" warning downgraded to Debug. With fewer changesets generated, this warning fires frequently and is expected behavior, not an error condition.

Warning: chaindata reset required if finality is reverted

Without changesets, the node cannot unwind finalized blocks. If a milestone is later invalidated, the node will require a full chaindata reset to recover.

This is not a theoretical risk. On September 10, 2025, a faulty Heimdall milestone pushed Bor nodes onto divergent forks. The Polygon Foundation resolved it via hard fork (Bor v2.2.11-beta2, Heimdall v0.3.1) that deleted the faulty milestone from node databases. A node running with this flag during that incident would have needed a chaindata reset after the hard fork.

In practice, resetting chaindata on an archive node with snapshots takes minutes, and in such a scenario the node would be down anyway waiting for the hard fork binary. This is why the trade-off is acceptable as an opt-in flag.

Why not upstream?

This optimization is specific to Polygon's finality model. Heimdall milestones are a sidechain consensus mechanism that validates spans of blocks with deterministic finality. The plumbing here (lastFinalizedBlockNum tracking, milestone-to-forkchoice bridging in sync.go) is entirely Polygon-specific. The execution-layer changes shouldGenerateChangeSets, pruning) are generic but gated behind the flag, which has no effect without the Polygon sync layer populating the finalized block.