Glamsterdam Shadow Fork Testing Plan

[misilva73]

1. Goals

Benchmarkoor measures isolated client re-execution of a single crafted block on a synthetic state. That is the right tool for pricing individual operations, and it is how we set the EIP-7904, EIP-8038 and EIP-2780 numbers. It cannot tell us whether a client stays a healthy network participant under sustained load.

This plan covers four gaps:

1. Block building. Can each client build a worst case block at the target gas limit in time?
2. CL-driven validation. Can clients validate blocks through the Engine API without missing attestation deadlines?
3. Syncing. Can fresh nodes catch up while the chain stays under load?
4. Mempool and propagation. Can transaction gossip, txpools and block propagation keep up?

Two conditions make the results meaningful, and both are missing from our current setup:

• Mainnet sized state. Our worst numbers come from cold, uncached state access. On a small synthetic state the caches hide the cost. The test_account_access result (8.6 Mgas/s on erigon) only appears with NO_CACHE against large code.
• Continuous high load. Test whether repeated heavy blocks cause disk saturation, growing queues or missed duties.

Use existing mainnet targets for sustained access tests and small batches of crafted contracts for single block worst cases (see 4.2).

Scope. Test the calibrated repricing bundle in a live network; deriving new gas costs is out of scope.

2. Metrics

2.1 Validator health

Metric	Notes
Missed slots	Proposals missed under load.
Orphaned and reorged blocks	Chain instability under load.
Missed and late attestations	Validators could not validate the head in time to attest.
Participation rate	Share of validators attesting.
Finalized and justified lag	Whether the chain is keeping up at all.
Invalid or rejected payloads	Record rejection reasons and client disagreements.

2.2 Resource load

Metric	Notes
CPU per EL and CL container	The basic saturation check.
Memory, including growth over the run	Track peak use and growth over time.
Disk IO (read and write throughput, IOPS, queue depth)	Track saturation during cold state access.
Datadir growth	State growth during each run.
engine_newPayload latency	Re-execution time as the CL actually sees it.
engine_getPayload latency	Payload retrieval latency; also record build duration and gas used.
engine_forkchoiceUpdated latency	Fork choice latency under load.
Txpool size and eviction	Pending, queued and discarded transactions.

2.3 Propagation and sync

Metric	Notes
Transaction and block propagation delay	Time from broadcast to receipt across nodes.
Block gas used / gas limit	Confirm the intended load reaches the chain.
Time to synced	From fresh-node startup to catching up with head, per client and sync mode.
Sync head lag	Whether a syncing node catches up and stays caught up.

3. Strategy

The core loop: fork mainnet, move it to Glamsterdam, then hold it under continuous spam, with the deployment-bound worst cases run as separate single block tests.

1. Fork mainnet. Sync a small set of EL and CL pairs to mainnet head.
2. Transition to Glamsterdam. Schedule the fork one or two epochs after shadow fork genesis.
3. Pre-position the crafted contracts. Deploy the max code size, JUMPDEST dense contracts in a setup phase, using a modified CL for speed.
4. Sustained spam. Use spamoor against random existing mainnet accounts for account reads, writes and ETH transfers, and XEN, top ERC-20s and random contracts for storage reads and writes. Deployment and create/selfdestruct scenarios create their own targets.
5. Single block worst cases. Against the contracts from step 3, submit one crafted block per worst case workload and measure build duration, engine_getPayload and engine_newPayload for that slot.
6. Test sync against the loaded chain. With the sustained spam of step 4 running, repeatedly launch fresh EL and CL pairs and measure time to synced per client.

4. Scenarios

4.1 What to test, and why

Priority follows the devnet-8 stateful benchmark results (suite 7eec9e8fcd40b131, 532 non baseline tests, four clients, against a 100 Mgas/s anchor). Clients below the anchor: reth 32 tests (6.0 percent), nethermind 60 (11.3), geth 100 (18.8), erigon 129 (24.2).

The Mode column records whether the workload can run as sustained load or only as a single block, per 4.2.

Priority	Workload	Mode	Benchmark evidence
P0	Account reads via cold account access	Sustained, random mainnet accounts	test_account_access (64 KiB, JUMPDEST/DIFF_MAX, NO_CACHE): 8.6 (erigon), 49.6 (geth), 59.5 (nethermind), 63.5 (reth) Mgas/s. 20 tests below the anchor on all clients.
P0	Large code loading at max code size	Single block only	Same 64 KiB tests: worst case 8.6 Mgas/s on erigon. Requires crafted contracts.
P0	ETH transfers to on-chain receivers	Sustained, random mainnet accounts	test_ether_transfers_onchain_receivers (64 KiB): worst case 29.9 Mgas/s on erigon; below the anchor in 68/72 configs. The 64 KiB variant is single block only.
P0	Account writes via value bearing cold calls	Sustained, random mainnet accounts	GAS_COLD_ACCOUNT_WRITE: 195,132 needed on erigon versus 6,700 current gas. Frozen bundle raised it from 8,000 to 9,000.
P1	Storage reads	Sustained, XEN, top ERC-20s and random contracts	test_sload_bloated (10 GB account, NO_CACHE): 81.4 Mgas/s on geth; 250–640 on other clients.
P1	Storage writes	Sustained, XEN, top ERC-20s and random contracts	test_sstore_bloated: 81.6 Mgas/s on geth; 89.3–97.1 on erigon.
P2	Contract deployment at 24 KiB and 64 KiB	Sustained	Expensive under EIP-8037 state gas. Measure during setup for the single block tests.
P2	Create and selfdestruct patterns	Sustained	test_selfdestruct_created: worst case 256 Mgas/s, above the anchor. Needs further testing.
P2	Worst case compute	Sustained	2,031 of 2,038 compute tests clear the anchor.

4.2 What can and cannot be sustained

Cold access to crafted 64 KiB contracts is tested one block at a time:

1. Cache reuse hides disk costs. Rotate accounts and distinct code, and check that accesses still cause disk reads.
2. Fresh code changes the state size. Assuming 60M gas blocks, 30,000 gas per target and 12-second slots, each block uses about 2,000 targets: 125 MiB of code per block, or 37 GiB per hour.
3. Preparation is expensive. At roughly 14M gas per deployment, before EIP-8037 increases, one test block needs about 500 deployment blocks. An hour of load needs weeks of preparation at full capacity, so we prioritize single block tests.

Sample random accounts from the forked mainnet state for sustained account reads, writes and ETH transfers. For storage reads and writes, use XEN, top ERC-20s and random contracts, with calls that exercise the intended slots. Check target counts and disk reads before choosing the run length. The large-code and bloated-state benchmarks motivate these tests; random targets may be less demanding.

For the 64 KiB worst cases, deploy only enough distinct contracts for one block (about 2,000), then measure build duration, engine_getPayload and engine_newPayload. This tests a single bad slot, not sustained behaviour.

4.3 Spamoor coverage

Assessed against Spamoor's scenarios and configs at commit 9224b73. Two knobs matter throughout: gas_limit: 0 makes a call burn all remaining block gas, and total_count bounds a spammer to a fixed number of transactions. Together they cover the single block cases of 4.2 as well as sustained load, so the gaps below are about targets, not about throughput control.

The gap in almost every P0 row is the same one: no scenario accepts a list of addresses. calltx call arguments support only {txid}, {random}, {random:N} and {randomaddr}, and {randomaddr} is a random address, so it is an empty account rather than an existing mainnet one. Random new addresses exercise state growth (already covered by statebloat-basics.yaml); our P0 rows need cold access to accounts that are already in the mainnet trie.

4.1 workload	Mode	Status	Available tooling / gap
P0 Account reads via cold account access	Sustained	Covered for synthetic targets	max-contract-read.yaml pairs factorydeploytx (CREATE2 deploys at deterministic addresses) with calltx calling AttackController.extcodehashAttack, which loops EXTCODEHASH over pseudo-random CREATE2 addresses until gas runs out. Targets are freshly deployed, so warmer on disk than untouched mainnet accounts. Mainnet sampling needs an address-list input.
P0 Large code loading at max code size	Single block	Partial	Same config's lastByteAttack does EXTCODESIZE plus EXTCODECOPY(size-32, 32) per target, forcing a full code load. Its fixture init code returns 24 KiB (0x6000) of mostly zeros; 64 KiB and JUMPDEST dense variants are new fixtures, accepted by factorydeploytx.init_code or deploytx --bytecodes-file.
P0 ETH transfers to on-chain receivers	Sustained	Gap	eoatx --to takes one fixed address and --random-target sends to random new addresses. Existing receivers need a sampled receiver list, or a geastx contract that CALLs an embedded address list (contract calls, not EOA transfers).
P0 Account writes via value bearing cold calls	Sustained	Gap	Same address-list gap. calltx --amount with --random-target gives cold writes to empty accounts, which is the EIP-8037 state growth case rather than the GAS_COLD_ACCOUNT_WRITE case; decide whether we want both.
P1 Storage reads	Sustained	Gap	No scenario loops SLOAD over an existing contract's slots. erc20_bloater writes two slots per sequential address but has no read path, and calling a view function such as balanceOf through calltx is one SLOAD per transaction. Needs an AttackController style read loop plus slot layouts for the chosen targets.
P1 Storage writes	Sustained	Partial	max-storage-update-write.yaml and storagespam fill a block with SSTOREs, but to sequential new slots in one contract. xentoken --xen-address can point at the real mainnet XEN contract on the fork, giving writes into genuinely bloated storage. Missing: updates of random existing slots, the test_sstore_bloated shape.
P2 Contract deployment at 24 KiB and 64 KiB	Sustained	Tooling covered, fixtures needed	deploytx --bytecodes-file and factorydeploytx.init_code both take arbitrary init code and run sustained. Needs size boundary and JUMPDEST dense fixtures; 64 KiB deployments depend on EIP-7907 being active on the fork.
P2 Create and selfdestruct patterns	Sustained	Covered	deploy-destruct: DeployTestChild's constructor creates nested children via CREATE and CREATE2 and destroys about 40 percent of them inside the same transaction, which is the EIP-6780 same-transaction pattern; the rest are destroyed in later transactions. Branch probabilities are hardcoded, so patterns are mixed rather than isolated.
P2 Worst case compute	Sustained	Covered	gasburnertx, geastx, test-opcodes.yaml and the perf-precompile-* configs.

Replaying our own benchmark fixtures. The replay-eest scenario runs EEST fixtures converted by spamoor-utils convert-eest, deploying the fixture pre-state and sending its transactions with post-execution state checks. Our benchmark blocks are EEST based, so the devnet-8 suite (7eec9e8fcd40b131) can be replayed against the shadow fork directly, at -t 1 roughly one worst case block per slot. This is the cheapest path to step 5 of section 3 and it reproduces exactly the tests the priorities in 4.1 come from. Caveat: the fixtures build their own small synthetic pre-state, so the replayed transactions do not get mainnet sized cold access unless we also point them at forked state.

Work needed:

1. Collect targets. Sample existing mainnet accounts. Select XEN, top ERC-20s and random contracts, and identify calls and slots for storage tests.
2. Add an address-list input. One target-list mechanism unblocks three P0 rows: either a {targetaddr} style placeholder fed from a file in calltx and eoatx, or a controller contract holding sampled addresses that loops account reads and value bearing calls over them. Check that calls perform the intended reads and writes.
3. Add a bloated-slot read loop. An AttackController variant doing random SLOADs over a slot range, pointed at erc20_bloater output and at real mainnet contracts.
4. Prepare fixtures. Build the 24 KiB and 64 KiB JUMPDEST dense init code for factorydeploytx, and convert the devnet-8 EEST fixtures for replay-eest. Prepare balances and permissions for storage calls; measure deployment during setup.

[LouisTsai-Csie]

Leaving some thoughts here:

P0 Account reads via cold account access

For the current scenario, I have a few concerns:

1. calltx may introduce additional execution overhead because the attack logic is implemented in Solidity rather than as minimal EVM bytecode.
2. The current logic repeatedly samples from the same set of 1,000 contracts. targetIndex is calculated as seed % contractCount, while contractCount is currently configured to 1000. A single high-gas transaction can execute more than 1,000 account accesses, so after enough iterations, many accesses will revisit accounts that are already warm within the transaction. This makes the workload less representative of the intended cold-account-access worst case.
3. The initcode for contract deployment using factory might not generate random bytecode each time.

TODO:

1. Revisit how target accounts are deployed and selected in max-contract-read.yaml.
2. Increase contractCount sufficiently to minimize duplicate accesses within a single transaction.
3. Simplify the attack logic, potentially by implementing it directly in EVM bytecode to reduce unrelated execution overhead.

[LouisTsai-Csie]

Found some useful contracts:

• Factory: 0x0000000000004946c0e9F43F4Dee607b0eF1fA1c
• Initcode: 0x746d4946c0e9f43f4dee607b0ef1fa1c3318585733ff6000526015600bf3
• Initcode hash: 0x3c1644c68e5d6cb380c36d1bf847fdbc0c7ac28030025a2fc5e63cce23c16348
• Salt range: [8,382,689, 10,561,146)
• Account state: Nonce 1, balance 0 (identical across all)
• Runtime code: 0x6d4946c0e9f43f4dee607b0ef1fa1c3318585733ff (21 bytes)
• Code hash: 0x1d93f60f105899172f7255c030301c3af4564edd4a48577dbdc448aec7ddb0ac

Note: Contains SELFDESTRUCT, this should not be triggered. Good for testing cold account access despite non-unique code.

[jochem-brouwer]

TODO: (also this is a note to self): add attack scenarios to default available spamoor scenarios, ready to execute on a live (test) network

[LouisTsai-Csie]

Address List Support:

• eoatx: feat(eoatx): support configurable receiver address list / CREATE2 target pools spamoor#289
• calltx: feat(calltx): support CREATE2 receiver address list as call data source spamoor#290
• feat(calltx): add CALL account-access attack contract spamoor#291