Skip to content

Conversation

LouisTsai-Csie
Copy link
Collaborator

@LouisTsai-Csie LouisTsai-Csie commented Sep 16, 2025

🗒️ Description

Enhanced Interface

In the original benchmark helpers, the repeated code pattern is structured as:

<setup><JUMPDEST><attack><attack>...<attack><JUMP> 

This PR adds a new cleanup section before each iteration, and the interface change accordingly:

<setup><JUMPDEST><attack><attack>...<attack><cleanup><JUMP> 

Refactoring target

Updating to benchmark wrapper

  • Update to benchmark test wrapper to handle eip7825 scenario
  • Use JumpLoopGenerator or ExtCodeGenerator for repeated pattern.

Unifying the variable names

  • Use setup for the code pattern before the bytecode loop (instead of code_prefix, calldata).
  • Use attack_block for the actual benchmark execution pattern (instead of code_loop_body, attack_iter, code_sequence, loop_body, code_segment, op_sequence, or iter_block).
  • Use cleanup for the phase after the execution loop (instead of code_suffix, code_loop_footer).
  • Use gas_benchmark_value for the benchmark gas limit (instead of attack_gas_limit).
  • Use tx_gas_limit for the transaction gas limit cap (instead of tx_gas_limit_cap) -> this should be avoided as the wrapper automatically handle such scenario.

Removing redundant parameters

  • Remove gas_benchmark_value as it is configured automatically
  • Remove fork, env param if not used in function

Implementation Note

Most of the test logic remains the same, but the following cases differ and may be worth a closer look from reviewers.

  • test_worst_returndatasize_zero
  • test_worst_keccak
  • test_worst_binop_simple
  • test_worst_unop
  • test_worst_calldatacopy
  • test_block_full_data
  • test_worst_blockhash

Some test cases have not been refactored, as they would require significantly more effort and may even need to be rewritten. I suggest skipping these for now and updating them in a separate PR:

  • test_worst_bytecode_single_opcode

Additionally, some test cases may fail on the Osaka fork:

  • test_worst_address_state_cold
  • test_worst_selfdestruct_existing
  • test_worst_selfdestruct_created
  • test_worst_selfdestruct_initcode

🔗 Related Issues or PRs

Requires PR #1956 and PR #1945
Relevant discussion: issue #1770

✅ Checklist

  • All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    uvx --with=tox-uv tox -e lint,typecheck,spellcheck,markdownlint
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered adding an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
  • Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
  • Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

@LouisTsai-Csie LouisTsai-Csie self-assigned this Sep 16, 2025
@LouisTsai-Csie LouisTsai-Csie changed the title Refactor benchmark tests refactor(benchmark): update to benchmark test wrapper Sep 16, 2025
@LouisTsai-Csie LouisTsai-Csie force-pushed the refactor-benchmark-tests branch 2 times, most recently from eb1fc44 to 619f1cf Compare September 19, 2025 07:11
@LouisTsai-Csie LouisTsai-Csie force-pushed the refactor-benchmark-tests branch from 41bc03e to cc03678 Compare September 25, 2025 06:08
@LouisTsai-Csie LouisTsai-Csie marked this pull request as ready for review September 25, 2025 08:48
@LouisTsai-Csie LouisTsai-Csie force-pushed the refactor-benchmark-tests branch from 110d7b6 to c8fc5fa Compare September 26, 2025 05:33
@LouisTsai-Csie LouisTsai-Csie force-pushed the refactor-benchmark-tests branch from beb239d to b51eba1 Compare September 29, 2025 09:24
pre=pre,
post={},
tx=tx,
code_generator=JumpLoopGenerator(setup=setup, attack_block=Op.POP(Op.RETURNDATASIZE)),
)


def test_worst_returndatasize_zero(
Copy link
Collaborator Author

@LouisTsai-Csie LouisTsai-Csie Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For test_worst_returndatasize_zero case, the test implementation has been changed.

In the original version, the logic followed a POP & PUSH pattern. This can be refactored so that contract A only calls the RETURNDATASIZE operation until reaching the max stack size, while contract B repeatedly performs STATICCALLs to contract A.

In this refactored approach, we can use ExtCallGenerator as a helper.

Related issue #1968


state_test(
benchmark_test(
pre=pre,
post={},
tx=tx,
)


def test_worst_keccak(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For test_worst_keccak test, the implementation has been changed.

The original pattern:

JUMPDEST
PUSH20 length
loopcode
loopcode
loopcode
POP
PUSH0
JUMP

This could be refactored to the following to avoid additional POP operations.

PUSH20 length
JUMPDEST
loopcode
loopcode
loopcode
PUSH0
JUMP

And the second pattern follows the JumpLoopGenerator pattern

@@ -2083,7 +1443,7 @@ def test_worst_jumpdests(
ids=lambda param: "" if isinstance(param, tuple) else param,
)
def test_worst_binop_simple(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For test_worst_binop_simple case, the implementation has been changed.

TBD

@@ -2122,35 +1474,21 @@ def test_worst_binop_simple(

@pytest.mark.parametrize("opcode", [Op.ISZERO, Op.NOT])
def test_worst_unop(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For test_worst_unop, the test case have been changed in the implementation.

Original pattern:

JUMPDEST
PUSH0
loopcode
loopcode
...
loopcode
POP
PUSH0
JUMP

It is changed to the following and updated to JumpLoopGenerator helper:

PUSH0
JUMPDEST
loopcode
loopcode
...
loopcode
POP
JUMP


state_test(
@pytest.mark.parametrize("zero_byte", [True, False])
def test_block_full_data(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For test_block_full_data, its implementation has been updated to apply eip7825 transaction gas limit cap.

)


def test_worst_blockhash(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation has been updated for test_worst_blockhash.

Optimize the case using the ExtCallGenerator and upgrade to apply eip7825 transaction gas limit cap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant