Skip to content

Conversation

@mempirate
Copy link
Collaborator

Description

Instead of verifying signatures on the system API, it just extracts the signing peer address from the header.

mempirate and others added 26 commits September 18, 2025 16:13
Changed to `buildernet-orderflow-proxy`
Co-authored-by: File Large <[email protected]>
Co-authored-by: Jonas Bostoen <[email protected]>
Fix the self-filtering behaviour like in Go proxy
Pins Rust nightly formatter and TOML formatter to reduce the number of
annoying diffs.

Adds a small justfile for recurring commands: clippy, fmt, test.
Blocked by:
- Merge of flashbots/rbuilder#716
- Knowing the response of `eth_sendBundle`
Makes integration tests more robust by mocking a local builder, reading
the data received by that local builder and using that to complete
tests.
This PR introduces an optional Clickhouse indexer configurable from CLI.
It tracks full bundles received from the user endpoints and writes them
to the database.

The goal of this is to being able to replicate real flow in a testing
environment, as such it provides also testing code from converting
clickhouse data to `RawBundle`s.
Should add benchmark results somewhere to keep track
Adds the `X-BuilderNet-Sent-At` HTTP header to any requests going over
the system API. This is useful for timing measurements. Format is
expected to be in UNIX microseconds.
Got it by some `Clap`s footguns, and indexing args were not really
optional. Now they are. Unit tests are also added to test this
functionality.

CLI logs:

```
#### NO CLICKHOUSE ARGS
 ~/oss/buildernet-orderflow-proxy-v2_flashbots  on lore/fix/optional-indexing  cargo run -- --user-listen-url 0.0.0.0:9754 --system-listen-url 0.0.0.0:9755 --builder-listen-url 0.0.0.0:8756 --builder-url http://0.0.0.0:2020 --builder-hub-url http://localhost:3000
   Compiling buildernet-orderflow-proxy v0.1.0 (/Users/birb/oss/buildernet-orderflow-proxy-v2_flashbots)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.08s
     Running `target/debug/buildernet-orderflow-proxy --user-listen-url '0.0.0.0:9754' --system-listen-url '0.0.0.0:9755' --builder-listen-url '0.0.0.0:8756' --builder-url 'http://0.0.0.0:2020' --builder-hub-url 'http://localhost:3000'`
2025-09-30T10:43:53.305946Z  INFO buildernet_orderflow_proxy::indexer: Running with mocked indexer
2025-09-30T10:43:53.306307Z  WARN buildernet_orderflow_proxy: No orderflow signer was configured, using a random signer. Fix this by passing `--orderflow-signer <PRIVATE KEY>`
2025-09-30T10:43:53.308121Z  INFO buildernet_orderflow_proxy: Orderflow signer configured address=0x1b1F00ce3C5fBE7A5404Fb0116A8F7ceFFBff646
Error: error sending request for url (http://localhost:3000/api/l1-builder/v1/register_credentials/orderflow_proxy)

Caused by:
   0: client error (Connect)
   1: tcp connect error
   2: Connection refused (os error 61)

Location:
    /Users/birb/oss/buildernet-orderflow-proxy-v2_flashbots/src/builderhub.rs:63:24

#### PARTIAL CLICKHOUSE ARGS
 ~/oss/buildernet-orderflow-proxy-v2_flashbots  on lore/fix/optional-indexing  cargo run -- --user-listen-url 0.0.0.0:9754 --system-listen-url 0.0.0.0:9755 --builder-listen-url 0.0.0.0:8756 --builder-url http://0.0.0.0:2020 --builder-hub-url http://localhost:3000 --clickhouse.user pronto
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.33s
     Running `target/debug/buildernet-orderflow-proxy --user-listen-url '0.0.0.0:9754' --system-listen-url '0.0.0.0:9755' --builder-listen-url '0.0.0.0:8756' --builder-url 'http://0.0.0.0:2020' --builder-hub-url 'http://localhost:3000' --clickhouse.user pronto`
error: the following required arguments were not provided:
  --clickhouse.host <HOST>
  --clickhouse.password <PASSWORD>
  --clickhouse.database <DATABASE>

Usage: buildernet-orderflow-proxy --user-listen-url <USER_LISTEN_URL> --system-listen-url <SYSTEM_LISTEN_URL> --builder-listen-url <BUILDER_LISTEN_URL> --builder-url <BUILDER_URL> --clickhouse.host <HOST> --clickhouse.user <USERNAME> --clickhouse.password <PASSWORD> --clickhouse.database <DATABASE> --builder-hub-url <BUILDER_HUB_URL>

For more information, try '--help'.

#### ALL CLICKHOUSE ARGS
 ~/oss/buildernet-orderflow-proxy-v2_flashbots  on lore/fix/optional-indexing  cargo run -- --user-listen-url 0.0.0.0:9754 --system-listen-url 0.0.0.0:9755 --builder-listen-url 0.0.0.0:8756 --builder-url http://0.0.0.0:2020 --builder-hub-url http://localhost:3000 --clickhouse.user pronto --clickhouse.password pronto --clickhouse.database pronto --clickhouse.host http://127.0.0.1:12345
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.34s
     Running `target/debug/buildernet-orderflow-proxy --user-listen-url '0.0.0.0:9754' --system-listen-url '0.0.0.0:9755' --builder-listen-url '0.0.0.0:8756' --builder-url 'http://0.0.0.0:2020' --builder-hub-url 'http://localhost:3000' --clickhouse.user pronto --clickhouse.password pronto --clickhouse.database pronto --clickhouse.host 'http://127.0.0.1:12345'`
2025-09-30T10:44:27.445219Z  INFO buildernet_orderflow_proxy::indexer: Running with clickhouse indexer host=http://127.0.0.1:12345
2025-09-30T10:44:27.646456Z  WARN buildernet_orderflow_proxy: No orderflow signer was configured, using a random signer. Fix this by passing `--orderflow-signer <PRIVATE KEY>`
2025-09-30T10:44:27.647287Z  INFO buildernet_orderflow_proxy: Orderflow signer configured address=0xA2fFaDC2B569b135a95C5Abc0095f84775E61215
2025-09-30T10:44:27.653720Z ERROR indexer: bundle tx channel closed, indexer will stop running
2025-09-30T10:44:27.653726Z ERROR indexer: transaction tx channel closed, indexer will stop running
Error: error sending request for url (http://localhost:3000/api/l1-builder/v1/register_credentials/orderflow_proxy)

Caused by:
   0: client error (Connect)
   1: tcp connect error
   2: Connection refused (os error 61)

Location:
    /Users/birb/oss/buildernet-orderflow-proxy-v2_flashbots/src/builderhub.rs:63:24

 ~/oss/buildernet-orderflow-proxy-v2_flashbots  on lore/fix/optional-indexing                                                                                                                                   1 ✘  system Node  at 12:44:27
```
Introduces support for bundle receipt indexing via parquet file,
configurable using `indexer.parquet.bundle-receipts-file-path` flag.
Parquet indexing is currently incompatible with Clickhouse indexing,
meaning that only one of the two can be chosen. File-based indexing has
been introduce for Shadow simulation support, because the isolated
network cannot talk with external servers (like Clickhouse).

**Limitations**

Parquet file format requires a writer to close the file before it can be
read successfully. The reason is that when the writer closes the file,
it flushes also a footer containing the metadata of what has been
written. As such, inspecting the resulting parquet files implies the
orderflow proxy indexing tasks must have been stopped.

To overcome this limitation, usually a file-rotation mechanism is
employed, where a new file is created every minuted/hour/etc. For now,
it is still TBD whether we want to go with this approach or we would
migrate to something simpler like CSV.
A root of some problems was a double flattened option. Now better CLI
tests are also added
This PR implements graceful shutdown of the orderflow proxy long-lived
tasks by adding a global cancellation token and list of tasks. Upon
detecting a shutdown signal or failure, all subscribed tasks are
notified and their cleanup is performed before exiting.

This is still fairly primitive, a next iteration could look something
like [reth_tasks](https://reth.rs/docs/reth_tasks/index.html) (ty
mempirate for resource)
## Description

Properly handle non-bytes parameter in `eth_sendRawTransaction` method.
## Description

Avoid re-encoding raw transactions when forwarding.
Revamp / continuation of #13

---------

Co-authored-by: Pierre-Louis Roman <[email protected]>
Co-authored-by: Lorenzo <[email protected]>
@mempirate mempirate requested a review from thedevbirb as a code owner October 3, 2025 14:25
@mempirate
Copy link
Collaborator Author

@claude review sir

@claude
Copy link

claude bot commented Oct 3, 2025

Claude finished @mempirate's task —— View job


Code Review Complete ✓

Tasks

  • Read the modified file and understand changes
  • Analyze security implications
  • Review code quality and best practices
  • Check for test coverage
  • Provide detailed review feedback

Summary

This PR removes cryptographic signature verification from the system API handler, replacing it with simple address extraction from the header. While the code changes are clean and well-implemented, this introduces a critical security vulnerability that needs to be addressed before merging.

🔴 Critical Issues

1. Authentication Bypass Vulnerability (src/ingress/mod.rs:280-282)

The change from maybe_verify_signature() to maybe_extract_signer() removes cryptographic proof of identity. This allows any client with HTTPS access to impersonate any peer address by simply setting the header.

Impact: An attacker can:

  • Send malicious bundles/transactions claiming to be from trusted peers
  • Bypass rate limiting and priority mechanisms tied to specific entities
  • Potentially disrupt the orderflow routing system

The comment states: "we can assume the channel (HTTPs session) is authenticated"

  • ⚠️ Standard HTTPS only authenticates the server to the client, NOT the client to the server
  • If this relies on mutual TLS (mTLS), this assumption must be validated and documented
  • If behind an authenticated proxy, this should be explicitly configured and validated

⚠️ Major Concerns

2. Missing Test Coverage

  • No tests exist for maybe_extract_signer() function
  • No tests validate the system_handler's authentication behavior
  • The existing test file (tests/ingress.rs) only tests the user handler flow
  • Risk: Security-critical code changes without test coverage

3. Insufficient Documentation

  • The security model and assumptions are not documented
  • No explanation of what "authenticated channel" means in practice
  • The maybe_extract_signer() function needs a security warning in its docs

✅ Positive Aspects

  1. Clean Implementation: The code changes are minimal and well-structured
  2. Proper Error Handling: Error messages updated appropriately
  3. Good Documentation: The new function has inline documentation (though needs security warnings)

📋 Recommendations

High Priority:

  1. Add Authentication Validation:

    // Option A: Validate mTLS client certificate
    if let Some(cert) = request.peer_cert() {
        let cert_address = extract_address_from_cert(cert)?;
        ensure!(cert_address == claimed_address, "Certificate mismatch");
    }
    
    // Option B: Add feature flag
    if config.system_api_requires_signature_verification {
        maybe_verify_signature(&headers, &body, USE_LEGACY_SIGNATURE)
    } else {
        maybe_extract_signer(&headers)
    }
  2. Add Comprehensive Tests:

    • Test that system_handler rejects requests with invalid/missing peer addresses
    • Test that system_handler correctly identifies known peers
    • Test behavior with unknown peer addresses
    • Add integration tests for the authentication flow
  3. Document Security Model:

    • Create/update documentation explaining the trust model
    • Document deployment requirements (mTLS configuration, network topology, etc.)
    • Add security warning to maybe_extract_signer() function

Medium Priority:

  1. Consider Feature Flag: Instead of removing signature verification entirely, make it configurable
  2. Add Metrics: Track authentication failures on system API for monitoring
  3. Audit Peer Registry: Ensure ingress.forwarders.find_peer() has proper access controls

🤔 Questions for Author

  1. What mechanism ensures the HTTPS session is authenticated? Is mTLS configured?
  2. Is this deployment behind a trusted proxy that handles authentication?
  3. Have you considered the attack surface if the peer registry is compromised?
  4. Why was signature verification removed rather than making it optional?

Verdict

❌ I recommend not merging this PR until the security concerns are addressed. The removal of cryptographic verification needs either:

  • Strong compensating controls (e.g., validated mTLS)
  • Clear documentation of the security assumptions
  • Test coverage proving the security model works as intended

The code quality is good, but the security implications are too significant to merge without additional safeguards.


Review Focus Areas: Security ⚠️ | Code Quality ✓ | Testing ⚠️ | Documentation ⚠️ | Performance ✓


[Job Run](https://github.com/flashbots/buildernet-orderflow-proxy-v2/actions/runs/${{ github.run_id }}) | Branch

if let Some(address) = maybe_verify_signature(&headers, &body, USE_LEGACY_SIGNATURE) {
// NOTE: We don't verify the signature here, because we can assume the channel (HTTPs
// session) is authenticated.
if let Some(address) = maybe_extract_signer(&headers) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Critical Security Issue: Bypassing Signature Verification

This change removes cryptographic signature verification for system API requests, replacing it with simple address extraction. This creates a significant security vulnerability:

Issues:

  1. Authentication Bypass: An attacker can claim to be any address by simply setting the header without proving ownership of the private key
  2. Trust Assumption: The comment states "we can assume the channel (HTTPs session) is authenticated" but:
    • HTTPS only authenticates the server, not the client (unless using mTLS)
    • If this relies on mTLS, that should be explicitly documented and validated
    • The header can be set arbitrarily by any client with HTTPS access

Attack Scenario:

// Attacker can impersonate any peer by setting:
// X-Flashbots-Signature: <any_valid_peer_address>:fake_signature
// Since we only extract the address and don't verify the signature

Recommendations:

  1. If mTLS is used, verify the client certificate's subject matches the claimed address
  2. If this is intentional for a specific deployment (e.g., behind authenticated proxy), add explicit configuration flag
  3. Document the security model and assumptions clearly
  4. Consider adding a feature flag to enable/disable signature verification rather than removing it entirely


/// Parse [`FLASHBOTS_SIGNATURE_HEADER`] header and extract the signer of the request (without
/// verifying the signature). For signature verification, use [`maybe_verify_signature`].
pub fn maybe_extract_signer(headers: &HeaderMap) -> Option<Address> {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Documentation Needed

While this helper function is well-documented, the security implications of using it instead of maybe_verify_signature should be clearly stated:

Suggested change
pub fn maybe_extract_signer(headers: &HeaderMap) -> Option<Address> {
/// Parse [`FLASHBOTS_SIGNATURE_HEADER`] header and extract the signer of the request (without
/// verifying the signature). For signature verification, use [`maybe_verify_signature`].
///
/// # Security Warning
///
/// This function does NOT verify the signature and should only be used in contexts where
/// authentication is handled by other means (e.g., mTLS). Using this function without
/// additional authentication allows any client to impersonate any address.
pub fn maybe_extract_signer(headers: &HeaderMap) -> Option<Address> {

@mempirate mempirate marked this pull request as draft October 7, 2025 07:26
@mempirate
Copy link
Collaborator Author

Converted to draft because even though with TLS, the client can verify the server, it doesn't work the other way around by default. So we need the signature verification to authenticate the client to the server, until we maybe do something like mTLS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants