feat: add Charon DV support#1
Open
KaloyanTanev wants to merge 42 commits into
Open
Conversation
…ethpandaops#1429) ## What When `mev_type: buildoor` is set, automatically add the "build a payload on every slot" flag to the **first participant's** consensus client. ## Why Buildoor connects directly to the first participant's beacon node (`all_cl_contexts[0]`) and subscribes to its `/eth/v1/events?topics=payload_attributes` SSE stream to know when to start building. Beacon nodes only emit these events for slots they propose unless explicitly told to always prepare a payload — so without this flag buildoor wouldn't reliably trigger block building. Previously this flag had to be added manually via `cl_extra_params`. ## Changes In `enrich_mev_extra_params` (`src/package_io/input_parser.star`), gated on `mev_type == buildoor` and `index == 0`: | CL | Flag added | |---|---| | lodestar | `--emitPayloadAttributes=true` | | prysm | `--prepare-all-payloads` | | lighthouse | `--always-prepare-payload` | | grandine | `--features=AlwaysPrepareExecutionPayload` | `teku`, `nimbus` and `consensoor` have no equivalent flag, so the parser now **fails fast** with a clear message naming the offending client when one of them is the first participant. > Notes: > - The lighthouse flag is `--always-prepare-payload` (singular); it requires `--suggested-fee-recipient`, which the beacon node already sets when an EL is attached (`lighthouse_launcher.star`). > - consensoor support is being added separately (it needs to *emit* the `payload_attributes` SSE event first). ## Testing `kurtosis run --dry-run` confirmed: - lodestar + buildoor → parses and runs end-to-end (exit 0). - grandine + buildoor → parses and runs end-to-end (exit 0). - teku + buildoor → fails with the intended error message.
Follow-up to ethpandaops#1429. teku can emit `payload_attributes` on every slot via `-Xfork-choice-updated-always-send-payload-attributes`, so it can serve as buildoor's first participant. Wire up the flag and drop teku from the fail list. ## Changes `enrich_mev_extra_params` (`src/package_io/input_parser.star`), gated on `mev_type == buildoor` and `index == 0`: | CL | Flag added | |---|---| | lodestar | `--emitPayloadAttributes=true` | | prysm | `--prepare-all-payloads` | | lighthouse | `--always-prepare-payload` | | grandine | `--features=AlwaysPrepareExecutionPayload` | | consensoor | `--emit-payload-attributes` | | **teku** | **`-Xfork-choice-updated-always-send-payload-attributes=true`** | | nimbus | fail fast (no equivalent flag) | Only **nimbus** is left unsupported. ## Testing `kurtosis run --dry-run`: - teku + buildoor → parses and runs end-to-end (exit 0). - nimbus + buildoor → fails with the intended error message.
ethpandaops#1433) ## Problem ethpandaops#1430 wired teku as a buildoor first participant using `-Xfork-choice-updated-always-send-payload-attributes=true` (**single** dash). Teku rejects that as an unknown option, so teku + `mev_type: buildoor` fails to boot: ``` Unknown option: '-Xfork-choice-updated-always-send-payload-attributes=true' ``` `main` is currently broken for this combination. ## Root cause Teku's experimental (`-X`) options use the **`--X`** prefix (double dash), not `-X`. The correct option, confirmed via `teku --Xhelp` and `Eth2NetworkOptions.java`: ``` --Xfork-choice-updated-always-send-payload-attributes[=<BOOLEAN>] Calculate and send payload attributes on every forkChoiceUpdated regardless if a connected validator is due to be a block proposer ``` `[=<BOOLEAN>]` → `=true` is valid (bare also defaults to true). ## Fix One character: `-X` → `--X`. ```diff - "-Xfork-choice-updated-always-send-payload-attributes=true" + "--Xfork-choice-updated-always-send-payload-attributes=true" ``` ## Verification - The double-dash form parses (no "Unknown option") on `ethpandaops/teku:glamsterdam-devnet-6` for bare, `=true`, and `=false`. - `kurtosis run --dry-run` with teku + buildoor parses and runs end-to-end. (Earlier confusion came from testing with `--version`, which short-circuits before teku validates options and masks unknown-option errors.)
… group (ethpandaops#1434) Bumps the actions group with 1 update: [actions/checkout](https://github.com/actions/checkout). Updates `actions/checkout` from 6.0.3 to 7.0.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/checkout/releases">actions/checkout's releases</a>.</em></p> <blockquote> <h2>v7.0.0</h2> <h2>What's Changed</h2> <ul> <li>block checking out fork pr for pull_request_target and workflow_run by <a href="https://github.com/aiqiaoy"><code>@aiqiaoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2454">actions/checkout#2454</a></li> <li>Bump actions/publish-immutable-action from 0.0.3 to 0.0.4 in the minor-actions-dependencies group across 1 directory by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2458">actions/checkout#2458</a></li> <li>Bump flatted from 3.3.1 to 3.4.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2460">actions/checkout#2460</a></li> <li>Bump js-yaml from 4.1.0 to 4.2.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2461">actions/checkout#2461</a></li> <li>Bump <code>@actions/core</code> and <code>@actions/tool-cache</code> and Remove uuid by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2459">actions/checkout#2459</a></li> <li>upgrade module to esm and update dependencies by <a href="https://github.com/aiqiaoy"><code>@aiqiaoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2463">actions/checkout#2463</a></li> <li>Bump the minor-npm-dependencies group across 1 directory with 3 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2462">actions/checkout#2462</a></li> <li>getting ready for checkout v7 release by <a href="https://github.com/aiqiaoy"><code>@aiqiaoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2464">actions/checkout#2464</a></li> <li>update error wording by <a href="https://github.com/aiqiaoy"><code>@aiqiaoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2467">actions/checkout#2467</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/aiqiaoy"><code>@aiqiaoy</code></a> made their first contribution in <a href="https://redirect.github.com/actions/checkout/pull/2454">actions/checkout#2454</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v6.0.3...v7.0.0">https://github.com/actions/checkout/compare/v6.0.3...v7.0.0</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/actions/checkout/blob/main/CHANGELOG.md">actions/checkout's changelog</a>.</em></p> <blockquote> <h1>Changelog</h1> <h2>v7.0.0</h2> <ul> <li>Block checking out fork PR for pull_request_target and workflow_run by <a href="https://github.com/aiqiaoy"><code>@aiqiaoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2454">actions/checkout#2454</a></li> <li>Bump actions/publish-immutable-action from 0.0.3 to 0.0.4 in the minor-actions-dependencies group across 1 directory by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2458">actions/checkout#2458</a></li> <li>Bump flatted from 3.3.1 to 3.4.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2460">actions/checkout#2460</a></li> <li>Bump js-yaml from 4.1.0 to 4.2.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2461">actions/checkout#2461</a></li> <li>Bump <code>@actions/core</code> and <code>@actions/tool-cache</code> and Remove uuid by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2459">actions/checkout#2459</a></li> <li>upgrade module to esm and update dependencies by <a href="https://github.com/aiqiaoy"><code>@aiqiaoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2463">actions/checkout#2463</a></li> <li>Bump the minor-npm-dependencies group across 1 directory with 3 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/actions/checkout/pull/2462">actions/checkout#2462</a></li> </ul> <h2>v6.0.3</h2> <ul> <li>Fix checkout init for SHA-256 repositories by <a href="https://github.com/yaananth"><code>@yaananth</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2439">actions/checkout#2439</a></li> <li>fix: expand merge commit SHA regex and add SHA-256 test cases by <a href="https://github.com/yaananth"><code>@yaananth</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2414">actions/checkout#2414</a></li> </ul> <h2>v6.0.2</h2> <ul> <li>Fix tag handling: preserve annotations and explicit fetch-tags by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2356">actions/checkout#2356</a></li> </ul> <h2>v6.0.1</h2> <ul> <li>Add worktree support for persist-credentials includeIf by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2327">actions/checkout#2327</a></li> </ul> <h2>v6.0.0</h2> <ul> <li>Persist creds to a separate file by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2286">actions/checkout#2286</a></li> <li>Update README to include Node.js 24 support details and requirements by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2248">actions/checkout#2248</a></li> </ul> <h2>v5.0.1</h2> <ul> <li>Port v6 cleanup to v5 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2301">actions/checkout#2301</a></li> </ul> <h2>v5.0.0</h2> <ul> <li>Update actions checkout to use node 24 by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2226">actions/checkout#2226</a></li> </ul> <h2>v4.3.1</h2> <ul> <li>Port v6 cleanup to v4 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2305">actions/checkout#2305</a></li> </ul> <h2>v4.3.0</h2> <ul> <li>docs: update README.md by <a href="https://github.com/motss"><code>@motss</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1971">actions/checkout#1971</a></li> <li>Add internal repos for checking out multiple repositories by <a href="https://github.com/mouismail"><code>@mouismail</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1977">actions/checkout#1977</a></li> <li>Documentation update - add recommended permissions to Readme by <a href="https://github.com/benwells"><code>@benwells</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2043">actions/checkout#2043</a></li> <li>Adjust positioning of user email note and permissions heading by <a href="https://github.com/joshmgross"><code>@joshmgross</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2044">actions/checkout#2044</a></li> <li>Update README.md by <a href="https://github.com/nebuk89"><code>@nebuk89</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2194">actions/checkout#2194</a></li> <li>Update CODEOWNERS for actions by <a href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2224">actions/checkout#2224</a></li> <li>Update package dependencies by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2236">actions/checkout#2236</a></li> </ul> <h2>v4.2.2</h2> <ul> <li><code>url-helper.ts</code> now leverages well-known environment variables by <a href="https://github.com/jww3"><code>@jww3</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1941">actions/checkout#1941</a></li> <li>Expand unit test coverage for <code>isGhes</code> by <a href="https://github.com/jww3"><code>@jww3</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1946">actions/checkout#1946</a></li> </ul> <h2>v4.2.1</h2> <ul> <li>Check out other refs/* by commit if provided, fall back to ref by <a href="https://github.com/orhantoy"><code>@orhantoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1924">actions/checkout#1924</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/actions/checkout/commit/9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0"><code>9c091bb</code></a> update error wording (<a href="https://redirect.github.com/actions/checkout/issues/2467">#2467</a>)</li> <li><a href="https://github.com/actions/checkout/commit/1044a6dea927916f2c38ba5aeffbc0a847b1221a"><code>1044a6d</code></a> getting ready for checkout v7 release (<a href="https://redirect.github.com/actions/checkout/issues/2464">#2464</a>)</li> <li><a href="https://github.com/actions/checkout/commit/f0282184c7ce73ab54c7e4ab5a617122602e575f"><code>f028218</code></a> Bump the minor-npm-dependencies group across 1 directory with 3 updates (<a href="https://redirect.github.com/actions/checkout/issues/2462">#2462</a>)</li> <li><a href="https://github.com/actions/checkout/commit/d914b262ffc244530a203ab40decab34c3abf34d"><code>d914b26</code></a> upgrade module to esm and update dependencies (<a href="https://redirect.github.com/actions/checkout/issues/2463">#2463</a>)</li> <li><a href="https://github.com/actions/checkout/commit/537c7ef99cef6e5ddb5e7ff5d16d14510503801d"><code>537c7ef</code></a> Bump <code>@actions/core</code> and <code>@actions/tool-cache</code> and Remove uuid (<a href="https://redirect.github.com/actions/checkout/issues/2459">#2459</a>)</li> <li><a href="https://github.com/actions/checkout/commit/130a169078a413d3a5246a393625e8e742f387f6"><code>130a169</code></a> Bump js-yaml from 4.1.0 to 4.2.0 (<a href="https://redirect.github.com/actions/checkout/issues/2461">#2461</a>)</li> <li><a href="https://github.com/actions/checkout/commit/7d09575332117a40b46e5e020664df234cd416f3"><code>7d09575</code></a> Bump flatted from 3.3.1 to 3.4.2 (<a href="https://redirect.github.com/actions/checkout/issues/2460">#2460</a>)</li> <li><a href="https://github.com/actions/checkout/commit/0f9f3aa320cb53abeb534aeb54048075d9697a0e"><code>0f9f3aa</code></a> Bump actions/publish-immutable-action (<a href="https://redirect.github.com/actions/checkout/issues/2458">#2458</a>)</li> <li><a href="https://github.com/actions/checkout/commit/f9e715a95fcd1f9253f77dd28f11e88d2d6460c7"><code>f9e715a</code></a> block checking out fork pr for pull_request_target and workflow_run (<a href="https://redirect.github.com/actions/checkout/issues/2454">#2454</a>)</li> <li>See full diff in <a href="https://github.com/actions/checkout/compare/df4cb1c069e1874edd31b4311f1884172cec0e10...9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…thpandaops#1432) ## What Adds support for **multiple dedicated buildoor builder instances**, configured independently of the participants via `buildoor_params.instances` and enabled as an `additional_services` entry. ```yaml participants: - el_type: geth cl_type: lighthouse - el_type: reth cl_type: prysm additional_services: - buildoor # enables buildoor (like dora/spamoor/etc) buildoor_params: builder_api: true epbs_builder: true lifecycle: true instances: - participant: 1 # 1-based participant index count: 1 - participant: 2 count: 2 # two competing builders wired to participant 2 ``` This spins up `buildoor-lighthouse-geth-1`, `buildoor-reth-prysm-2-1`, and `buildoor-reth-prysm-2-2`, each wired to the named participant's own CL/EL. ## Why A block builder is independent of the network — it reads one CL's `payload_attributes` stream and (under ePBS) gossips bids to everyone. The previous model only allowed a single shared builder via `mev_type: buildoor` against participant 0. This lets you run many builders, target specific participants, and run more than one per participant. ## How - **buildoor as an additional_service** — buildoor only spins up when `buildoor` is in `additional_services` (consistent with other optional services), instead of being implied by config. With `buildoor` enabled and no `instances` set, it defaults to a single builder on the first participant. Mutually exclusive with the (deprecated) `mev_type: buildoor`. - **`buildoor_params.instances`** — `[{participant, count}]` declares which participants get builders and how many each. Validated for index range, duplicates, and count. - **Cleaner key derivation** — buildoor derives its builder BLS key itself via `--builder-mnemonic` + `--builder-key-index`, dropping the per-instance `ethdo` `run_sh` containers. - **Lifecycle / post-genesis onboarding (default on)** — each builder onboards itself after genesis via its lifecycle deposit (`--lifecycle`), so builders no longer need genesis registration and **gloas does not have to be at genesis** (works at any gloas epoch). Builder keys are derived at indices after any genesis-registered builders to avoid collisions. - **Block attribution** — each instance passes `--extra-data=<service-name>` so built blocks can be traced back to the builder that produced them. - **Service naming** — `buildoor-<cl>-<el>-<participant>` (with a `-<n>` suffix when count > 1), matching the cl/el naming convention. - **Deprecation** — `mev_type: buildoor` (single shared builder) is marked deprecated and will be dropped after the gloas fork; a runtime warning is printed. - **Docs + validation + tests** — README updated; sanity checks added for the new params; nightly tests added (`buildoor-instances.yaml`, `buildoor-instances-matrix.yaml`). ## Notes / limitations - The `--extra-data` flag and lifecycle/early-deposit onboarding require a buildoor build that supports them (in `glamsterdam-devnet-6`; not yet in `:main`). Bump the default `buildoor_params.image` once buildoor PRs ethpandaops#115/ethpandaops#110 land on main. - The CL has a single external-builder endpoint, so it is wired to its participant's first buildoor instance; additional instances still run (e.g. competing ePBS builders) but are not the CL's Builder-API endpoint. - Each instance uses a distinct prefunded account so concurrent buildoors don't collide on tx nonces; with a custom single-account `prefunded_accounts` they would share a wallet. ## Testing Validated with `kurtosis run . --dry-run`: - `additional_services: [buildoor]` + instances → all instances spin up. - `additional_services: [buildoor]`, no instances → one builder on participant 1. - instances set without `buildoor` in additional_services → clear failure. - gloas at a non-genesis epoch → builders spin up (lifecycle onboarding). - combining with `mev_type: buildoor` → conflict failure; deprecated `mev_type: buildoor` alone still works. - `black`-formatted. --------- Co-authored-by: Claude <noreply@anthropic.com>
Charon DV monitoring (local scraping of all Charon nodes/VCs) is kept; only the remote_write feature and the personal prometheus-package fork import are removed, so this branch depends solely on the upstream kurtosis-tech/prometheus-package. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces `ethpandaops/ethereum-genesis-generator:6.1.0` with `6.1.1`. Updated in: - `src/package_io/constants.star` - `network_params.yaml` - `README.md`
Adds the optional Prometheus remote_write feature on top of the cleaned charon branch: prometheus_params gains remote_write_url/token/relabel_configs (generic, inert unless a URL is set) and the prometheus import is pinned to the fork branch that implements remote_write. The Charon example documents the Charon-specific relabel rules. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat: add remote write via KaloyanTanev/prometheus-package fork
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add native support for Charon DV. More details can be found in the updated
README.md.One thing to note is that Charon does support Vouch VC, so it's been added as well. It is allowed only under Charon for the time being though.