diff --git a/development-updates.md b/development-updates.md index c193b58a..7781d634 100644 --- a/development-updates.md +++ b/development-updates.md @@ -125,7 +125,7 @@ The actual project development is the main part of the program. Post your regula | [Abhivansh](https://github.com/akronim26) | [Update 6](https://hackmd.io/@astaxxoo/BkYy61VrMx) | [Update 7](https://hackmd.io/@astaxxoo/HyHHzW6rMg) | [Update 8](https://hackmd.io/@astaxxoo/H1ntay_LGe) | [Update 9](https://hackmd.io/@astaxxoo/SJqUMvfvfg) | [Update 10](https://hackmd.io/@astaxxoo/r1rVUgFPfg) | [Update 11](https://hackmd.io/@astaxxoo/BJBkj7QOze) | [Update 12](https://hackmd.io/@astaxxoo/SyzuPBiuGg) | [Update 13](https://hackmd.io/@astaxxoo/Sy4ZLYNKMl) | [Update 14](https://hackmd.io/@astaxxoo/Sk4cyeJ9Gx) | [Update 15](https://hackmd.io/@astaxxoo/rJSEnov5Me) | [Update 16](https://hackmd.io/@astaxxoo/Sy7nMdfizl) | [Update 17](https://hackmd.io/@astaxxoo/rJYhVB5oMx) | [Update 18](https://hackmd.io/@astaxxoo/rkxfb_H2fe) | [Update 19](https://hackmd.io/@astaxxoo/Hk7yqinhfx) | | | [Aiden](https://github.com/sgtSong) | | [Update 7](https://hackmd.io/@25ukZNryRaCcOYnb7k0kLA/SkksWaM8fg) | [Update 8](https://hackmd.io/@25ukZNryRaCcOYnb7k0kLA/rkPry9aIMe) | | | | | | | | | | | | | | [Arsh](https://github.com/ArshLabs) | [Update 6](https://hackmd.io/@arshlabs/rJAxRDXBfl) | [Update 7](https://hackmd.io/@arshlabs/r1LJ-FarMx) | [Update 8](https://hackmd.io/@arshlabs/SJyiEwdUze) | [Update 9](https://hackmd.io/@arshlabs/r17wQFywMl) | [Update 10](https://hackmd.io/@arshlabs/H1bPFrtPGg) | [Update 11](https://hackmd.io/@arshlabs/BJ4wxFzOzl) | [Update 12](https://hackmd.io/@arshlabs/By7KqQJtzg) | [Update 13](https://hackmd.io/@arshlabs/ryMGytHFfg) | [Update 14](https://hackmd.io/@arshlabs/rJJn1npKfe) | [Update 15](https://hackmd.io/@arshlabs/S19iHlvcfx) | [Update 16](https://hackmd.io/@arshlabs/rkgNMqXizl) | [Update 17](https://hackmd.io/@arshlabs/ryFWMVGhze) | [Update 18](https://hackmd.io/@arshlabs/Bkf_tKdhfe) | [Update 19](https://hackmd.io/@arshlabs/rJAIF_k6Gx) | [Update 20](https://hackmd.io/@arshlabs/H1AsDDe6Mx) | - | [Astrion](https://github.com/astrion-coder) | | | [Update 5,6,7,8](https://hackmd.io/@Astrion/rJKFH49Izx) | [Update 9](https://hackmd.io/@Astrion/HkblAvmPGx) | [Update 10](https://hackmd.io/@Astrion/SJakDEiPMl) | | [Update 11 & 12](https://hackmd.io/@Astrion/ByjdxDeYGg) | | [Update 13 & 14](https://hackmd.io/@Astrion/rkgS_OWcfe) | [Update 15](https://hackmd.io/@Astrion/BJBcp3q5Gl) | [Update 16](https://hackmd.io/@Astrion/ryKxLMVjfx) | | | | | | + | [Astrion](https://github.com/astrion-coder) | | | [Update 5,6,7,8](https://hackmd.io/@Astrion/rJKFH49Izx) | [Update 9](https://hackmd.io/@Astrion/HkblAvmPGx) | [Update 10](https://hackmd.io/@Astrion/SJakDEiPMl) | | [Update 11 & 12](https://hackmd.io/@Astrion/ByjdxDeYGg) | | [Update 13 & 14](https://hackmd.io/@Astrion/rkgS_OWcfe) | [Update 15](https://hackmd.io/@Astrion/BJBcp3q5Gl) | [Update 16](https://hackmd.io/@Astrion/ryKxLMVjfx) | | | | [Update 17-20](https://hackmd.io/@Astrion/Hk75AJt6fe) | | | [Ayush](https://github.com/0xAysh/) | | [Update 6 & 7](https://hackmd.io/@0xAysh/EPF-7-week-7-update) | | | [Update 8-10](https://hackmd.io/@0xAysh/week-10) | | [Update 11 & 12](https://hackmd.io/@0xAysh/week-12) | [Update 13](https://hackmd.io/@0xAysh/week-13) | [Update 14](https://hackmd.io/@0xAysh/week-14) | | [Update 15&16](https://hackmd.io/@0xAysh/week-16) | | [Update 17 & 18](https://hackmd.io/@0xAysh/week-18) | | | | [Bankat](https://github.com/mananuf/) | | | | | | | | | | | | | | | | | [Blessing](https://github.com/dicethedev/) | [Update 6](https://hackmd.io/@dicethedev/SyoRCkSBGe) | [Update 7](https://hackmd.io/@dicethedev/Hkwft-PIMx) | [Update 8](https://hackmd.io/@dicethedev/HkAHjBxDMg) | [Update 9](https://hackmd.io/@dicethedev/rk3n0SePze) | [Update 10](https://hackmd.io/@dicethedev/HkXdFcCFzx) | [Update 11](https://hackmd.io/@dicethedev/ByUvt5CtMe) | [Update 12](https://hackmd.io/@dicethedev/ryXziqRFfx) | [Update 13](https://hackmd.io/@dicethedev/HJ0ajqRYMg) | [Update 14](https://hackmd.io/@dicethedev/By4wh5CYGe) | | | | | | | @@ -181,7 +181,7 @@ The final weeks of the cohort should mark the finalization of your project. Subm | [Abhivansh](https://github.com/akronim26) | [Light Client Protocol in Teku](https://hackmd.io/@astaxxoo/HJdb9g6hMe) | | | [Aiden](https://github.com/sgtSong) | | | | [Arsh](https://github.com/ArshLabs) | | | -| [Astrion](https://github.com/astrion-coder) | | | +| [Astrion](https://github.com/astrion-coder) | [Final Report: EIP-8435](https://hackmd.io/@Astrion/SkwpEt_6zl) | [Slides](https://astrion-coder.github.io/epf_final_presentation_eip_8435/) | | [Ayush](https://github.com/0xAysh/) | | | | [Blessing](https://github.com/dicethedev/) | | | | [Charles](https://github.com/creese) | | | diff --git a/projects/last-written-block-pbt.md b/projects/last-written-block-pbt.md new file mode 100644 index 00000000..2b4224fd --- /dev/null +++ b/projects/last-written-block-pbt.md @@ -0,0 +1,139 @@ +# State tiering for the Partitioned Binary Tree (EIP-8435) + +This project defines the `last_written_block` write-age signal inside the account and storage leaves of the Partitioned Binary Tree. + +## Motivation + +Ethereum's state keeps growing as users create new accounts and storage slots. The article "[Not All State is Equal](https://ethereum-magicians.org/t/not-all-state-is-equal/25508)" analyzes the state from the genesis block to block 22,431,083 and finds it uneven. Most contracts go inactive right after creation, a majority of storage slots are never touched again after their first write, and a small number of contracts and factories account for most of the state. Around 80% of the full node state has not been accessed in over a year ([EIP-8188 ACD Slides](https://weiihann.github.io/20260521-eip8188-acd/)). + +Clients can use a write-age signal to split the state into a "hot" part, written recently, and a "cold" part, not written in a long time. That lets them optimize the commit-critical mutable path: trie updates, state-root computation and other work that repeatedly rewrites state. [EIP-8188](https://eips.ethereum.org/EIPS/eip-8188) defines the signal as a consensus-verified `last_written_block` field on every account and storage slot, and state tiering designs such as [EIP-8295](https://eips.ethereum.org/EIPS/eip-8295) build on it. + +EIP-8188 defines the field for the Merkle Patricia Trie (MPT) in RLP, as a fifth element in the account list and a two-element list around each storage slot. That design has two problems: + +1. **EIP-8188 was Declined for Inclusion in Hegotá** ([EIP-8081](https://eips.ethereum.org/EIPS/eip-8081)), so the MPT version of the signal will not ship in that fork. +2. **The state tree is expected to change.** [EIP-8297](https://eips.ethereum.org/EIPS/eip-8297) proposes replacing the hexary MPT with a Partitioned Binary Tree (PBT), expected around the I* fork. + +The PBT has no RLP. It packs account fields at fixed byte offsets in one 32-byte `BASIC_DATA` leaf, a storage leaf holds only the raw 32-byte word, and there is no `storageRoot`. So EIP-8188's encoding cannot carry over to the new tree. For the tiering signal to exist after the switch, an EIP has to define it for the PBT. That EIP should activate together with the PBT, so the tree has the right layout from the start. + +## Project description + +[EIP-8435: Last-Written Block in PBT Leaves](https://eips.ethereum.org/EIPS/eip-8435), co-authored with my mentor, puts `last_written_block` into the leaves of the PBT: + +* For accounts, the EIP packs `last_written_block` into the existing `BASIC_DATA` leaf, so accounts grow by zero bytes. +* For storage slots, the leaf value grows from 32 to 36 bytes. It holds the unchanged slot value followed by a 4-byte block number. +* The update rules follow EIP-8188, except that a storage write updates only the slot's field, not the account's. +* The field is part of each leaf's committed value, so a client can read it from the leaf and verify it with a normal inclusion proof. + +EIP-8435 is standalone. It does not require EIP-8188 and can activate whether or not EIP-8188 ever activates on the MPT. It requires [EIP-7954](https://eips.ethereum.org/EIPS/eip-7954), [EIP-8037](https://eips.ethereum.org/EIPS/eip-8037) and [EIP-8297](https://eips.ethereum.org/EIPS/eip-8297). It was merged into the [EIPs repo](https://github.com/ethereum/EIPs) as a Draft on October 24, 2026. + +## Specification + +### Account encoding + +EIP-8297 packs the `BASIC_DATA` leaf as follows: + +| Name | Offset | Size | +| ----------- | ------ | ---- | +| `version` | 0 | 1 | +| `reserved` | 1 | 3 | +| `code_size` | 4 | 4 | +| `nonce` | 8 | 8 | +| `balance` | 16 | 16 | + +EIP-8435 changes it to: + +| Name | Offset | Size | +| -------------------- | ------ | ---- | +| `version` | 0 | 1 | +| `last_written_block` | 1 | 4 | +| `code_size` | 5 | 3 | +| `nonce` | 8 | 8 | +| `balance` | 16 | 16 | + +`last_written_block` takes the three reserved bytes plus one byte freed by narrowing `code_size` from four bytes to three. Three bytes still hold up to 16,777,215 bytes, about 256 times the 64 KiB code size limit of [EIP-7954](https://eips.ethereum.org/EIPS/eip-7954). The earlier unified binary tree design, [EIP-7864](https://eips.ethereum.org/EIPS/eip-7864), used the same `code_size` width and offset. + +### Storage encoding + +Every storage slot leaf, both header slots 0 to 63 and slots in the storage zone, holds a 36-byte value: + +| Name | Offset | Size | +| -------------------- | ------ | ---- | +| `slot_value` | 0 | 32 | +| `last_written_block` | 32 | 4 | + +The slot value stays at offset 0, so `SLOAD` reads the first 32 bytes as before. A write of zero still deletes the leaf, metadata included. EIP-8188 estimated about 1.5 billion slots in early 2026. Across those slots this adds about 6 GB of raw data, less than the roughly 9 GB EIP-8188 adds with its RLP framing. With BLAKE3, a storage leaf's hash input is at most 103 bytes, which still fits in two 64-byte blocks. + +### Update rules + +Each state change sets `last_written_block` to the current block number: + +* **Storage.** A value change or a new slot sets the slot leaf's field. Deleting a slot removes the leaf and its metadata. A no-op write changes nothing. +* **Accounts.** These events set the account's field: nonzero balance transfers for sender and receiver, nonce increments including [EIP-7702](https://eips.ethereum.org/EIPS/eip-7702) authorizations, nonzero priority fee credits to the fee recipient, nonzero [EIP-4895](https://eips.ethereum.org/EIPS/eip-4895) withdrawals, new accounts, and `SELFDESTRUCT` balance changes under [EIP-6780](https://eips.ethereum.org/EIPS/eip-6780). +* **Reads and reverts.** Reads never update the field. Reverts restore it together with its leaf. +* **No storage-to-account cascade.** In the MPT, every storage write already rewrites the account because the account stores `storageRoot`. The PBT has no `storageRoot`, so a cascade would add an extra `BASIC_DATA` write for every storage write. Each leaf records its own last write instead. + +### Retrieval + +A client can read the `last_written_block` of any account or storage slot at any time. Reading it needs only the leaf, with no transaction execution, no side index and no state change. The read works the same way from a client's primary database, a cold store, a snapshot or a block witness. `leaf_hash` covers the field, so a standard EIP-8297 inclusion proof also proves it. + +### State-creation gas and activation + +* EIP-8435 increases `STATE_BYTES_PER_STORAGE_SET` from [EIP-8037](https://eips.ethereum.org/EIPS/eip-8037) by 4 bytes. `STATE_BYTES_PER_NEW_ACCOUNT` does not change. +* EIP-8435 should activate in the same fork as EIP-8297. Activating it later would mean rewriting every `BASIC_DATA` leaf and every storage leaf. +* State that exists at activation has `last_written_block = 0` until its first write. + +### Key design decisions + +* **Why 4 bytes?** Three bytes (16,777,215) is already below mainnet's block height. Four bytes last until block `2^32`, about 1,600 years at 12-second slots. +* **Why the raw block number?** A client can derive any coarser value a tiering or pricing scheme needs, such as EIP-8295 periods, from the block number, but not the other way round. +* **Why pack into `BASIC_DATA`?** Every account access already opens that leaf. A separate header leaf would add one leaf per account and one extra write per account update. +* **Why widen the storage leaf?** All 32 bytes of a storage word are meaningful, so there is no room to pack the field. A parallel metadata leaf would double the number of storage leaves. Widening keeps the slot and its metadata in one leaf, so clients read, write, delete and prove them together. + +## Roadmap + +| Phase | Weeks | Deliverables | +| --- | --- | --- | +| 1: Design | 15-16 | Account and storage leaf encoding, update rules, retrieval and the state-creation gas change. EIP draft, PR to the EIPs repo and a discussion thread on Ethereum Magicians. | +| 2: Review and merge | 17-20 | Pass the EIPs repo CI (`eipw`, markdownlint, codespell), go through EIP editor review and merge as a Draft on October 24, 2026. Gather feedback from PBT contributors and the Magicians thread. | + +> **Note.** I spent weeks 0-14 on the original version of this proposal, a Reth implementation of EIP-8188 state tiering, before the project changed direction in week 15. + +## Possible challenges + +* **EIP-8297 is still a Draft.** The PBT layout and its hash function are not final. EIP-8435 has to track any changes to `BASIC_DATA`, the storage leaf or the hashing. +* **Timeline.** EIP-8435 is meant to ship with EIP-8297, which is expected around the I* fork, roughly two years away. Presenting it at ACD only makes sense closer to that point. Until then, the main task is keeping the EIP current and getting feedback. +* **Fixed width.** Unlike EIP-8188's RLP field, a 4-byte field cannot grow on its own. A later fork must widen it before mainnet reaches block `2^32`. +* **Test vectors.** Pinning digests has to wait until EIP-8297 chooses the merkelization hash. +* **Feedback.** EIP-8435 needs input from the people designing the PBT, so that it fits the tree they are building. + +## Goal of the project + +Success looks like: + +* **EIP-8435 merged into the EIPs repo** as a Draft that defines the account and storage encoding, the update rules, retrieval and the state-creation gas change for the PBT. Merged on October 24, 2026. +* **A standalone design** that does not depend on EIP-8188 activating, and that state tiering proposals such as EIP-8295 and EIP-8296 can use. +* **An open discussion** on Ethereum Magicians, with feedback from people working on the PBT. +* **Consideration for inclusion** of EIP-8435 together with EIP-8297, in the longer term. + +## Collaborators + +### Fellows + +[Astrion](https://github.com/astrion-coder) + +### Mentors + +[Wei Han Ng](https://github.com/weiihann) + +## Resources + +* [EIP-8435: Last-Written Block in PBT Leaves](https://eips.ethereum.org/EIPS/eip-8435) +* [EIP-8435 PR](https://github.com/ethereum/EIPs/pull/12420) +* [EIP-8435 discussion thread](https://ethereum-magicians.org/t/eip-8435-last-written-block-in-pbt-leaves/29860) +* [EIP-8297: Partitioned Binary Tree](https://eips.ethereum.org/EIPS/eip-8297) +* [EIP-8188: Last-Written Block for Accounts and Slots](https://eips.ethereum.org/EIPS/eip-8188) +* [EIP-8295: State Tiering by Periods](https://eips.ethereum.org/EIPS/eip-8295) +* [EIP-8081: Hardfork Meta - Hegotá](https://eips.ethereum.org/EIPS/eip-8081) +* [EIP-7864: Ethereum state using a unified binary tree](https://eips.ethereum.org/EIPS/eip-7864) +* [Not All State is Equal](https://ethereum-magicians.org/t/not-all-state-is-equal/25508) +* [EIP-8188 ACD Slides](https://weiihann.github.io/20260521-eip8188-acd/) diff --git a/projects/reth-state-tiering.md b/projects/reth-state-tiering.md deleted file mode 100644 index b49689d6..00000000 --- a/projects/reth-state-tiering.md +++ /dev/null @@ -1,226 +0,0 @@ -# State Tiering using EIP-8188 and EIP-8295 for Reth - -Implementing Ethereum state tiering through `last_written_block` and `last_written_period` signal to allow for efficient state management. - -## Motivation - -Ethereum's state is ever evolving. Being one of the most used cryptocurrencies worldwide, there is a constant influx of account data, storage slot data and multiple other things. According to the article "[Not All State is Equal](https://ethereum-magicians.org/t/not-all-state-is-equal/25508)" which contains a detailed analysis of the Ethereum state from **genesis** block to block **22,431,083**, the Ethereum state is very uneven: most contracts go inactive right after creation, a majority of storage slots are never touched again after their first write, and a small number of contracts and factories account for most of the state. Out of the full node state around **80%** has been unaccessed for over a year ([EIP-8188 ACD Slides](https://weiihann.github.io/20260521-eip8188-acd/)) - -This proposal intends to form the baseline for splitting the Ethereum state into a "hot" and "cold" one based on a write-age signal. In other words, "cold" state would be states which have not had a write operation in a long time and "hot" state would be those which have had a recent write operation. This allows individual clients to optimize the commit-critical mutable path, say things like trie updates, state-root computation and other work associated with repeatedly rewriting state. - -## Project description - -This project contains a complete implmentation of the signals described in [EIP-8188](https://eips.ethereum.org/EIPS/eip-8188) and [EIP-8295](https://eips.ethereum.org/EIPS/eip-8295) for Reth following the [Geth Implementation](https://github.com/weiihann/go-ethereum/tree/feat/eip-8188/inject) by the original author. The following diagram contains a simplistic description of the hot-cold state separation. - -``` -hot state cold state -─────────────────────────── ───────────────────────────────────────── -standard MPT nodes (RLP) -+ stubs pointing to ──→ blobs where each blob = one originally-stubbed subtree -``` - -The "hot" state is stored in a database where write access is easier and cheaper (RocksDB) and "cold" state is basically a simple append-only flat file. Commands that allow us to backfill the necessary data into the state and tier the states accordingly will be added to the `reth db` suite: - -1. `reth db inject-periods` : This command is going to allow us to enter the necessary `period` data, as described in [EIP-8295](https://eips.ethereum.org/EIPS/eip-8295) into the state of the chain. - -2. `reth db convert-inactive` : This command is going to identify the inactive states, convert them into a format writable in the append-only flat file and connect the necessary stubs in the hot state to the blobs in the cold state. This hot-cold separation will ensure the tiering happens appropriately. - -Some other support commands may be added as necessary. - -## Specification - -### Data Collection - -To collect the necessary data, we will be using data collected through [Xatu](https://github.com/ethpandaops/xatu) and stored in ClickHouse. The ClickHouse instance allows us to extract necessary data using simple SQL queries. - -#### Account Data - - To collect all account write events in the specified block range, we will query balance changes table (`canonical_execution_balance_diffs`), nonce updates table (`canonical_execution_nonce_diffs`), storage modifications table (`canonical_execution_storage_diffs`) and contract creations table (`canonical_execution_contracts`). We will then group the results by account address and select the max `block_number` for each account. This value will correspond to the account’s `last_written_block`, following the definition of “Account Modified” as defined under the “Account Rules” section in [EIP-8188](https://eips.ethereum.org/EIPS/eip-8188), since it represents the most recent block in which the account was modified. - -#### Storage Slot Data - -To collect all necessary storage slot data, we will look at `canonical_execution_storage_diffs` table to find which storage slot changes have taken place within the given range . We will ignore all write where the new value is zero (`0x0`, `0x00`, or `0`), because [EIP-8188](https://eips.ethereum.org/EIPS/eip-8188) only tracks storage slots still allocated after the write. We will then group the results by account address and storage slot, and select the maximum `block_number` for each pair. This value is the storage slot's `last_written_block`, i.e. the latest block in which that non-zero storage slot was modified. - -From this `last_written_block` data, `last_written_period` is calculated as follows: - -``` -last_written_period = max(0, (last_written_block - PERIOD_START_BLOCK) // PERIOD_LENGTH) -``` - -This definition comes from [EIP-8295](https://eips.ethereum.org/EIPS/eip-8295). The `inject-periods` command described earlier uses these SQL queries to inject the period data into the state. - -### Inactive Subtree Identification - -The inactive-subtree identification routine finds **maximal inactive subtrees** in the Ethereum state trie according to the inactivity criteria described in [EIP-8188](https://eips.ethereum.org/EIPS/eip-8188). A subtree is considered **inactive** if all accounts (or storage slots, for a storage trie) in the subtree have an age greater than or equal to a user-defined threshold (`inactive_min_age`). The age of an object is determined by the `last_written_period` stored in the [EIP-8295](https://eips.ethereum.org/EIPS/eip-8295) snapshot metadata. - -The identification algorithm makes one sequential depth-first-search (DFS) traversal of the account trie, while moving a snapshot iterator over the corresponding metadata. Because both iterators enumerate leaves in the same deterministic order, each account or storage slot is processed exactly once, with no need for random lookups. This makes the algorithm amenable to execution on mainnet-sized state and integration into clients such as Reth. - -During traversal, a stack of internal-node frames is maintained. Each frame aggregates the inactivity status of its children in a bottom-up manner. Child subtrees that are completely inactive are temporarily stored as candidates. When an internal node has been fully processed, the algorithm determines whether all of its descendants are inactive. If so, the node itself becomes the representative inactive subtree and its candidate children are discarded. Otherwise, the node is mixed, and all previously collected inactive child candidates are emitted as maximal inactive subtree roots. This guarantees that the output contains the largest possible inactive subtrees without redundancy. - -Before the traversal begins, the snapshot is synchronized to the latest committed state by flushing any in-memory or layered state to the persistent snapshot view. This makes sure that any recently modified accounts and storage slots are available to both the snapshot iterator and the trie traversal, thus preventing the stale metadata from having any kind of effect on subtree identification. - -**Pseudocode** - -```rust -fn identify_inactive_subtrees(trie: Trie, snapshot: Snapshot, min_age: u64) { - let mut iterator = trie.node_iterator(); - let mut snapshot_iter = snapshot.leaf_iterator(); - let mut stack = Vec::new(); - - while let Some(node) = iterator.next() { - match node { - Node::Internal => { - stack.push(Frame::new(node.hash())); - } - - Node::Leaf(leaf) => { - let metadata = snapshot_iter.next().unwrap(); - - assert_eq!(leaf.key(), metadata.key()); - - let inactive = - current_period - metadata.last_written_period >= min_age; - - if let Some(parent) = stack.last_mut() { - parent.record_leaf(inactive); - } - } - - Node::EndInternal => { - let frame = stack.pop().unwrap(); - - if frame.all_children_inactive() { - if !frame.is_embedded() { - if let Some(parent) = stack.last_mut() { - parent.add_candidate(frame.root_hash()); - } else { - emit(frame.root_hash()); - } - } - } else { - for subtree in frame.candidates() { - emit(subtree); - } - - if let Some(parent) = stack.last_mut() { - parent.record_child_active(); - } - } - } - } - } - - for contract in trie.contract_accounts() { - if contract.storage_root() != EMPTY_ROOT { - identify_inactive_subtrees( - contract.storage_trie(), - snapshot.storage_snapshot(contract.address()), - min_age, - ); - } - } -} -``` - -After the inactive subtree has been identified, it gets stored in the cold state and a reference for it is stored in the hot state. - -### Cold State Format -When a maximal inactive subtree is identified and moved to the cold file, it is replaced in the hot database by a compact 17-byte "primary stub". This stub acts as a filesystem pointer, allowing the client to resolve the data only when needed. - -The Primary Stub has the following structure: - -``` -primary stub -─────────────────────────── -marker + blobOffset + rootInBlob + rootSize -``` - -* Marker (1 Byte): 0x00 -> marker of the database entry as a cold state stub. - -* Blob Offset (8 Bytes): Absolute byte position in the append-only file of the beginning of the subtree blob. - -* Root in Blob (4 Bytes): Relative offset in this blob entry of the root node of the subtree. - -* Root Size (4 Bytes): Byte Size of the root entry. - -The actual Cold State File is comprised of concatenated **blobs**, which contain the following structure: - -``` -blob -─────────────────────────── -- header - versionNumber | reservedBytes | rootOffset | rootSize - -- body - Contains the serialized trie in post-order DFS format -``` - -**Header (16 bytes)**: The header contains some basic metadata about the blob. It includes the version number which indicates the version number of the blob, some reserved bytes, offset of the root node and size of the root. The offset allows the reader of the file to find the root of the serialized trie. - -**Body**: The body of the blob contains the actual trie in post-order DFS format, meaning the children of the trie are written before the parent. Because of this post-order format, the root node becomes the last node of the blob. - -**Trie Node Structure** - -Each Trie node can be one of two types: - -* **Full Node**: This node represents a branch node in the Merkle Patricia Trie. It contains 17 child slots, one for each nibble (`0-15`) and one value slot. - -* **Short Node**: This node represents a leaf node. It contains the key part followed by one child slot. - -**Child Slot Structure** - -Each child slot contains a one-byte **kind** field that describes how the child is encoded. The possible kinds of the child slots include: - -* **Empty**: means that there is no child. -* **Hashed Reference**: contains the child's Keccak-256 hash, as well as its offset and size within the blob. -* **Embedded Reference**: only includes the offset and size of the child, as the embedded nodes do not have any independent hashes. -* **Inline Value**: stores the serialized value directly within the node (typically the account or storage value associated with a leaf). - -This layout preserves the complete Merkle Patricia Trie structure while allowing inactive subtrees to be stored as self-contained blobs that can later be reconstructed or materialized without requiring access to the active state database. - -## Roadmap - -| **Phase** | **Weeks** | **Deliverables** | -| --------------------------------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **1: Data Collection & Period Generation** | **6** | Implementation of the data collection pipeline with Xatu and ClickHouse and calculation of EIP-8188 and EIP-8295 metadata. -| **2: Period Injection (`reth db inject-periods`)** | **7-8** | Extension of the state schema with `last_written_period` and implementation of the `reth db inject-periods` command. | -| **3: Inactive Subtree Identification** | **9-11** | Implementation of the DFS algorithm for detecting inactive subtrees and detection of maximal inactive subtrees. | -| **4: Cold State Conversion (`reth db convert-inactive`)** | **12-14** | Generation of the blob format and primary stubs, serialization of inactive subtrees into the cold state file and replacement of inactive subtrees in the hot state with stubs | -| **5: Testing, Benchmarking & Optimization** | **15-18** | Test functionality, benchmark hot & cold states | -| **(Stretch)** | **18+** | Extend for Ethrex as well if there is any extra time left. | - - -## Possible challenges - -* **Uncertainty about Parameters**: [EIP-8188](https://eips.ethereum.org/EIPS/eip-8188) and [EIP-8295](https://eips.ethereum.org/EIPS/eip-8295) are very new EIPs and plenty of constant parameters like `blocks_per_period` and `inactive_min_age` are still TBD. Without those parameters, a lot of sample experimentation is necessary to figure out which are the optimal values for those parameters. - -* **Performance at Mainnet Scale**: The inactive-subtree DFS traverses the full account trie, plus every contract's storage trie, in lockstep with a snapshot iterator. Mainnet state has hundreds of millions of accounts and billions of storage slots, so keeping the traversal, iterator synchronization, and internal-node frame stack performant enough to run in reasonable time (and without excessive memory use) is a real risk that will likely require profiling and iteration. - -* **Crash Safety of Hot/Cold Conversion**: `reth db convert-inactive` needs to move a subtree out of the hot database into the append-only cold file and replace it with a stub, ideally atomically. If the process is interrupted mid-conversion, the hot and cold states could end up inconsistent (e.g. a stub pointing at a partially-written blob, or a subtree present in both places at once). Designing a safe commit protocol that tolerates crashes/restarts without corrupting state will be non-trivial. - -## Goal of the Project - -Success looks like: - -* A complete implementation of the state tiering primitives described in **EIP-8188** and **EIP-8295** within Reth. -* A working `reth db inject-periods` command that computes and injects `last_written_period` metadata for all accounts and storage slots using data collected from Xatu. -* A working `reth db convert-inactive` command that correctly identifies maximal inactive subtrees, stores them in a proper format in the cold state file, and replaces them in the hot state with primary stubs that reference them in the cold state. -* A cold state format that properly preserves the original Merkle Patricia Trie structure and allows inactive subtrees to be accessed and materialized on demand. - -## Collaborators - -### Fellows - -[Astrion](https://github.com/astrion-coder) - -### Mentors - -[Ng Wei Han](https://github.com/weiihann) - -## Resources - -* [Not All State is Equal](https://ethereum-magicians.org/t/not-all-state-is-equal/25508) -* [EIP-8188: Last-Written Block for Accounts and Slots](https://eips.ethereum.org/EIPS/eip-8188) -* [Geth implementation of State Tiering](https://github.com/weiihann/go-ethereum/tree/feat/eip-8188/inject) -* [Xatu](https://github.com/ethpandaops/xatu) -* [EIP-8188 ACD Slides](https://weiihann.github.io/20260521-eip8188-acd/) \ No newline at end of file