From 4fffb1bfae03c96027fdac08264182fb14731252 Mon Sep 17 00:00:00 2001 From: unnawut Date: Wed, 13 Aug 2025 18:32:43 +0700 Subject: [PATCH 1/3] docs: add networking specs draft --- src/lean_spec/client/networking.md | 174 +++++++++++++++++++++++++++++ 1 file changed, 174 insertions(+) create mode 100644 src/lean_spec/client/networking.md diff --git a/src/lean_spec/client/networking.md b/src/lean_spec/client/networking.md new file mode 100644 index 00000000..24aab962 --- /dev/null +++ b/src/lean_spec/client/networking.md @@ -0,0 +1,174 @@ +# Networking + +## Setup + +- Transport: QUIC on IPv4 +- Encryption and identification: [Libp2p-noise](https://github.com/libp2p/specs/tree/master/noise) with `secp256k1` identities +- Protocol negotiation: [multistream-select 1.0](https://github.com/multiformats/multistream-select/) +- Multiplexing: Native support by QUIC +- Gossip: [gossipsub v1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md) + +## Gossip domain + +**Protocol ID:** `/meshsub/1.1.0` + +**Gossipsub Parameters** + +The following gossipsub +[parameters](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md#parameters) +will be used: + +- `D` (topic stable mesh target count): 8 +- `D_low` (topic stable mesh low watermark): 6 +- `D_high` (topic stable mesh high watermark): 12 +- `D_lazy` (gossip target): 6 +- `heartbeat_interval` (frequency of heartbeat, seconds): 0.7 +- `fanout_ttl` (ttl for fanout maps for topics we are not subscribed to but have + published to, seconds): 60 +- `mcache_len` (number of windows to retain full messages in cache for `IWANT` + responses): 6 +- `mcache_gossip` (number of windows to gossip about): 3 +- `seen_ttl` (expiry time for cache of seen message ids, seconds): + SECONDS_PER_SLOT * SLOTS_PER_EPOCH * 2 + +#### Topics and messages + +Topics are plain UTF-8 strings and are encoded on the wire as determined by +protobuf (gossipsub messages are enveloped in protobuf messages). Topic strings +have form: `/leanconsensus/devnet0/Name/Encoding`. This defines both the type of +data being sent on the topic and how the data field of the message is encoded. + +- `Name` - see table below +- `Encoding` - the encoding strategy describes a specific representation of + bytes that will be transmitted over the wire. See the [Encodings](#Encodings) + section for further details. + +The optional `from` (1), `seqno` (3), `signature` (5) and `key` (6) protobuf +fields are omitted from the message, since messages are identified by content, +anonymous, and signed where necessary in the application layer. + +The `message-id` of a gossipsub message MUST be the following 20 byte value +computed from the message data: + +- If `message.data` has a valid snappy decompression, set `message-id` to the + first 20 bytes of the `SHA256` hash of the concatenation of + `MESSAGE_DOMAIN_VALID_SNAPPY` with the snappy decompressed message data, i.e. + `SHA256(MESSAGE_DOMAIN_VALID_SNAPPY + snappy_decompress(message.data))[:20]`. +- Otherwise, set `message-id` to the first 20 bytes of the `SHA256` hash of the + concatenation of `MESSAGE_DOMAIN_INVALID_SNAPPY` with the raw message data, + i.e. `SHA256(MESSAGE_DOMAIN_INVALID_SNAPPY + message.data)[:20]`. + +Where relevant, clients MUST reject messages with `message-id` sizes other than +20 bytes. + +*Note*: The above logic handles two exceptional cases: (1) multiple snappy +`data` can decompress to the same value, and (2) some message `data` can fail to +snappy decompress altogether. + +The payload is carried in the `data` field of a gossipsub message, and varies +depending on the topic: + +| Name | Message Type | +| -------------------------------- | ------------------------- | +| `lean_block` | `Block` | +| `lean_attestation` | `Attestation` | + +Clients MUST reject (fail validation) messages containing an incorrect type, or +invalid payload. + +#### Encodings + +Topics are post-fixed with an encoding. Encodings define how the payload of a +gossipsub message is encoded. + +- `ssz_snappy` - All objects are SSZ-encoded and then compressed with + [Snappy](https://github.com/google/snappy) block compression. Example: The + beacon aggregate attestation topic string is + `/eth2/446a7232/beacon_aggregate_and_proof/ssz_snappy`, the fork digest is + `446a7232` and the data field of a gossipsub message is an `AggregateAndProof` + that has been SSZ-encoded and then compressed with Snappy. + +Snappy has two formats: "block" and "frames" (streaming). Gossip messages remain +relatively small (100s of bytes to 100s of kilobytes) so +[basic snappy block compression](https://github.com/google/snappy/blob/master/format_description.txt) +is used to avoid the additional overhead associated with snappy frames. + +Implementations MUST use a single encoding for gossip. Changing an encoding will +require coordination between participating implementations. + +### The Req/Resp domain + +#### Protocol identification + +Each message type is segregated into its own libp2p protocol ID, which is a +case-sensitive UTF-8 string of the form: + +``` +/leanconsensus/req/MessageName/SchemaVersion/Encoding +``` + +With: + +- `MessageName` - each request is identified by a name consisting of English + alphabet, digits and underscores (`_`). +- `SchemaVersion` - an ordinal version number (e.g. 1, 2, 3…). Each schema is + versioned to facilitate backward and forward-compatibility when possible. +- `Encoding` - while the schema defines the data types in more abstract terms, + the encoding strategy describes a specific representation of bytes that will + be transmitted over the wire. See the [Encodings](#Encoding-strategies) + section for further details. + +This protocol segregation allows libp2p `multistream-select 1.0` to handle the +request type, version, and encoding negotiation before establishing the +underlying streams. + +#### Encoding strategies + +The token of the negotiated protocol ID specifies the type of encoding to be +used for the req/resp interaction. Only one value is possible at this time: + +- `ssz_snappy`: The contents are first + [SSZ-encoded](../../ssz/simple-serialize.md) and then compressed with + [Snappy](https://github.com/google/snappy) frames compression. For objects + containing a single field, only the field is SSZ-encoded not a container with + a single field. For example, the `LeanBlocksByRoot` request is an + SSZ-encoded list of `Root`'s. This encoding type MUST be supported by all + clients. + +#### Messages + +##### LeanBlocksByRoot v1 + +**Protocol ID:** `/leanconsensus/req/lean_blocks_by_root/1/` + +Request Content: + +``` +( + List[Root, MAX_REQUEST_BLOCKS] +) +``` + +Response Content: + +``` +( + List[SignedLeanBlock, MAX_REQUEST_BLOCKS] +) +``` + +Requests blocks by block root (= `hash_tree_root(SignedLeanBlock.message)`). +The response is a list of `SignedLeanBlock` whose length is less than or equal +to the number of requested blocks. It may be less in the case that the +responding peer is missing blocks. + +`LeanBlocksByRoot` is primarily used to recover recent blocks (e.g. when +receiving a block or attestation whose parent is unknown). + +The request MUST be encoded as an SSZ-field. + +The response MUST consist of zero or more `response_chunk`. Each _successful_ +`response_chunk` MUST contain a single `SignedLeanBlock` payload. + +Clients MUST respond with at least one block, if they have it. Clients MAY limit +the number of blocks in the response. From 9f3175dcc1059c06155ef9d0bb22e4ac6f5e4952 Mon Sep 17 00:00:00 2001 From: unnawut Date: Wed, 13 Aug 2025 20:55:23 +0700 Subject: [PATCH 2/3] docs: add validator ID and proposer selection --- src/lean_spec/client/validator.md | 33 +++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 src/lean_spec/client/validator.md diff --git a/src/lean_spec/client/validator.md b/src/lean_spec/client/validator.md new file mode 100644 index 00000000..312d089b --- /dev/null +++ b/src/lean_spec/client/validator.md @@ -0,0 +1,33 @@ +# Validator + +## Validator identification + +Validators are defined as a list of their ENRs in a yaml file named `validators.yaml`. + +```yaml +- enr:-Ku4QHqVeJ8PPICcWk1vSn_XcSkjOkNiTg6Fmii5j6vUQgvzMc9L1goFnLKgXqBJspJjIsB91LTOleFmyWWrFVATGngBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpC1MD8qAAAAAP__________gmlkgnY0gmlwhAMRHkWJc2VjcDI1NmsxoQKLVXFOhp2uX6jeT0DvvDpPcU8FWMjQdR4wMuORMhpX24N1ZHCCIyg +- enr:-Ku4QPn5eVhcoF1opaFEvg1b6JNFD2rqVkHQ8HApOKK61OIcIXD127bKWgAtbwI7pnxx6cDyk_nI88TrZKQaGMZj0q0Bh2F0dG5ldHOIAAAAAAAAAACEZXRoMpC1MD8qAAAAAP__________gmlkgnY0gmlwhDayLMaJc2VjcDI1NmsxoQK2sBOLGcUb4AwuYzFuAVCaNHA-dy24UuEKkeFNgCVCsIN1ZHCCIyg +- enr:-Ku4QG-2_Md3sZIAUebGYT6g0SMskIml77l6yR-M_JXc-UdNHCmHQeOiMLbylPejyJsdAPsTHJyjJB2sYGDLe0dn8uYBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpC1MD8qAAAAAP__________gmlkgnY0gmlwhBLY-NyJc2VjcDI1NmsxoQORcM6e19T1T9gi7jxEZjk_sjVLGFscUNqAY9obgZaxbIN1ZHCCIyg +``` + +The validator ID is the zero-index of the list in the yaml file. For example, +`3.17.30.69`, `54.178.44.198`, `18.216.248.220` are `validator_id: 0`, +`validator_id: 1`, `validator_id: 2` respectively. + +Because pqsignature has not been implemented yet, the `secp256k1` field will +still be the 33 bytes compressed secp256k1 public key for the time being +and shall be replaced in subsequent devnet iterations. + +## Block proposer selection + +The block proposer shall be determined by the modulo of current slot number by +the total number of validators, such that block proposers are determined in +a round-robin manner of the validator IDs. + +```py +def is_proposer(state: BeaconState, validator_index: ValidatorIndex) -> bool: + return get_current_slot() % state.config.num_validators == validator_index +``` + +To ensure block proposal duties are distributed equally between 2 participating +clients, the validator IDs may be assigned to 2 clients in odd/even manner. From 68735303dfcf635c88ab28bb39bc79e595624180 Mon Sep 17 00:00:00 2001 From: unnawut Date: Wed, 13 Aug 2025 20:59:30 +0700 Subject: [PATCH 3/3] fix: remove obvious wordings --- src/lean_spec/client/networking.md | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/src/lean_spec/client/networking.md b/src/lean_spec/client/networking.md index 24aab962..ff657f30 100644 --- a/src/lean_spec/client/networking.md +++ b/src/lean_spec/client/networking.md @@ -83,18 +83,9 @@ gossipsub message is encoded. - `ssz_snappy` - All objects are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy) block compression. Example: The - beacon aggregate attestation topic string is - `/eth2/446a7232/beacon_aggregate_and_proof/ssz_snappy`, the fork digest is - `446a7232` and the data field of a gossipsub message is an `AggregateAndProof` - that has been SSZ-encoded and then compressed with Snappy. - -Snappy has two formats: "block" and "frames" (streaming). Gossip messages remain -relatively small (100s of bytes to 100s of kilobytes) so -[basic snappy block compression](https://github.com/google/snappy/blob/master/format_description.txt) -is used to avoid the additional overhead associated with snappy frames. - -Implementations MUST use a single encoding for gossip. Changing an encoding will -require coordination between participating implementations. + lean block topic string is `/leanconsensus/devnet0/lean_block/ssz_snappy`, + and the data field of a gossipsub message is an `Attestation` that has been + SSZ-encoded and then compressed with Snappy. ### The Req/Resp domain