Abstract OpenRaft Core Data Structures into Traits #1278

drmingdrmer · 2024-12-18T02:48:44Z

Abstract OpenRaft Core Data Structures into Traits

Goal

Abstract OpenRaft's foundational data structures into traits to provide greater flexibility, allowing users to customize implementations according to their needs.
To enable users to fully define Entry types using Protobuf, we need to remove Entry's hard dependencies on other basic types (such as LogId, Membership, etc.). This means these basic types also need to be abstracted into traits to achieve complete type customization.

Add protocol buffers support for messages and gRPC services #1271

Types to Abstract

Basic Types:

Term
~~LogId~~ LogId will not be abstracted in this version for simplicity. It is primarily used internally and is not a critical component for applications, so it can remain as is.
LeaderId and its associated type CommittedLeaderId

Composite Types:

Entry
~~Membership~~ Membership will not be abstracted as it is not on the hot path and can be reconstructed as needed.
Vote

Implementation Plan

Start with abstracting the most fundamental Term type into a trait
Gradually complete abstraction of other basic types
Finally handle composite types that depend on these basic types

Expected Benefits

Provides greater flexibility, allowing users to fully customize data structures
Better support for different serialization schemes (e.g., protobuf)
Maintains compatibility with existing code

The text was updated successfully, but these errors were encountered:

github-actions · 2024-12-18T02:48:55Z

👋 Thanks for opening this issue!

Get help or engage by:

/help : to print help messages.
/assignme : to assign this issue to you.

This refactoring moves Vote from a per-NodeId type to a per-TypeConfig type, to make it consistent with `RaftTypeConfig` usage across the codebase. - Part of: databendlabs#1278 Upgrade tip: Vote is now parameterized by `RaftTypeConfig` instead of `NodeId` - Change `Vote<NodeId>` to `Vote<C> where C: RaftTypeConfig`, for example, change `Vote<u64>` to `Vote<YourTypeConfig>`.

This refactoring moves LogId from a per-NodeId type to a per-TypeConfig type, to make it consistent with `RaftTypeConfig` usage across the codebase. - Part of: databendlabs#1278 Upgrade tip: LogId is now parameterized by `RaftTypeConfig` instead of `NodeId` - Change `LogId<NodeId>` to `LogId<C> where C: RaftTypeConfig`, for example, change `LogId<u64>` to `LogId<YourTypeConfig>`. - Change trait `RaftLogId<NID: NodeId>` to `RaftLogId<C: RaftTypeConfig>`.

This refactoring moves LeaderId from a per-NodeId type to a per-TypeConfig type, to make it consistent with `RaftTypeConfig` usage across the codebase. - Part of: databendlabs#1278 Upgrade tip: LeaderId is now parameterized by `RaftTypeConfig` instead of `NodeId` - Change `LeaderId<NodeId>` to `LeaderId<C> where C: RaftTypeConfig`, for example, change `LeaderId<u64>` to `LeaderId<YourTypeConfig>`. - Change `CommittedLeaderId<NodeId>` to `CommittedLeaderId<RaftTypeConfig>`.

This refactoring moves Vote from a per-NodeId type to a per-TypeConfig type, to make it consistent with `RaftTypeConfig` usage across the codebase. - Part of: #1278 Upgrade tip: Vote is now parameterized by `RaftTypeConfig` instead of `NodeId` - Change `Vote<NodeId>` to `Vote<C> where C: RaftTypeConfig`, for example, change `Vote<u64>` to `Vote<YourTypeConfig>`.

This refactoring moves LeaderId from a per-NodeId type to a per-TypeConfig type, to make it consistent with `RaftTypeConfig` usage across the codebase. - Part of: databendlabs#1278 Upgrade tip: LeaderId is now parameterized by `RaftTypeConfig` instead of `NodeId` - Change `LeaderId<NodeId>` to `LeaderId<C> where C: RaftTypeConfig`, for example, change `LeaderId<u64>` to `LeaderId<YourTypeConfig>`. - Change `CommittedLeaderId<NodeId>` to `CommittedLeaderId<RaftTypeConfig>`.

This refactoring moves LogId from a per-NodeId type to a per-TypeConfig type, to make it consistent with `RaftTypeConfig` usage across the codebase. - Part of: #1278 Upgrade tip: LogId is now parameterized by `RaftTypeConfig` instead of `NodeId` - Change `LogId<NodeId>` to `LogId<C> where C: RaftTypeConfig`, for example, change `LogId<u64>` to `LogId<YourTypeConfig>`. - Change trait `RaftLogId<NID: NodeId>` to `RaftLogId<C: RaftTypeConfig>`.

This refactoring moves LeaderId from a per-NodeId type to a per-TypeConfig type, to make it consistent with `RaftTypeConfig` usage across the codebase. - Part of: databendlabs#1278 Upgrade tip: LeaderId is now parameterized by `RaftTypeConfig` instead of `NodeId` - Change `LeaderId<NodeId>` to `LeaderId<C> where C: RaftTypeConfig`, for example, change `LeaderId<u64>` to `LeaderId<YourTypeConfig>`. - Change `CommittedLeaderId<NodeId>` to `CommittedLeaderId<RaftTypeConfig>`.

This refactoring moves LeaderId from a per-NodeId type to a per-TypeConfig type, to make it consistent with `RaftTypeConfig` usage across the codebase. - Part of: #1278 Upgrade tip: LeaderId is now parameterized by `RaftTypeConfig` instead of `NodeId` - Change `LeaderId<NodeId>` to `LeaderId<C> where C: RaftTypeConfig`, for example, change `LeaderId<u64>` to `LeaderId<YourTypeConfig>`. - Change `CommittedLeaderId<NodeId>` to `CommittedLeaderId<RaftTypeConfig>`.

Add associated type `Term: RaftTerm` to `RaftTypeConfig` so that application can customize the data type for Raft `term`. By default `Term` is `u64` and user application does not need to modify to upgrade. - Part of databendlabs#1278

Add associated type `Term: RaftTerm` to `RaftTypeConfig` so that application can customize the data type for Raft `term`. By default `Term` is `u64` and user application does not need to modify to upgrade. - Part of #1278

Add `LeaderId: RaftLeaderId` to `RaftTypeConfig` to allow customizing leader ID implementations. This maintains backward compatibility: - Default `LeaderId`: Advanced mode, allowing multiple leaders per term; - With `single-term-leader` feature flag enabled: Standard Raft mode, one leader per term. No code changes required for existing applications to upgrade. - Part of databendlabs#1278

Add `LeaderId: RaftLeaderId` to `RaftTypeConfig` to allow customizing leader ID implementations. - Part of databendlabs#1278

impl CompareByKey for RaftVote implementations RaftTypeConfig: add associated type Vote impl RaftVote for Vote; add C to CompareByKey - Part of databendlabs#1278

- The `RaftEntry` trait now requires `AsRef<LogIdOf<C>>` and `AsMut<LogIdOf<C>>`, providing a more standard API for accessing the log ID of a log entry. As a result, the `RaftEntry: RaftLogId` requirement is no longer needed. - A new method, `new_normal()`, has been added to the `RaftEntry` trait to replace the `FromAppData` trait. - Additional utility methods for working with entries are now provided in the `RaftEntryExt` trait. - Part of databendlabs#1278 Upgrade tips: 1. **For applications using a custom `RaftEntry` implementation** (e.g., declared with `declare_raft_types!(MyTypes: Entry = MyEntry)`): - Update the `RaftEntry` implementation for your custom entry type (`MyEntry`) by adding the `new_normal()` method. - Implement `AsRef<LogId>` and `AsMut<LogId>` for your custom entry type. 2. **For applications using the default `Entry` provided by OpenRaft**: - No changes are required.

Existing `LogIdOf<C>` provides a minimal storage implementation for a log ID with essential properties. In contrast, `RefLogId` offers the same information as `LogIdOf<C>` while adding additional system-defined properties. For example, in the future, `LogIdOf<C>` defined by the application will not need to implement `Ord`. However, `RefLogId`, used internally, will provide a system-defined `Ord` implementation. This change updates internal components to return `RefLogId` or accept it as an argument where possible, enabling more flexibility and consistency in handling log IDs. Change: refine the `RaftEntry` trait - The `RaftEntry` trait now requires `AsRef<LogIdOf<C>>` and `AsMut<LogIdOf<C>>`, providing a more standard API for accessing the log ID of a log entry. As a result, the `RaftEntry: RaftLogId` requirement is no longer needed. - A new method, `new_normal()`, has been added to the `RaftEntry` trait to replace the `FromAppData` trait. - Additional utility methods for working with entries are now provided in the `RaftEntryExt` trait. - Part of databendlabs#1278 Upgrade tips: 1. **For applications using a custom `RaftEntry` implementation** (e.g., declared with `declare_raft_types!(MyTypes: Entry = MyEntry)`): - Update the `RaftEntry` implementation for your custom entry type (`MyEntry`) by adding the `new_normal()` method. - Implement `AsRef<LogId>` and `AsMut<LogId>` for your custom entry type. 2. **For applications using the default `Entry` provided by OpenRaft**: - No changes are required. empty: try to remove log id from RaftTypeConfig

- Remove trait `FromAppData` that is used to create log `Entry` from application data. Add `RaftEntry::new()`. Application should implement this method. - Change: trait `RaftLogId`: before this commit, it is used to defined a `LogId` container, requiring methods `get_log_id()` and `set_log_id()`. In this commit, `RaftLogId` defines the behavior of `LogId` itself, requiring methods `new()`, `committed_leader_id()` and `index()`. - The `RaftEntry` trait removed requirement of `RaftLogId`, due to the `RaftLogId` behavior change, and requires application to implements methods: - `new()` to create a log `Entry`, - `log_id_parts()`: to return references to the components of this entry's log ID: the committed leader ID and index, - `set_log_id()`: to update the log entry id. And `RaftEntry` provides several default implemented methods: - `new_blank()`, `new_normal()`, `new_membership()` to create different kind of log `Entry`. - `log_id()` returns an owned `LogId` instance, - `index()` returns the log id index. - Refactor: Introduce `RefLogId` as a reference to a log ID Existing `LogIdOf<C>` provides a minimal storage implementation for a log ID with essential properties. In contrast, `RefLogId` which references a storage `LogIdOf<C>`, offers the same information as `LogIdOf<C>` while adding additional system-defined properties. For example, `LogIdOf<C>` defined by the application will not need to implement `Ord`. However, `RefLogId`, used internally, will provide a system-defined `Ord` implementation. This change updates internal components to return `RefLogId` or accept it as an argument where possible, enabling more flexibility and consistency in handling log IDs. - Update example `raft-kv-memstore-grpc`: implement log `Entry` and other types with protobuf, such as state machine and RPC message types; Implement snapshot streaming transmission; Remove `serde` from this example. - Part of databendlabs#1278 Upgrade tips: 1. **For applications using a custom `RaftEntry` implementation** (e.g., declared with `declare_raft_types!(MyTypes: Entry = MyEntry)`), Update the `RaftEntry` implementation: - Remove implementation of `FromAppData`; - Add implementation of `new()`, `log_id_parts()` and `set_log_id()` method. 2. **For applications using the default `Entry` provided by OpenRaft**: - No changes are required.

This commit refines the `RaftEntry` and related traits, to better support application defined log `Entry` type. Key Changes: 1. Remove `FromAppData` trait: - The `FromAppData` trait, which was used to create log `Entry` from application data, is removed. - Applications should now implement the new `RaftEntry::new()` method to create log entries directly. 2. Remove `RaftLogId` trait, it becomes an internal trait. 3. Update `RaftEntry` trait: - The `RaftEntry` trait no longer requires `RaftLogId` (due to its redefinition) and now mandates the implementation of: - `new()`: For creating a log `Entry`. - `log_id_parts()`: To return references to the log ID's committed leader ID(term) and index. - `set_log_id()`: To update the log entry's ID. - Default methods are provided: - `new_blank()`, `new_normal()`, `new_membership()` for creating different types of log entries. - `log_id()` returns an owned `LogId` instance. - `index()` returns the index of the log entry. 4. Introduce `RefLogId`: - `RefLogId` is a reference-based representation of a log ID, complementing the existing `LogIdOf<C>` (a storage-based implementation). - `RefLogId` adds system-defined properties (e.g., `Ord` implementation) while referencing an existing `LogIdOf<C>`. - Internal components now use `RefLogId` where possible, improving flexibility and consistency. 5. Update example `raft-kv-memstore-grpc`: - Updated to implement log `Entry` and related types using protobuf, including state machine and RPC message types. - Added snapshot streaming transmission implementation. - Removed `serde` dependency from the example. - Part of databendlabs#1278. --- Upgrade tips: 1. For Applications with Custom `RaftEntry` Implementations: If you've declared a custom `RaftEntry` (e.g., `declare_raft_types!(MyTypes: Entry = MyEntry)`): - Remove the implementation of `FromAppData`. - Implement the following new methods: - `new()` - `log_id_parts()` - `set_log_id()` 2. For Applications Using OpenRaft's Default `Entry`: - No changes are required.

drmingdrmer · 2025-02-03T01:02:08Z

Abstraction is done except for LogId and Membership:

~~LogId~~ LogId will not be abstracted in this version for simplicity. It is primarily used internally and is not a critical component for applications, so it can remain as is.
~~Membership~~ Membership will not be abstracted as it is not on the hot path and can be reconstructed as needed.

These already provided abstraction, including Term, LeaderId, CommittedLeaderId, Vote and Entry are enough to address #1271

Closing this issue.

This commit refines the `RaftEntry` and related traits, to better support application defined log `Entry` type. Key Changes: 1. Remove `FromAppData` trait: - The `FromAppData` trait, which was used to create log `Entry` from application data, is removed. - Applications should now implement the new `RaftEntry::new()` method to create log entries directly. 2. Remove `RaftLogId` trait, it becomes an internal trait. 3. Update `RaftEntry` trait: - The `RaftEntry` trait no longer requires `RaftLogId` (due to its redefinition) and now mandates the implementation of: - `new()`: For creating a log `Entry`. - `log_id_parts()`: To return references to the log ID's committed leader ID(term) and index. - `set_log_id()`: To update the log entry's ID. - Default methods are provided: - `new_blank()`, `new_normal()`, `new_membership()` for creating different types of log entries. - `log_id()` returns an owned `LogId` instance. - `index()` returns the index of the log entry. 4. Introduce `RefLogId`: - `RefLogId` is a reference-based representation of a log ID, complementing the existing `LogIdOf<C>` (a storage-based implementation). - `RefLogId` adds system-defined properties (e.g., `Ord` implementation) while referencing an existing `LogIdOf<C>`. - Internal components now use `RefLogId` where possible, improving flexibility and consistency. 5. Update example `raft-kv-memstore-grpc`: - Updated to implement log `Entry` and related types using protobuf, including state machine and RPC message types. - Added snapshot streaming transmission implementation. - Removed `serde` dependency from the example. - Part of #1278. --- Upgrade tips: 1. For Applications with Custom `RaftEntry` Implementations: If you've declared a custom `RaftEntry` (e.g., `declare_raft_types!(MyTypes: Entry = MyEntry)`): - Remove the implementation of `FromAppData`. - Implement the following new methods: - `new()` - `log_id_parts()` - `set_log_id()` 2. For Applications Using OpenRaft's Default `Entry`: - No changes are required.

drmingdrmer mentioned this issue Dec 24, 2024

Change: change Vote<NID:NodeId> to Vote<C:RaftTypeConfig> #1279

Merged

drmingdrmer mentioned this issue Dec 25, 2024

Change: change LogId<NID:NodeId> to LogId<C:RaftTypeConfig> #1281

Merged

drmingdrmer mentioned this issue Dec 25, 2024

Change: change LeaderId<NID:NodeId> to LeaderId<C:RaftTypeConfig> #1282

Merged

drmingdrmer mentioned this issue Dec 26, 2024

Feature: Abstract Term #1284

Merged

drmingdrmer mentioned this issue Dec 30, 2024

Example of Grpc and Rocksdb Please #1287

Closed

drmingdrmer added a commit to drmingdrmer/openraft that referenced this issue Dec 30, 2024

Feature: Abstract LeaderId and CommittedLeaderId

b230bc2

Add `LeaderId: RaftLeaderId` to `RaftTypeConfig` to allow customizing leader ID implementations. - Part of databendlabs#1278

drmingdrmer mentioned this issue Dec 30, 2024

Change: Remove feature flag single-term-leader #1289

Merged

drmingdrmer added a commit to drmingdrmer/openraft that referenced this issue Jan 2, 2025

Feature: Abstract Vote

d05e107

impl CompareByKey for RaftVote implementations RaftTypeConfig: add associated type Vote impl RaftVote for Vote; add C to CompareByKey - Part of databendlabs#1278

drmingdrmer mentioned this issue Feb 2, 2025

Change: Refine Log Entry Traits #1320

Merged

drmingdrmer self-assigned this Feb 3, 2025

drmingdrmer closed this as completed Feb 3, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Abstract OpenRaft Core Data Structures into Traits #1278

Abstract OpenRaft Core Data Structures into Traits #1278

drmingdrmer commented Dec 18, 2024 •

edited

Loading

github-actions bot commented Dec 18, 2024

drmingdrmer commented Feb 3, 2025

Abstract OpenRaft Core Data Structures into Traits #1278

Abstract OpenRaft Core Data Structures into Traits #1278

Comments

drmingdrmer commented Dec 18, 2024 • edited Loading

Abstract OpenRaft Core Data Structures into Traits

Goal

Types to Abstract

Basic Types:

Composite Types:

Implementation Plan

Expected Benefits

github-actions bot commented Dec 18, 2024

drmingdrmer commented Feb 3, 2025

drmingdrmer commented Dec 18, 2024 •

edited

Loading