Skip to content

Commit 5ecad22

Browse files
github-actions[bot]ti-chi-bot
authored andcommitted
Auto-sync: Update English docs from Chinese PR pingcap/docs-cn#20703
Synced from: pingcap/docs-cn#20703 Target PR: pingcap#21962 AI Provider: gemini Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
1 parent e4e8bc4 commit 5ecad22

File tree

3 files changed

+64
-32
lines changed

3 files changed

+64
-32
lines changed

follower-read.md

Lines changed: 57 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -5,69 +5,96 @@ summary: This document describes the use and implementation of Follower Read.
55

66
# Follower Read
77

8-
When a read hotspot appears in a Region, the Region leader can become a read bottleneck for the entire system. In this situation, enabling the Follower Read feature can significantly reduce the load of the leader, and improve the throughput of the whole system by balancing the load among multiple followers. This document introduces the use and implementation mechanism of Follower Read.
8+
In TiDB, to ensure high availability and data security, TiKV stores multiple replicas for each Region, one of which is the leader and the others are followers. By default, all read and write requests are processed by the leader. The Follower Read feature allows reading data from the follower replicas of a Region while maintaining strong consistency, thereby distributing the read pressure on the leader and improving the overall read throughput of the cluster.
99

10-
## Overview
10+
When performing Follower Read, TiDB selects the appropriate replica based on the topology information. Specifically, TiDB uses the `zone` label to determine whether a replica is a local replica: when the `zone` label of TiDB is the same as that of the target TiKV, TiDB considers the replica as a local replica. For more information, see [Schedule replicas by topology labels](schedule-replicas-by-topology-labels.md).
1111

12-
The Follower Read feature refers to using any follower replica of a Region to serve a read request under the premise of strongly consistent reads. This feature improves the throughput of the TiDB cluster and reduces the load of the leader. It contains a series of load balancing mechanisms that offload TiKV read loads from the leader replica to the follower replica in a Region. TiKV's Follower Read implementation provides users with strongly consistent reads.
12+
By allowing followers to participate in data reading, Follower Read can achieve the following goals:
13+
14+
- Distribute read hotspots and reduce leader load.
15+
- In multi-AZ or multi-datacenter deployments, prioritize reading local replicas to reduce cross-region traffic.
16+
17+
## Applicable scenarios
18+
19+
Follower Read is suitable for the following scenarios:
20+
21+
- Businesses with large read request volume and obvious read hotspots.
22+
- Scenarios where it is desirable to prioritize reading local replicas in multi-AZ deployments to save bandwidth.
23+
- In a read-write separation architecture, it is desirable to further improve the overall read performance of the cluster.
1324

1425
> **Note:**
1526
>
16-
> To achieve strongly consistent reads, the follower node currently needs to request the current execution progress from the leader node (that is `ReadIndex`), which causes an additional network request overhead. Therefore, the main benefits of Follower Read are to isolate read requests from write requests in the cluster and to increase overall read throughput.
27+
> To ensure the strong consistency of the read results, Follower Read needs to communicate with the leader to confirm the current commit progress (that is, execute the Raft `ReadIndex` operation) before reading, which introduces an additional network interaction. Therefore, Follower Read is most effective when there are a large number of read requests or read-write isolation is required; however, the performance improvement may not be significant for low-latency single queries.
1728
1829
## Usage
1930

20-
To enable TiDB's Follower Read feature, modify the value of the `tidb_replica_read` variable as follows:
31+
To enable TiDB's Follower Read feature, set the value of the `tidb_replica_read` variable to the desired value:
2132

2233
{{< copyable "sql" >}}
2334

2435
```sql
25-
set [session | global] tidb_replica_read = '<target value>';
36+
set [session | global] tidb_replica_read = '<目标值>';
2637
```
2738

2839
Scope: SESSION | GLOBAL
2940

3041
Default: leader
3142

32-
This variable is used to set the expected data read mode.
43+
This variable is used to set the expected data read mode. From v8.5.4, this variable only takes effect on read-only SQL statements.
3344

34-
- When you set the value of `tidb_replica_read` to `leader` or an empty string, TiDB maintains its default behavior and sends all read operations to the leader replica to perform.
35-
- When you set the value of `tidb_replica_read` to `follower`, TiDB selects a follower replica of the Region to perform read operations. If the Region has learner replicas, TiDB also considers them for reads with the same priority. If no available follower or learner replicas exist for the current Region, TiDB reads from the leader replica.
36-
- When the value of `tidb_replica_read` is set to `leader-and-follower`, TiDB can select any replicas to perform read operations. In this mode, read requests are load balanced between the leader and follower.
37-
- When the value of `tidb_replica_read` is set to `prefer-leader`, TiDB prefers to select the leader replica to perform read operations. If the leader replica is obviously slow in processing read operations (such as caused by disk or network performance jitter), TiDB will select other available follower replicas to perform read operations.
38-
- When the value of `tidb_replica_read` is set to `closest-replicas`, TiDB prefers to select a replica in the same availability zone to perform read operations, which can be a leader or a follower. If there is no replica in the same availability zone, TiDB reads from the leader replica.
39-
- When the value of `tidb_replica_read` is set to `closest-adaptive`:
45+
In scenarios where you need to save cross-region traffic by reading local replicas, the following configurations are recommended:
4046

41-
- If the estimated result of a read request is greater than or equal to the value of [`tidb_adaptive_closest_read_threshold`](/system-variables.md#tidb_adaptive_closest_read_threshold-new-in-v630), TiDB prefers to select a replica in the same availability zone for read operations. To avoid unbalanced distribution of read traffic across availability zones, TiDB dynamically detects the distribution of availability zones for all online TiDB and TiKV nodes. In each availability zone, the number of TiDB nodes whose `closest-adaptive` configuration takes effect is limited, which is always the same as the number of TiDB nodes in the availability zone with the fewest TiDB nodes, and the other TiDB nodes automatically read from the leader replica. For example, if TiDB nodes are distributed across 3 availability zones (A, B, and C), where A and B each contains 3 TiDB nodes and C contains only 2 TiDB nodes, the number of TiDB nodes whose `closest-adaptive` configuration takes effect in each availability zone is 2, and the other TiDB node in each of the A and B availability zones automatically selects the leader replica for read operations.
42-
- If the estimated result of a read request is less than the value of [`tidb_adaptive_closest_read_threshold`](/system-variables.md#tidb_adaptive_closest_read_threshold-new-in-v630), TiDB can only select the leader replica for read operations.
47+
- The default value `leader` provides the best performance.
48+
- `closest-adaptive` saves traffic as much as possible with minimal performance loss.
49+
- `closest-replicas` can save network traffic to the greatest extent.
4350

44-
- When you set the value of `tidb_replica_read` to `learner`, TiDB reads data from the learner replica. If no learner replica is available for the current Region, TiDB reads from an available leader or follower replica.
51+
If you are currently using other configurations, refer to the following table to modify them to the recommended configurations:
4552

46-
<CustomContent platform="tidb">
53+
| Configuration being used | Recommended configuration to modify to |
54+
| ------------- | ------------- |
55+
| `follower` | `closest-replicas` |
56+
| `leader-and-follower` | `closest-replicas` |
57+
| `prefer-leader` | `closest-adaptive` |
58+
| `learner` | `closest-replicas` |
4759

48-
> **Note:**
49-
>
50-
> When you set `tidb_replica_read` to `closest-replicas` or `closest-adaptive`, to ensure that replicas are distributed across availability zones according to the specified configuration, you need to configure `location-labels` for PD and set the correct `labels` for TiDB and TiKV according to [Schedule replicas by topology labels](/schedule-replicas-by-topology-labels.md). TiDB depends on the `zone` label to match TiKV nodes in the same availability zone, so you need to make sure that the `zone` label is included in the `location-labels` of PD and `zone` is included in the configuration of each TiDB and TiKV node. If your cluster is deployed using TiDB Operator, refer to [High availability of data](https://docs.pingcap.com/tidb-in-kubernetes/stable/configure-a-tidb-cluster#high-availability-of-data).
51-
>
52-
> For TiDB v7.5.0 and earlier versions:
53-
>
54-
> - If you set `tidb_replica_read` to `follower` and no follower or learner replicas are available, TiDB returns an error.
55-
> - If you set `tidb_replica_read` to `learner` and no learner replicas are available, TiDB returns an error.
60+
If you want to use a more precise read replica selection policy, refer to the complete list of optional configurations:
61+
62+
## Basic monitoring
5663

57-
</CustomContent>
64+
By observing the [**TiDB** > **KV Request** > **Read Req Traffic** panel (new in v8.5.4)](/grafana-tidb-dashboard.md#kv-request), you can determine whether you need to use Follower Read and view the effect of saving traffic after enabling Follower Read.
5865

5966
## Implementation mechanism
6067

6168
Before the Follower Read feature was introduced, TiDB applied the strong leader principle and submitted all read and write requests to the leader node of a Region to handle. Although TiKV can distribute Regions evenly on multiple physical nodes, for each Region, only the leader can provide external services. The other followers can do nothing to handle read requests but receive the data replicated from the leader at all times and prepare for voting to elect a leader in case of a failover.
6269

63-
To allow data reading in the follower node without violating linearizability or affecting Snapshot Isolation in TiDB, the follower node needs to use `ReadIndex` of the Raft protocol to ensure that the read request can read the latest data that has been committed on the leader. At the TiDB level, the Follower Read feature simply needs to send the read request of a Region to a follower replica based on the load balancing policy.
70+
Follower Read includes a series of load balancing mechanisms that offload TiKV read load from Region leader replicas to follower replicas. To allow data reading in the follower node without violating linearizability or affecting Snapshot Isolation in TiDB, the follower node needs to use `ReadIndex` of the Raft protocol to ensure that the read request can read the latest data that has been committed on the leader node. At the TiDB level, the Follower Read feature simply needs to send the read request of a Region to a follower replica based on the load balancing policy.
6471

6572
### Strongly consistent reads
6673

67-
When the follower node processes a read request, it first uses `ReadIndex` of the Raft protocol to interact with the leader of the Region, to obtain the latest commit index of the current Raft group. After the latest commit index of the leader is applied locally to the follower, the processing of a read request starts.
74+
When the follower node processes a read request, it first uses `ReadIndex` of the Raft protocol to interact with the leader node of the Region to obtain the latest commit index (read index) of the current Raft group. After the latest commit index of the leader node is applied locally to the follower, the processing of a read request starts.
75+
76+
![read-index-flow](/media/follower-read/read-index.png)
6877

6978
### Follower replica selection strategy
7079

71-
Because the Follower Read feature does not affect TiDB's Snapshot Isolation transaction isolation level, TiDB adopts the round-robin strategy to select the follower replica. Currently, for the coprocessor requests, the granularity of the Follower Read load balancing policy is at the connection level. For a TiDB client connected to a specific Region, the selected follower is fixed, and is switched only when it fails or the scheduling policy is adjusted.
80+
The Follower Read feature does not affect TiDB's Snapshot Isolation transaction isolation level. TiDB selects a replica for the first time based on the configuration of `tidb_replica_read`. From the second retry, TiDB will prioritize ensuring successful reading. Therefore, when the selected follower node has an inaccessible fault or other errors, it will switch to the leader for service.
81+
82+
#### `leader`
83+
84+
- Select the leader replica for reading, regardless of replica location.
85+
86+
#### `closest-replicas`
87+
88+
- When the replica in the same AZ as TiDB is the leader node, Follower Read is not used.
89+
- When the replica in the same AZ as TiDB is not the leader node, Follower Read is used.
90+
91+
#### `closest-adaptive`
92+
93+
- If the estimated return result is not large enough, use the `leader` policy and do not perform Follower Read.
94+
- If the estimated return result is large enough, use the `closest-replicas` policy.
95+
96+
### Follower Read performance overhead
97+
98+
To ensure strong data consistency, Follower Read needs to perform a `ReadIndex` regardless of how much data is read, which inevitably consumes more TiKV CPU resources. Therefore, in small query (such as point query) scenarios, the performance loss of Follower Read is relatively more obvious. At the same time, because the traffic saved by local reading for small queries is limited, it is more recommended to use Follower Read in large query or batch reading scenarios.
7299

73-
However, for the non-coprocessor requests, such as a point query, the granularity of the Follower Read load balancing policy is at the transaction level. For a TiDB transaction on a specific Region, the selected follower is fixed, and is switched only when it fails or the scheduling policy is adjusted. If a transaction contains both point queries and coprocessor requests, the two types of requests are scheduled for reading separately according to the preceding scheduling policy. In this case, even if a coprocessor request and a point query are for the same Region, TiDB processes them as independent events.
100+
When `tidb_replica_read` is `closest-adaptive`, TiDB does not use Follower Read for small queries, so the extra overhead of TiKV CPU compared to the `leader` policy is generally within +10% in various workloads.

grafana-tidb-dashboard.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,9 +123,14 @@ The following metrics relate to requests sent to TiKV. Retry requests are counte
123123
- **local**: the number of requests per second that attempt a stale read in the local zone
124124
- Stale Read Req Traffic:
125125
- **cross-zone-in**: the incoming traffic of responses to requests that attempt a stale read in a remote zone
126-
- **cross-zone-out**: the outgoing traffic of requests that attempt a stale read in a remote zone
126+
- **cross-zone-out**: the outgoing traffic of responses to requests that attempt a stale read in a remote zone
127127
- **local-in**: the incoming traffic of responses to requests that attempt a stale read in the local zone
128128
- **local-out**: the outgoing traffic of requests that attempt a stale read in the local zone
129+
- Read Req Traffic
130+
- **leader-local**: Traffic generated by Leader Read processing read requests in the local zone
131+
- **leader-cross-zone**: Traffic generated by Leader Read processing read requests in the remote zone
132+
- **follower-local**: Traffic generated by Follower Read processing read requests in the local zone
133+
- **follower-cross-zone**: Traffic generated by Follower Read processing read requests in the remote zone
129134

130135
### PD Client
131136

system-variables.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5375,7 +5375,7 @@ SHOW WARNINGS;
53755375
- Type: Enumeration
53765376
- Default value: `leader`
53775377
- Possible values: `leader`, `follower`, `leader-and-follower`, `prefer-leader`, `closest-replicas`, `closest-adaptive`, and `learner`. The `learner` value is introduced in v6.6.0.
5378-
- This variable is used to control where TiDB reads data.
5378+
- This variable is used to control where TiDB reads data. From v8.5.4, this variable only takes effect on read-only SQL statements.
53795379
- For more details about usage and implementation, see [Follower read](/follower-read.md).
53805380
53815381
### tidb_restricted_read_only <span class="version-mark">New in v5.2.0</span>

0 commit comments

Comments
 (0)