-
Notifications
You must be signed in to change notification settings - Fork 291
feat(mria): Add commands related to the core/replicant rebalance #2910
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
ieQu1
wants to merge
2
commits into
release-5.8
Choose a base branch
from
dev/ieQu1/mria-core-repl-rebalance-cli
base: release-5.8
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -102,9 +102,9 @@ Loaded 'Mod' module on []: ok | |
|
|
||
| ## conf cluster_sync | ||
|
|
||
| This command is mostly for troubleshooting when there is something wrong with cluster-calls used to sync configuration changes between the nodes in the cluster. | ||
| This command is mostly for troubleshooting when there is something wrong with cluster-calls used to sync configuration changes between the nodes in the cluster. | ||
|
|
||
| ::: tip | ||
| ::: tip | ||
|
|
||
| In EMQX 5.0.x, this command was named `cluster_call`. This old command is still available in EMQX 5.1 but it is not displayed in the help information. | ||
|
|
||
|
|
@@ -275,6 +275,62 @@ $ emqx ctl cluster status --json | |
| } | ||
| ``` | ||
|
|
||
| ### cluster core rebalance | ||
|
|
||
| Cluster core rebalance feature allows to balance load in a core/replicant cluster. | ||
|
|
||
| It provides a semi-automatic way to evenly distribute ownership over replicants among the core nodes, which may be necessary after bringing a new core node into the cluster or restart of a core node. | ||
|
|
||
| ::: tip | ||
| Core node rebalance procedure may introduce traffic disturbance. | ||
| During the procedure clients attached to the replicants may experience increased latency and loss of messages may be observed. | ||
| ::: | ||
|
|
||
| Rebalance is performed in three stages: plan, check and execute. Each stage is triggered by a CLI command. All commands should be executed on a core node. | ||
|
|
||
| #### cluster core rebalance plan | ||
|
|
||
| This command collects statistics about the number of replicants attached to the cores and derives a plan for evening out this number. | ||
|
|
||
| ```bash | ||
| $ bin/emqx ctl cluster core rebalance plan | ||
| {ok,...} | ||
| ``` | ||
|
|
||
| This command doesn't perform any actions on its own and can be used for a dry run. | ||
| A rebalance plan that was not confirmed will automatically expire in one minute. | ||
|
|
||
| #### cluster core rebalance status | ||
|
|
||
| This command displays status of the rebalance, planned or ongoing. | ||
|
|
||
| *Output*: | ||
|
|
||
| 1. `not_started` is printed if `cluster core rebalance plan` was not previously executed or rebalance was aborted. | ||
| 1. If plan stage detected an imbalance, then the output will contain `wait_confirmation` status and a list of actions that will be taken to reach the balance. | ||
| ``` | ||
| {wait_confirmation,[{kick,emqx_telemetry_shard,'[email protected]', | ||
| [<93747.6968.0>]}, | ||
| {kick,emqx_dashboard_shard,'[email protected]', | ||
| [<93747.6969.0>]}, | ||
| {kick,emqx_ds_builtin_metadata_shard,'[email protected]', | ||
| [<93747.7110.0>]}]} | ||
| ``` | ||
| 1. If the rebalance is ongoing, status will change to `running` and list of remaining actions will be printed. | ||
| 1. If rebalance is complete, status will change to `complete` and list of remaining actions will become empty: | ||
|
|
||
| ``` | ||
| {complete,[]} | ||
| ``` | ||
|
|
||
| #### cluster core rebalance confirm | ||
|
|
||
| This command puts the planned rebalance into action by executing the scheduled commands with a cooldown interval. | ||
|
|
||
| #### cluster core rebalance abort | ||
|
|
||
| This commands aborts the ongoing rebalance. | ||
ieQu1 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## clients | ||
|
|
||
| This command is to view and manage connected clients. | ||
|
|
@@ -704,7 +760,7 @@ This command is like the `trace` command, but applies on all nodes in the cluste | |
|
|
||
| ## traces | ||
|
|
||
| This command is similar to the `trace` command, but it starts or stops a tracer across all nodes in the cluster. | ||
| This command is similar to the `trace` command, but it starts or stops a tracer across all nodes in the cluster. | ||
|
|
||
| | Command | Description | | ||
| | ------------------------------------------------------- | --------------------------------- | | ||
|
|
@@ -832,7 +888,7 @@ Restarted tcp:default listener successfully. | |
| Restarting a listener causes all the connected clients to disconnect. | ||
| ::: | ||
|
|
||
| ### listeners enable \<Identifier\> <true/false> | ||
| ### listeners enable \<Identifier\> <true/false> | ||
|
|
||
| ```bash | ||
| $ emqx ctl listeners enable tcp:default true | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.