-
Notifications
You must be signed in to change notification settings - Fork 628
Enhanced the documentation for the convert_index_to_remote ISM action with comprehensive details and examples: #11491 #11495
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -415,16 +415,36 @@ Parameter | Description | Type | Required | Default | |||||||||
|
|
||||||||||
| ### convert_index_to_remote | ||||||||||
|
|
||||||||||
| Converts an index from a local snapshot repository to a remote repository. | ||||||||||
| Converts an index to a searchable snapshot by restoring it from a remote snapshot repository. This action is useful for reducing storage costs by moving infrequently accessed data to remote storage while keeping it searchable. The action performs a restore operation from the specified snapshot and automatically deletes the original index once the restore is successfully accepted, ensuring only the remote index remains. | ||||||||||
|
|
||||||||||
| The `convert_index_to_remote` operation has the following parameters. | ||||||||||
|
|
||||||||||
| Parameter | Description | Type | Required | Default | ||||||||||
| :--- | :--- |:--- |:--- | | ||||||||||
| `repository` | The repository name registered through the native snapshot API operations. | String | Yes | N/A | ||||||||||
| `snapshot` | The snapshot name created through the snapshot action. | String | Yes | N/A | ||||||||||
| `repository` | The repository name registered through the native snapshot API operations. Must be a remote repository type (for example, S3, Azure, or GCS). | `string` | Yes | N/A | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What are the actual valid values for this parameter? Is it
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please format data types without code font (so, "String", "Boolean", etc.) |
||||||||||
| `snapshot` | The snapshot name created through the snapshot action. | `string` | Yes | N/A | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| `include_aliases` | Whether to include index aliases during the restore operation. When set to `true`, all aliases associated with the original index are restored along with the remote index. Set to `true` if your application references the index through aliases. | `boolean` | No | `false` | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| `ignore_index_settings` | A comma-separated list of index settings to ignore during the restore operation. For example, `"index.refresh_interval,index.number_of_replicas"`. This is useful when you want to apply different settings to the restored remote index than what was configured in the original index. | `string` | No | Empty string | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| `number_of_replicas` | The number of replicas to configure for the restored remote index. This allows you to control replica allocation during the conversion process without requiring a separate update operation. Setting this during conversion helps prevent cluster yellow state or unnecessary load from replica allocation. | `integer` | No | `0` | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
|
|
||||||||||
| Make sure that the repository name used in the `convert_index_to_remote` operation matches the repository name specified during the snapshot action. Additionally, you can reference the snapshot using `{% raw %}{{ctx.index}}{% endraw %}`, as shown in the following example policy: | ||||||||||
| #### Prerequisites | ||||||||||
|
|
||||||||||
| Before using the `convert_index_to_remote` action, ensure the following: | ||||||||||
|
|
||||||||||
| - A remote repository (S3, Azure, or GCS) is registered and accessible. | ||||||||||
| - A snapshot of the index exists in the specified repository, typically created using the `snapshot` action. | ||||||||||
| - The repository name matches the one used in the snapshot action. | ||||||||||
|
|
||||||||||
| #### Usage notes | ||||||||||
|
|
||||||||||
| - **Automatic deletion**: The original index is automatically deleted after the remote snapshot restore is successfully accepted. This ensures that only the searchable snapshot version remains, completing the conversion process. | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| - **Repository matching**: The repository name used in the `convert_index_to_remote` operation must match the repository name specified during the snapshot action. | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| - **Variable references**: You can reference the snapshot using Mustache variables like `{% raw %}{{ctx.index}}{% endraw %}` or `{% raw %}{{ctx.indexUuid}}{% endraw %}` for dynamic naming. | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| - **Replica considerations**: Consider your cluster's capacity when setting `number_of_replicas`. If there aren't enough eligible nodes for replica restoration, the cluster may enter a yellow state. | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
|
|
||||||||||
| #### Basic example | ||||||||||
|
|
||||||||||
| The following example shows a basic conversion using the minimum required parameters: | ||||||||||
|
|
||||||||||
| ```json | ||||||||||
| { | ||||||||||
|
|
@@ -440,6 +460,72 @@ Make sure that the repository name used in the `convert_index_to_remote` operati | |||||||||
| ``` | ||||||||||
| {% include copy.html %} | ||||||||||
|
|
||||||||||
| #### Advanced configuration example | ||||||||||
|
|
||||||||||
| The following example demonstrates using all available configuration options. This configuration includes aliases, ignores certain index settings during restore, and configures two replicas for the searchable snapshot: | ||||||||||
|
|
||||||||||
| ```json | ||||||||||
| { | ||||||||||
| "convert_index_to_remote": { | ||||||||||
| "repository": "my_backup", | ||||||||||
| "snapshot": "daily-snapshot", | ||||||||||
| "include_aliases": true, | ||||||||||
| "ignore_index_settings": "index.refresh_interval,index.number_of_replicas", | ||||||||||
| "number_of_replicas": 0 | ||||||||||
| } | ||||||||||
| } | ||||||||||
| ``` | ||||||||||
| {% include copy.html %} | ||||||||||
|
|
||||||||||
| #### Complete policy example | ||||||||||
|
|
||||||||||
| The following policy moves indexes older than 30 days to searchable snapshots with optimized settings for cost efficiency: | ||||||||||
|
|
||||||||||
| ```json | ||||||||||
| { | ||||||||||
| "policy": { | ||||||||||
| "description": "Convert old indexes to searchable snapshots", | ||||||||||
| "default_state": "active", | ||||||||||
| "states": [ | ||||||||||
| { | ||||||||||
| "name": "active", | ||||||||||
| "actions": [], | ||||||||||
| "transitions": [ | ||||||||||
| { | ||||||||||
| "state_name": "archive", | ||||||||||
| "conditions": { | ||||||||||
| "min_index_age": "30d" | ||||||||||
| } | ||||||||||
| } | ||||||||||
| ] | ||||||||||
| }, | ||||||||||
| { | ||||||||||
| "name": "archive", | ||||||||||
| "actions": [ | ||||||||||
| { | ||||||||||
| "snapshot": { | ||||||||||
| "repository": "remote-repo", | ||||||||||
| "snapshot": "{% raw %}{{ctx.index}}{% endraw %}" | ||||||||||
| } | ||||||||||
| }, | ||||||||||
| { | ||||||||||
| "convert_index_to_remote": { | ||||||||||
| "repository": "remote-repo", | ||||||||||
| "snapshot": "{% raw %}{{ctx.index}}{% endraw %}", | ||||||||||
| "include_aliases": true, | ||||||||||
| "ignore_index_settings": "index.refresh_interval,index.number_of_replicas", | ||||||||||
| "number_of_replicas": 0 | ||||||||||
| } | ||||||||||
| } | ||||||||||
| ], | ||||||||||
| "transitions": [] | ||||||||||
| } | ||||||||||
| ] | ||||||||||
| } | ||||||||||
| } | ||||||||||
| ``` | ||||||||||
| {% include copy.html %} | ||||||||||
|
|
||||||||||
| ### index_priority | ||||||||||
|
|
||||||||||
| Set the priority for the index in a specific state. Unallocated shards of indexes are recovered in the order of their priority, whenever possible. The indexes with higher priority values are recovered first followed by the indexes with lower priority values. | ||||||||||
|
|
||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.