Add docs for Analyze Embedded in DDL and new system variable #22014
Conversation
Introduces the 'Analyze Embedded in DDL' feature documentation and adds a new page describing its behavior for index creation and reorganization. Updates TOC to include the new doc and documents the 'tidb_stats_update_during_ddl' system variable, which controls this feature.
ti-chi-bot
bot
added
missing-translation-status
Summary of ChangesHello @hfxsd, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request adds new documentation for TiDB's 'Analyze Embedded in DDL' feature, which is designed to improve query plan stability and performance. By automatically collecting statistics for new or reorganized indexes during DDL operations, the feature ensures that the optimizer has accurate and up-to-date information immediately, preventing suboptimal query plans that could arise from missing statistics. The documentation details the feature's purpose, use cases, and how it's controlled by the new Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
hfxsd
added
translation/from-docs-cn
hfxsd
added
type/compatibility-or-feature-change
hfxsd
changed the title
ti-chi-bot
bot
added
the
size/L
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces documentation for the 'Analyze Embedded in DDL' feature and its corresponding system variable. The documentation is well-structured and informative. I've provided a few suggestions to correct some typos in the examples and to align the text with the repository's style guide, primarily concerning the formatting of command names.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
@AilinKid: adding LGTM is restricted to approvers and reviewers in OWNERS files. In response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
|
||
| When this feature is enabled, TiDB automatically runs an `ANALYZE` (statistics collection) operation before the new or reorganized index becomes visible to users. This prevents inaccurate optimizer estimates and potential plan changes caused by temporarily unavailable statistics after index creation or reorganization. | ||
|
|
||
| ## Use scenarios |
There was a problem hiding this comment.
| ## Use scenarios | |
| ## Usage scenarios |
|
|
||
| When `tidb_stats_update_during_ddl` is `ON`, executing [`ADD INDEX`](/sql-statements/sql-statement-add-index.md) automatically runs an embedded `ANALYZE` operation after the Reorg phase finishes. This `ANALYZE` operation collects statistics for the newly created index before the index becomes visible to users, and then `ADD INDEX` proceeds with its remaining phases. | ||
|
|
||
| Considering that `ANALYZE` can take time, TiDB sets a timeout threshold based on the execution time of the first Reorg. If `ANALYZE` times out, `ADD INDEX` will stop waiting synchronously for `ANALYZE` to finish and will continue the subsequent process so that the index becomes visible earlier to users. This means the index statistics will be updated after `ANALYZE` completes asynchronously. |
There was a problem hiding this comment.
| Considering that `ANALYZE` can take time, TiDB sets a timeout threshold based on the execution time of the first Reorg. If `ANALYZE` times out, `ADD INDEX` will stop waiting synchronously for `ANALYZE` to finish and will continue the subsequent process so that the index becomes visible earlier to users. This means the index statistics will be updated after `ANALYZE` completes asynchronously. | |
| Considering that `ANALYZE` can take time, TiDB sets a timeout threshold based on the execution time of the first Reorg. If `ANALYZE` times out, `ADD INDEX` stops waiting synchronously for `ANALYZE` to finish and continues the subsequent process, making the index visible earlier to users. This means the index statistics will be updated after `ANALYZE` completes asynchronously. |
| When `tidb_stats_update_during_ddl` is `ON`, executing [`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md) or [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md) that reorganizes an index will also run an embedded `ANALYZE` operation after the Reorg phase completes. The mechanism is the same as for `ADD INDEX`: | ||
|
|
||
| - Start collecting statistics before the index becomes visible. | ||
| - If `ANALYZE` times out, [`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md) and [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md) will not synchronously wait for `ANALYZE` to finish and will continue so the index becomes visible earlier to users. This means that the index statistics will be updated when `ANALYZE` finishes asynchronously. |
There was a problem hiding this comment.
| - If `ANALYZE` times out, [`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md) and [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md) will not synchronously wait for `ANALYZE` to finish and will continue so the index becomes visible earlier to users. This means that the index statistics will be updated when `ANALYZE` finishes asynchronously. | |
| - If `ANALYZE` times out, [`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md) and [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md) stops waiting synchronously for `ANALYZE` to finish and continues the subsequent process, making the index visible earlier to users. This means that the index statistics will be updated when `ANALYZE` finishes asynchronously. |
| - Persists to cluster: Yes | ||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||
| - Default value: `OFF` | ||
| - This variable controls whether to enable DDL-embedded `ANALYZE`. When enabled, DDL statements that create new indexes ([`ADD INDEX`](/sql-statements/sql-statement-add-index.md)) and DDL statements that reorganize existing indexes ([`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md) and [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md)) automatically run statistics collection before the index becomes visible. For more information, see [`ANALYZE` Embedded in DDL Statements](/ddl_embedded_analyze.md). |
There was a problem hiding this comment.
| - This variable controls whether to enable DDL-embedded `ANALYZE`. When enabled, DDL statements that create new indexes ([`ADD INDEX`](/sql-statements/sql-statement-add-index.md)) and DDL statements that reorganize existing indexes ([`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md) and [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md)) automatically run statistics collection before the index becomes visible. For more information, see [`ANALYZE` Embedded in DDL Statements](/ddl_embedded_analyze.md). | |
| - This variable controls whether to enable DDL-embedded `ANALYZE`. When enabled, DDL statements that create new indexes ([`ADD INDEX`](/sql-statements/sql-statement-add-index.md)) or reorganize existing indexes ([`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md) and [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md)) automatically collect statistics before the index becomes visible. For more information, see [`ANALYZE` Embedded in DDL Statements](/ddl_embedded_analyze.md). |
|
|
||
| Considering that `ANALYZE` can take time, TiDB sets a timeout threshold based on the execution time of the first Reorg. If `ANALYZE` times out, `ADD INDEX` will stop waiting synchronously for `ANALYZE` to finish and will continue the subsequent process so that the index becomes visible earlier to users. This means the index statistics will be updated after `ANALYZE` completes asynchronously. | ||
|
|
||
| Example: |
There was a problem hiding this comment.
| Example: | |
| For example: |
|
|
||
| - Start collecting statistics before the index becomes visible. | ||
| - If `ANALYZE` times out, [`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md) and [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md) will not synchronously wait for `ANALYZE` to finish and will continue so the index becomes visible earlier to users. This means that the index statistics will be updated when `ANALYZE` finishes asynchronously. | ||
|
|
There was a problem hiding this comment.
| For example: | |
| 1 rows in set (0.001 sec) | ||
| ``` | ||
|
|
||
| From the `MODIFY COLUMN` example, when `tidb_stats_update_during_ddl` is `ON`, you can see that in the following `EXPLAIN` the index `idx` has its statistics automatically collected and loaded into memory (you can verify it by executing `SHOW STATS_HISTOGRAMS`), so the optimizer can immediately use those statistics for a range scan. If index creation or reorganization and `ANALYZE` take a long time, check the DDL job status by executing `ADMIN SHOW DDL JOBS`. If the `COMMENTS` column contains `analyzing`, it indicates that the DDL job is collecting statistics. |
There was a problem hiding this comment.
| From the `MODIFY COLUMN` example, when `tidb_stats_update_during_ddl` is `ON`, you can see that in the following `EXPLAIN` the index `idx` has its statistics automatically collected and loaded into memory (you can verify it by executing `SHOW STATS_HISTOGRAMS`), so the optimizer can immediately use those statistics for a range scan. If index creation or reorganization and `ANALYZE` take a long time, check the DDL job status by executing `ADMIN SHOW DDL JOBS`. If the `COMMENTS` column contains `analyzing`, it indicates that the DDL job is collecting statistics. | |
| From the `MODIFY COLUMN` example, when `tidb_stats_update_during_ddl` is `ON`, you can see that after the execution of the `MODIFY COLUMN` DDL statement, the subsequent `EXPLAIN` output shows that statistics for the index `idx` have been automatically collected and loaded into memory (you can verify it by executing `SHOW STATS_HISTOGRAMS`). As a result, the optimizer can immediately use these statistics for range scans. If index creation or reorganization and `ANALYZE` take a long time, you can check the DDL job status by executing `ADMIN SHOW DDL JOBS`. When the `COMMENTS` column in the output contains `analyzing`, it means that the DDL job is collecting statistics. |
| 1 rows in set (0.001 sec) | ||
| ``` | ||
|
|
||
| In the `ADD INDEX` example, when `tidb_stats_update_during_ddl` is `ON`, you can see that in the subsequent `EXPLAIN`, the index `idx` has its statistics automatically collected and loaded into memory (you can verify it by running `SHOW STATS_HISTOGRAMS`). Therefore, the optimizer can immediately use those statistics for a range scan. If index creation or reorganization and `ANALYZE` take a long time, you can check the DDL job status by executing `ADMIN SHOW DDL JOBS`. If the `COMMENTS` column contains `analyzing`, it means that the DDL job is collecting statistics. |
There was a problem hiding this comment.
| In the `ADD INDEX` example, when `tidb_stats_update_during_ddl` is `ON`, you can see that in the subsequent `EXPLAIN`, the index `idx` has its statistics automatically collected and loaded into memory (you can verify it by running `SHOW STATS_HISTOGRAMS`). Therefore, the optimizer can immediately use those statistics for a range scan. If index creation or reorganization and `ANALYZE` take a long time, you can check the DDL job status by executing `ADMIN SHOW DDL JOBS`. If the `COMMENTS` column contains `analyzing`, it means that the DDL job is collecting statistics. | |
| From the `ADD INDEX` example, when `tidb_stats_update_during_ddl` is `ON`, you can see that after the execution of the `ADD INDEX` DDL statement, the subsequent `EXPLAIN` output shows that statistics for the index `idx` have been automatically collected and loaded into memory (you can verify it by executing `SHOW STATS_HISTOGRAMS`). As a result, the optimizer can immediately use these statistics for range scans. If index creation or reorganization and `ANALYZE` take a long time, you can check the DDL job status by executing `ADMIN SHOW DDL JOBS`. When the `COMMENTS` column in the output contains `analyzing`, it means that the DDL job is collecting statistics. |
[LGTM Timeline notifier]Timeline:
|
3 participants
Introduces the 'Analyze Embedded in DDL' feature documentation and adds a new page describing its behavior for index creation and reorganization. Updates TOC to include the new doc and documents the 'tidb_stats_update_during_ddl' system variable, which controls this feature.
First-time contributors' checklist
What is changed, added or deleted? (Required)
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?