Skip to content

[HUDI-9421] Update restore/rollback/indexing planning and instant generation #13368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 30, 2025

Conversation

the-other-tim-brown
Copy link
Contributor

@the-other-tim-brown the-other-tim-brown commented May 28, 2025

Change Logs

  • Updates the instant time and plan generation to happen in the same lock for restore and rollback actions

Impact

  • Ensures timeline consistency and avoids possible race conditions in the gap between generating the instant time and writing out the plan

Risk level (write none, low medium or high below)

Low

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".

  • The config description must be updated if new configs are added or the default value of the configs are changed
  • Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
    ticket number here and follow the instruction to make
    changes to the website.

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@github-actions github-actions bot added the size:S PR with lines of changes in (10, 100] label May 28, 2025
@the-other-tim-brown the-other-tim-brown marked this pull request as ready for review May 29, 2025 00:40
@the-other-tim-brown the-other-tim-brown changed the title [HUDI-941] Update restore/rollback planning and instant generation [HUDI-941] Update restore/rollback/indexing planning and instant generation May 29, 2025
}
try {
rollbackInstantTime = suppliedRollbackInstantTime.orElseGet(() -> createNewInstantTime(false));
if (commitInstantOpt.isEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can move it to 1144 to avoid unnecessary lock.

@github-actions github-actions bot added size:M PR with lines of changes in (100, 300] and removed size:S PR with lines of changes in (10, 100] labels May 29, 2025
@vinothchandar vinothchandar self-assigned this May 29, 2025
@hudi-bot
Copy link

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

Copy link
Member

@vinothchandar vinothchandar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Existing tests should cover this. We need some functional tests that test some of these concurrently and ensure the requested instants meet the guarantees.. Can we file a separate JIRA for "improve concurrency FTs" and we can come back to it after code freeze.

@the-other-tim-brown
Copy link
Contributor Author

Existing tests should cover this. We need some functional tests that test some of these concurrently and ensure the requested instants meet the guarantees.. Can we file a separate JIRA for "improve concurrency FTs" and we can come back to it after code freeze.

@vinothchandar Created this JIRA: https://issues.apache.org/jira/browse/HUDI-9478

@danny0405 danny0405 merged commit a23faa2 into apache:master May 30, 2025
60 checks passed
@danny0405
Copy link
Contributor

danny0405 commented May 30, 2025

@the-other-tim-brown Can you fix the title with correct JIRA ID, 941 seems not right.

@the-other-tim-brown the-other-tim-brown changed the title [HUDI-941] Update restore/rollback/indexing planning and instant generation [HUDI-9421] Update restore/rollback/indexing planning and instant generation May 30, 2025
@the-other-tim-brown
Copy link
Contributor Author

@the-other-tim-brown Can you fix the title with correct JIRA ID, 941 seems not right.

Fixed, it is HUDI-9421

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:M PR with lines of changes in (100, 300]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants