Enable target_file_size_is_upper_bound by default #14126
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: This change the default value of target_file_size_is_upper_bound from false to true. This option enforces that SST files strictly respect the `target_file_size_base` limit as an upper bound (hard limit) rather than a soft target. Test Plan: Existing tests. Updated multiple tests to explicitly disable the option where the old behavior is needed for testing specific behavior Reviewers: Subscribers: Tasks: Tags:
`num_data_blocks` is only written by the emit thread, never by worker threads, and read by emit thread when deciding when to cut an output file during compaction
Contributor
Author
|
Crash tests are failing due to an assertion failure in BlockBasedTableBuilder::Finish that checks the estimated file size is greater than the actual file size. This assertion exists to validate target_file_size_is_upper_bound option strictly enforces the target_file_size limit for compaction files. Due to this, some crash tests fail because the estimated file size is underestimated. I plan to improve the accuracy of the tail size estimate in a follow-up PR, as this issue does not affect correctness. |
pdillinger
requested changes
Nov 20, 2025
Contributor
pdillinger
left a comment
pdillinger
left a comment
There was a problem hiding this comment.
Waiting for more production validation first. Also, we need to keep the crash test in a relatively clean state best we can.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Enable
target_file_size_is_upper_boundoption by default to enforce stricter file size limits when deciding whether to cut a compaction output file. This prevents output SST files from exceedingtarget_file_size_base. When false, SST files might exceed thetarget_file_size_base.Changes:
Test Plan:
Existing tests.