feat: support co-partitioned range right-side equi hash joins#23484
Open
gmhelmold wants to merge 2 commits into
Open
feat: support co-partitioned range right-side equi hash joins#23484gmhelmold wants to merge 2 commits into
gmhelmold wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Partitioning::Rangeinputs for right-side hash joins #23453.Rationale for this change
#23184 let compatible range-partitioned inputs satisfy inner partitioned hash joins without repartitioning. Right-side partitioned equi joins have the same locality guarantee:
Right,RightSemi,RightAntiandRightMarkall anchor every output row on the probe (right) partition (on_lr_is_preservedis probe-side for all four), so when both inputs are co-partitioned by the join keys, execution stays partition-local and the hash repartition is unnecessary.This removes unnecessary
RepartitionExecs for already-co-located inputs, extending the inner-join behavior from #23184 to the right-side variants. No micro-benchmark included, consistent with #23184; correctness is demonstrated by matched/unmatched execution results below.What changes are included in this PR?
The only production change is widening the existing inner-only gate in
HashJoinExec::input_distribution_requirementsfromjoin_type == JoinType::Innertomatches!(join_type, Inner | Right | RightSemi | RightAnti | RightMark)(underPartitionMode::Partitioned). Theco_partitioned/ range-satisfaction machinery from #23184 is unchanged — the sanity checker and enforce_distribution consume range satisfaction join-type-agnostically.One behavior note for #23376: range co-partitioned right joins now stay
Range/Range, so partitioned dynamic filters are disabled for them (has_partitioned_dynamic_filter_routingreturns false), the same safe delta #23184 introduced for inner joins. These variants are probe-not-preserved for pruning, so dynamic filters were not eligible to prune their probe rows regardless.Are these changes tested?
Yes.
enforce_distribution.rs): 4 reuse tests (one per join type) proving compatible range/range inputs keepRangepartitioning with noRepartitionExec; 4 incompatibility tests proving that mismatched split points, sort options, partition counts, or join-key expressions still insert a hash repartition; sanity-check pairs mirroring Support co-partitioned range inner equi joins #23184.range_partitioning.slt):EXPLAINplan pins plus matched/unmatched result checks forRight(incl.NULLleft values),RightSemi,RightAnti, and an incompatible-layoutRightjoin (repartitions, correct results).mainwithout the gate change (verified by reverting the production diff: exactly the 4 reuse tests fail, everything else passes).RightMarktesting note:RightMarkis not reachable from SQL in sqllogictest (IN-subquery decorrelation emitsLeftMark; physicalRightMarkonly appears via a statistics-based swap), so its co-partitioning behavior is pinned at the optimizer/plan level, and mark null-marker semantics (matched/unmatched/NULLbuild keys) are pinned via theLeftMarkpath in the slt.Are there any user-facing changes?
No API changes. Plans over compatible range-partitioned inputs avoid a hash repartition for right-side equi hash joins.