Skip to content

feat: support co-partitioned range right-side equi hash joins#23484

Open
gmhelmold wants to merge 2 commits into
apache:mainfrom
gmhelmold:feat/right-join-range-copartition
Open

feat: support co-partitioned range right-side equi hash joins#23484
gmhelmold wants to merge 2 commits into
apache:mainfrom
gmhelmold:feat/right-join-range-copartition

Conversation

@gmhelmold

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

#23184 let compatible range-partitioned inputs satisfy inner partitioned hash joins without repartitioning. Right-side partitioned equi joins have the same locality guarantee: Right, RightSemi, RightAnti and RightMark all anchor every output row on the probe (right) partition (on_lr_is_preserved is probe-side for all four), so when both inputs are co-partitioned by the join keys, execution stays partition-local and the hash repartition is unnecessary.

This removes unnecessary RepartitionExecs for already-co-located inputs, extending the inner-join behavior from #23184 to the right-side variants. No micro-benchmark included, consistent with #23184; correctness is demonstrated by matched/unmatched execution results below.

What changes are included in this PR?

The only production change is widening the existing inner-only gate in HashJoinExec::input_distribution_requirements from join_type == JoinType::Inner to matches!(join_type, Inner | Right | RightSemi | RightAnti | RightMark) (under PartitionMode::Partitioned). The co_partitioned / range-satisfaction machinery from #23184 is unchanged — the sanity checker and enforce_distribution consume range satisfaction join-type-agnostically.

One behavior note for #23376: range co-partitioned right joins now stay Range/Range, so partitioned dynamic filters are disabled for them (has_partitioned_dynamic_filter_routing returns false), the same safe delta #23184 introduced for inner joins. These variants are probe-not-preserved for pruning, so dynamic filters were not eligible to prune their probe rows regardless.

Are these changes tested?

Yes.

  • Optimizer (enforce_distribution.rs): 4 reuse tests (one per join type) proving compatible range/range inputs keep Range partitioning with no RepartitionExec; 4 incompatibility tests proving that mismatched split points, sort options, partition counts, or join-key expressions still insert a hash repartition; sanity-check pairs mirroring Support co-partitioned range inner equi joins #23184.
  • Execution (range_partitioning.slt): EXPLAIN plan pins plus matched/unmatched result checks for Right (incl. NULL left values), RightSemi, RightAnti, and an incompatible-layout Right join (repartitions, correct results).
  • The new reuse tests fail on main without the gate change (verified by reverting the production diff: exactly the 4 reuse tests fail, everything else passes).
  • RightMark testing note: RightMark is not reachable from SQL in sqllogictest (IN-subquery decorrelation emits LeftMark; physical RightMark only appears via a statistics-based swap), so its co-partitioning behavior is pinned at the optimizer/plan level, and mark null-marker semantics (matched/unmatched/NULL build keys) are pinned via the LeftMark path in the slt.

Are there any user-facing changes?

No API changes. Plans over compatible range-partitioned inputs avoid a hash repartition for right-side equi hash joins.

@github-actions github-actions Bot added core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) physical-plan Changes to the physical-plan crate labels Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate physical-plan Changes to the physical-plan crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow co-partitioned Partitioning::Range inputs for right-side hash joins

1 participant