[VL] Diable spark join keys rewrite#11643
Closed
JkSelf wants to merge 3 commits into
Closed
Conversation
|
Run Gluten Clickhouse CI on x86 |
JkSelf
force-pushed
the
disableJoinKeyRewrite
branch
from
February 24, 2026 21:08
bbe1180 to
27ff7a0
Compare
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
JkSelf
force-pushed
the
disableJoinKeyRewrite
branch
from
February 25, 2026 09:52
ddf6973 to
c627aeb
Compare
|
Run Gluten Clickhouse CI on x86 |
|
|
||
| override def enableJoinKeysRewrite(): Boolean = false | ||
| override def enableJoinKeysRewrite(): Boolean = { | ||
| GlutenConfig.get.hashJoinKeysRewrite |
Contributor
There was a problem hiding this comment.
Looks like the previous value is false?
| trait HashJoin extends JoinCodegenSupport { | ||
| def buildSide: BuildSide | ||
|
|
||
| override def simpleStringWithNodeId(): String = { |
Contributor
There was a problem hiding this comment.
Why do you need to copy the file?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
When offloading Hash Join to Velox, Gluten currently cannot fully leverage the performance benefits of the Hash Join Key Rewriting feature. In fact, enabling this feature can lead to performance degradation.
During our optimization of BHJ—aimed at building the hash table only once per executor—we discovered that Dynamic Filter Push-down DFP is enabled when Key Rewriting is enabled. This occurs because the rewriting process introduces CAST(int AS long) expressions to both the Build and Probe sides within the Project operator. Since Velox only supports filter push-down on columns without complex expressions, these added casts block the optimization, resulting in a performance degradation.
To address this, we have disabled the Key Rewriting feature specifically within the HashJoin#canRewriteAsLongType method.
How was this patch tested?
Existing unit tests
Was this patch authored or co-authored using generative AI tooling?
No