Repository navigation
Refactor outputPartitioning for few execs and add tests #14229
Description
Is your feature request related to a problem? Please describe.
This is a follow up issue based on review comments from PR - #14164
- Check if we need to add tests for these since we have added outputPartitiong.
GpuProjectExecLike: Fixed outputPartitioning to remap expressions through aliases (was missing, unlike Spark's PartitioningPreservingUnaryExecNode).
GpuBroadcastHashJoinExecBase: Added missing outputPartitioning override
GpuCustomShuffleReaderExec: Fixed outputPartitioning for AQE coalesced reads to preserve HashPartitioning (matching Spark's AQEShuffleReadExec).
GpuShuffledHashJoinExec: Added outputPartitioning override matching Spark's HashJoin trait behavior.
GpuShuffledSymmetricHashJoinExec: Added outputPartitioning override for InnerLike and FullOuter join types.
GpuShuffledAsymmetricHashJoinExec: Added outputPartitioning override for LeftOuter and RightOuter join types.
GpuBroadcastNestedLoopJoinExecBase: Added outputPartitioning override matching Spark's BroadcastNestedLoopJoinExec behavior.
-
Evaluate if we have to make GpuProjectExecLike extend PartitioningPreservingUnaryExecNode instead of copying the code.- covered in Fix incorrect GPU partitioning and ordering metadata [databricks] #16620 -
In GpuUnionExec, can we take the outputPartitioning of the cpu exec as a parameter of GpuUnionExec instead of duplicating the Spark code?
override def outputPartitioning: Partitioning =
GpuUnionExecShim.getOutputPartitioning(children, output, conf)
Activity