Skip to content

[GLUTEN-11550][VL] Fix GlutenWholeStageCodegenSuite in Spark 4.x #12507

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zml1206 merged 2 commits into from
Jul 15, 2026
+1,544 −6
Merged

[GLUTEN-11550][VL] Fix GlutenWholeStageCodegenSuite in Spark 4.x #12507

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1815bae
[VL] Fix GlutenWholeStageCodegenSuite in Spark 4.x
zml1206 Jul 14, 2026
9ce6539
remove disable suite
zml1206 Jul 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -760,7 +760,34 @@ class VeloxTestSettings extends BackendTestSettings {
enableSuite[GlutenUnsafeKVExternalSorterSuite]
enableSuite[GlutenUnsafeRowSerializerSuite]
// TODO: 4.x enableSuite[GlutenWholeStageCodegenSparkSubmitSuite] // 1 failure
// TODO: 4.x enableSuite[GlutenWholeStageCodegenSuite] // 24 failures
enableSuite[GlutenWholeStageCodegenSuite]
// Rewrite with Gluten-aware native whole-stage plan assertions.
.exclude("range/filter should be combined")
.exclude("HashAggregate should be included in WholeStageCodegen")
.exclude("SortAggregate should be included in WholeStageCodegen")
Comment on lines +763 to +767
.exclude("GenerateExec should be included in WholeStageCodegen (whole-stage-codegen on)")
.exclude("HashAggregate with grouping keys should be included in WholeStageCodegen")
.exclude("BroadcastHashJoin should be included in WholeStageCodegen")
.exclude("Inner ShuffledHashJoin should be included in WholeStageCodegen")
.exclude(
"Full Outer ShuffledHashJoin and SortMergeJoin should be included in WholeStageCodegen")
.exclude("SPARK-44060 Code-gen for build side outer shuffled hash join")
.exclude("Left/Right Outer SortMergeJoin should be included in WholeStageCodegen")
.exclude("Left Semi SortMergeJoin should be included in WholeStageCodegen")
.exclude("Left Anti SortMergeJoin should be included in WholeStageCodegen")
.exclude("Inner/Cross BroadcastNestedLoopJoinExec should be included in WholeStageCodegen")
.exclude("Left/Right outer BroadcastNestedLoopJoinExec should be included in WholeStageCodegen")
.exclude("Left semi/anti BroadcastNestedLoopJoinExec should be included in WholeStageCodegen")
.exclude("Sort should be included in WholeStageCodegen")
.exclude("Control splitting consume function by operators with config")
.exclude("Skip splitting consume function when parameter number exceeds JVM limit")
.exclude(
"including codegen stage ID in generated class name should not regress codegen caching")
.exclude("SPARK-26572: evaluate non-deterministic expressions for aggregate results")
.exclude("SPARK-28520: WholeStageCodegen does not work properly for LocalTableScanExec")
.exclude("Give up splitting aggregate code if a parameter length goes over the limit")
.exclude("Give up splitting subexpression code if a parameter length goes over the limit")
.exclude("SPARK-47238: Test broadcast threshold for generated code")
enableSuite[GlutenBroadcastExchangeSuite]
enableSuite[GlutenLocalBroadcastExchangeSuite]
enableSuite[GlutenCoalesceShufflePartitionsSuite]
Expand Down
Loading
Loading