Skip to content

Fix GPU slice validation to match CPU behavior with null inputs #13230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Copilot wants to merge 6 commits into from
Closed

Fix GPU slice validation to match CPU behavior with null inputs #13230

Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9a0489a
Initial plan
Copilot Aug 1, 2025
441223b
Fix slice negative length validation to check nulls first
Copilot Aug 1, 2025
4531619
Fix GPU slice validation to match CPU behavior with null inputs
Copilot Aug 1, 2025
71251d2
Fix GPU slice validation to match CPU behavior with null inputs
Copilot Aug 1, 2025
97e5b48
Fix method call for checking null count in GpuSlice implementation
Copilot Aug 1, 2025
4e60e36
Fix line length issue in GpuSlice comment and condition
Copilot Aug 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion sql-plugin/src/main/scala/org/apache/spark/sql/rapids/collectionOperations.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,8 @@ case class GpuSlice(x: Expression, start: Expression, length: Expression)

override def doColumnar(listCol: GpuColumnVector, startS: GpuScalar,
lengthS: GpuScalar): ColumnVector = {
// When either start or length is null, return all nulls like the CPU does.
// When input column is all nulls or either start or length is null, return all nulls like the CPU does.

@pxLi pxLi Aug 1, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line is too long which caused

error file=/home/runner/work/spark-rapids/spark-rapids/sql-plugin/src/main/scala/org/apache/spark/sql/rapids/collectionOperations.scala message=File line length exceeds 100 characters. line=172
Processed 1332 file(s)

Copilot AI Aug 1, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the line length issue by breaking the long comment and condition into multiple lines in commit 4e60e36.

// This matches CPU behavior for slice function with null inputs.
if (listCol.getRowCount == listCol.numNulls() || !startS.isValid || !lengthS.isValid) {
GpuColumnVector.columnVectorFromNull(listCol.getRowCount.toInt, dataType)
} else {
Expand Down Expand Up @@ -216,6 +217,11 @@ case class GpuSlice(x: Expression, start: Expression, length: Expression)
} else {
val list = listCol.getBase
val start = startCol.getBase
withResource(GpuDivModLike.mergeNulls(list, start)) { mergedNulls =>
if (mergedNulls.getRowCount == mergedNulls.getNullCount) {
return GpuColumnVector.columnVectorFromNull(listCol.getRowCount.toInt, dataType)
}
}
val length = lengthS.getValue.asInstanceOf[Int]
withResource(Scalar.fromInt(0)) { zero =>
if (start.contains(zero)) {
Expand All @@ -237,6 +243,14 @@ case class GpuSlice(x: Expression, start: Expression, length: Expression)
val list = listCol.getBase
val start = startCol.getBase
val length = lengthCol.getBase
val mergedNullsList = withResource(GpuDivModLike.mergeNulls(list, start)) { mergedNulls =>
GpuDivModLike.mergeNulls(mergedNulls, length)
}
withResource(mergedNullsList) { _ =>
if (mergedNullsList.getRowCount == mergedNullsList.getNullCount) {
return GpuColumnVector.columnVectorFromNull(listCol.getRowCount.toInt, dataType)
}
}
withResource(Scalar.fromInt(0)) { zero =>
if (start.contains(zero)) {
throw RapidsErrorUtils.unexpectedValueForStartInFunctionError(prettyName)
Expand Down
Loading