Skip to content

Support array_union scalar expr#1362

Closed
dharanad wants to merge 2 commits intoapache:mainfrom
dharanad:dharanad/feat/support-array_union
Closed

Support array_union scalar expr#1362
dharanad wants to merge 2 commits intoapache:mainfrom
dharanad:dharanad/feat/support-array_union

Conversation

@dharanad
Copy link
Copy Markdown
Contributor

@dharanad dharanad commented Jan 31, 2025

Which issue does this PR close?

Part of #1042

Rationale for this change

What changes are included in this PR?

How are these changes tested?

Signed-off-by: Dharan Aditya <[email protected]>
@dharanad dharanad marked this pull request as ready for review January 31, 2025 13:42
@dharanad dharanad marked this pull request as draft January 31, 2025 15:56
@dharanad
Copy link
Copy Markdown
Contributor Author

dharanad commented Jan 31, 2025

There is this one case which i haven't added.

          checkSparkAnswerAndOperator(
            sql("SELECT array_union(array(_2 * 1), array(_9, _10)) from t1"))

Because it was failing due to mismatch in ordering between spark and df
Example

  == Results ==
  !== Correct Answer - 1000 ==                                        == Spark Answer - 1000 ==
   struct<array_union(array((_2 * -1)), array(_9, _10)):array<int>>   struct<array_union(array((_2 * -1)), array(_9, _10)):array<int>>
  ![ArraySeq(128, -128, -896)]                                        [ArraySeq(128, -896, -128)]

I am not sure how to deal with this, open to suggestions

Signed-off-by: Dharan Aditya <[email protected]>
@andygrove
Copy link
Copy Markdown
Member

I'm closing this issue because it has been stale for a while. @dharanad feel free to reopen this if you resume work on this featrure.

@andygrove andygrove closed this Jun 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants