[VL] Support function ToPrettyString#11566
Conversation
|
Run Gluten Clickhouse CI on x86 |
rui-mo
left a comment
There was a problem hiding this comment.
Would you resolve the below failure?
Error: /__w/incubator-gluten/incubator-gluten/gluten-substrait/src/main/scala/org/apache/gluten/expression/ExpressionMappings.scala:110: error: not found: type ToPrettyString
Error: Sig[ToPrettyString](TO_PRETTY_STRING),
|
Run Gluten Clickhouse CI on x86 |
|
The version spark 3.5 works well, but spark4.0 and Spark 4.1 introduces |
|
Run Gluten Clickhouse CI on x86 |
2 similar comments
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
91095e9 to
09ea9d1
Compare
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Could you help approve this PR? Thanks! @rui-mo |
|
Thanks |
rui-mo
left a comment
There was a problem hiding this comment.
It would be better to add a test to ensure this function could be offloaded by verifying the plan.
|
I try to add a unit test +-----------+ plan is VeloxColumnarToRow But the plan does not contain the to_pretty_string, so it is hard to verify if it is really offload to native unless we can check the native plan. But ToPrettyString is really offloaded since the unit test getRows::binary failed with this PR |
The expression is available since Spark 3.5. Prior versions use CAST for binary output.
Spark 4.0 introduces the optional config spark.sql.binaryOutputStyle.
This PR supports only the default behavior (HEX_DISCRETE), which is used when the config is unset.