[WIP] feat: Prefer to columnar

[wForget]

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

How are these changes tested?

[codecov-commenter]

Codecov Report

❌ Patch coverage is 46.42857% with 90 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.26%. Comparing base (f09f8af) to head (8fb241b).
⚠️ Report is 544 commits behind head on main.

Files with missing lines	Patch %	Lines
...n/scala/org/apache/comet/rules/CometExecRule.scala	45.45%	69 Missing and 15 partials ⚠️
.../scala/org/apache/spark/sql/comet/util/Utils.scala	0.00%	5 Missing ⚠️
...n/scala/org/apache/comet/ExtendedExplainInfo.scala	0.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##               main    #2444       +/-   ##
=============================================
- Coverage     56.12%   42.26%   -13.87%     
- Complexity      976     1120      +144     
=============================================
  Files           119      147       +28     
  Lines         11743    13572     +1829     
  Branches       2251     2357      +106     
=============================================
- Hits           6591     5736      -855     
- Misses         4012     6858     +2846     
+ Partials       1140      978      -162     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mbutrovich]

So is the high level here to prefer ColumnarToColumnar instead of RowToColumnar? If so, I was looking at improving the performance of ArrowWriter and might have a follow up for this. Current ColumnarToColumnar (which I worked on a year ago so I don't feel bad saying this) is quite naive.

[wForget]

So is the high level here to prefer ColumnarToColumnar instead of RowToColumnar?

Thanks for pointing this out. I’m guessing it might prefer RowToColumnar, since the fallback spark operators may not support columnar. I’ll add some logging to verify it.