Unshimmify Arm helper methods to expose as public API#13424
Conversation
|
build |
| pattern are exposed via the public Spark RAPIDS API under the `Arm` object: | ||
|
|
||
| ```scala | ||
| import com.nvidia.spark.rapids.Arm.{withResource, closeOnExcept} |
There was a problem hiding this comment.
If we show both we should explain the difference and when to use one over the other
| intermediate data generated while producing the final result that is | ||
| returned. | ||
|
|
||
| For Scala, helper methods implementing the automatic resource management |
There was a problem hiding this comment.
It is worth pointing out that this originated as a solution for Scala 2.12- before Using was introduced in Scala 2.13. Thus, with 2.13 builds of Apache Spark it is an alternative as well.
sameerz
added
the
task
|
build |
2 similar comments
|
build |
|
build |
|
build |
1 similar comment
|
build |
|
Putting it in draft since more refactoring work is needed |
|
NOTE: release/25.12 has been created from main. Please retarget your PR to release/25.12 if it should be included in the release. |
What refactoring work is needed? |
Arm depends on RapidsPluginImplicits which in turn depends on shim specific Spark code |
|
NOTE: release/26.02 has been created from main. Please retarget your PR to release/26.02 if it should be included in the release. |
|
NOTE: release/26.04 has been created from main. Please retarget your PR to release/26.04 if it should be included in the release. |
|
NOTE: release/26.06 has been created from main. Please retarget your PR to release/26.06 if it should be included in the release. |
|
We can use the methodology from #14834 to focus on Arm |
5 participants
Fixes #13423.
Description
Unshimmifies Arm (and dependency CloseableHolder) to expose publicly as part of the sql-plugin. This makes it importable e.g. via
import com.nvidia.spark.rapids.Arm.{withResource, closeOnExcept}.Checklists
(Please explain in the PR description how the new code paths are tested, such as names of the new/existing tests that cover them.)