Integrate install_gpu_driver.sh into spark-rapids.sh to avoid duplicate CUDA/driver installation logic #1360
Description
The recent MR #1359 updates the Rocky instructions to use the .run file, which is not compatible with Rocky 9.
We should investigate whether spark-rapids.sh can reuse the existing install_gpu_driver.sh script, as it’s already robust and handles most cases across all supported operating systems. The goal is to avoid copying the CUDA and driver installation logic from install_gpu_driver.sh into spark-rapids.sh, since maintaining two separate implementations would lead to inconsistencies and higher maintenance overhead.
At the same time, we want to make the user experience simpler by allowing them to use a single init script instead of having to provide one for Spark and another for CUDA/driver installation. Ideally, spark-rapids.sh should invoke the existing install_gpu_driver.sh during initialization.
Some known limitations of install_gpu_driver.sh are discussed in #1269 (comment).