Add support for exchange spooling on Alluxio in FTE

[hqbhoho]

Description

Support exchange spooling on Alluxio

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## General
* Support exchange spooling on Alluxio when fault-tolerant execution is enabled.

[losipiuk]

Why do we need this one? We do not have it for io.trino.filesystem.alluxio.
If we need to be able to change some specific Alluxio config parameters it is better to expose those as specific getters/setters in ExchangeAlluxioConfig

[hqbhoho]

Thank you for your feedback!

In Alluxio filesystem doc, the Alluxio cluster connection is configured in the alluxio-site.properties properties file. The same config file must be located in /opt/alluxio/conf on all Trino cluster nodes.

Since the Alluxio client automatically loads configurations from /opt/alluxio/conf/alluxio-site.properties, a custom configuration is likely necessary for exchange spooling, as its requirements may differ from those of the Alluxio filesystem used for the catalog. So if Alluixo is used for exchange spooling, should we avoid loading the configuration from /opt/alluxio/conf/alluxio-site.properties which is only used for catalog?

This configuration is similar to hdfs.config.resources which would be used if HDFS serves as the exchange storage.

​​Due to the vast number of configuration settings available in Alluxio, It seems that using configuration files would be more convenient.

[losipiuk]

Yeah. It makes sense. Thanks

[losipiuk]

This looks good.
I am a bit concerned about using syncronous APIs for implementing asynchronous SPI but let's see how it works in practice.

Would you be fine adding some minimal documentation to fault-tolerant-execution.md?

[hqbhoho]

This looks good. I am a bit concerned about using syncronous APIs for implementing asynchronous SPI but let's see how it works in practice.

Would you be fine adding some minimal documentation to fault-tolerant-execution.md?

Currently, I use Alluxio as exchange storage by HDFS client, yet the P99 query latency decreased by 1/3 compared to using HDFS.

[losipiuk]

Thanks

[raunaqmorarka]

Now that this is supported natively, do we still need exchange.hdfs.skip-directory-scheme-validation ?
Since using that requires dropping jars into Trino, that's a more error prone way of doing the same thing

[hqbhoho]

Now that this is supported natively, do we still need exchange.hdfs.skip-directory-scheme-validation ? Since using that requires dropping jars into Trino, that's a more error prone way of doing the same thing

The configuration exchange.hdfs.skip-directory-scheme-validation is used to ​​enable the use of Hadoop-Compatible File Systems with the HDFS client​​.
Not only for Alluxio, it enables users with more choices. In that case, I think it should be retained.