Add Iceberg integration

[andygrove]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
There is a PR to add Ballista integration to iceberg-rust apache/iceberg-rust#2613 and there is a discussion about moving it to a new location.

Describe the solution you'd like
Let's discuss bringing this into Ballista repo.

Describe alternatives you've considered

Additional context

[andygrove]

@milenkovicm I was thinking we could add a separate crate (not part of main workspace) and have independent release cycle so that it doesn't impact current Ballista release cycle

[milenkovicm]

Same like python support, good idea

[NoahKusaba]

Sounds good to me. I can close my PR on iceberg-rust and re-open it here if that works with everyone?
We can merge it in once I get the necessary iceberg-datafusion changes in.
** Given that the changes are tightly coupled to the Iceberg-Datafusion changes, I'm going to migrate the PR once those get merged in as it is more convenient.

[martin-g]

+1

[sandugood]

+1
Would like to contribute and then test Ballista on my current workloads :)

[phillipleblanc]

This makes sense to me!

[NoahKusaba]

Opened the PR at
#2217
It's slow going getting the iceberg-datafusion changes merged in, so currently it's pointed to my iceberg-rust branch.