In xarray SQL, I figured out a good interface between Python and Rust via Arrow. Check this out: xqlsystems/xarray-sql#100
TL;DR: we can pass in a Python callback into Rust that returns an PyCapsule of an arrow stream ABI.
Why is this interesting? It means I believe that we could treat Python objects, namely, Xarray Datasets, as a "streaming" file format.
This means that in theory, we could join data served via tile servers, like Xee, into this database library.
That would be pretty cool.
In xarray SQL, I figured out a good interface between Python and Rust via Arrow. Check this out: xqlsystems/xarray-sql#100
TL;DR: we can pass in a Python callback into Rust that returns an PyCapsule of an arrow stream ABI.
Why is this interesting? It means I believe that we could treat Python objects, namely, Xarray Datasets, as a "streaming" file format.
This means that in theory, we could join data served via tile servers, like Xee, into this database library.
That would be pretty cool.