-
Notifications
You must be signed in to change notification settings - Fork 211
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Change default .arrow
compression to "uncompressed"
#656
Conversation
Fixes #655 Adds support for overriding any defaults on a per-`Spec` basis. This may be helpful if we later run into issues with vega/vega#3961 **The file size of `flights-200k.arrow` has a 7.2x increase.** However, this is still roughly half the size of `flights-200k.json`.
I'm not sure how to reproduce this check on the buffer you have in I think this PR should resolve the issue. |
I can confirm that the file parses successfully with both Arrow-JS and Flechette. However, the internal format has changed, which may affect downstream use:
I would think that the simplest fix would be to revert to how the file was originally constructed. Why make changes if they are not needed? |
Thanks for testing it out @jheer I'm going to take a deeper look at the previous version to try and more faithfully recreate it. I think the datatypes should be easy to fix |
Fixes #655
The file size of
flights-200k.arrow
has a7.2x2.4x increase.However, this is still
roughly half0.16x the size offlights-200k.json
.Also, adds support for overriding any
write_options
on a per-Spec
basis.This may be helpful if we later run into issues with vega/vega#3961