Skip to content

Conversation

@kronnk
Copy link

@kronnk kronnk commented Jul 17, 2024

Pull Request Title: Implement compression

Overview

This pull request implements following compression options ["none", "gzip", "snappy","brotli","lz4", "zstd"] for writing files to Aws S3

Details

  • added ["none", "gzip", "snappy","brotli","lz4", "zstd"] compression options
  • fixed filename for parquet files to avoid wrong filenames ex("parquet.gzip.parquet")

Feedback & Discussion

Feel free to leave comments or ask questions below. Your feedback is greatly appreciated!

@Ehsan-Mirzaei
Copy link

can this PR get merge please @kronnk
it's a really good feature and everyone will need that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants