This release introduces several substantial changes:
- Decimal Formats and Scaling Enhancements:
Users of Parquet4s can now choose from multiple formats for decimal values. In addition to the default binary format, you can useint
andlong
columns. Furthermore, you can customize the scale and precision of decimals. When reading decimals, you have the option to rescale the original source format or retain it as stored in the Parquet file. Please refer to the documentation for further details. - Simplified Filtering Type Classes:
The filtering type classes have undergone significant simplification. TheFilterDecoder
and, consequently, theFilterCodec
are no longer required to define custom filters. From now on, only theFilterEncoder
is needed.
Over the course of Parquet4s's evolution, the UDP class remained the last feature requiring filter decoding. To simplify the filtering mechanism and eliminate this last dependency, UDP must now rely on Java column types instead of Scala ones. This is a breaking change, but it is a necessary tradeoff to enhance the overall simplicity of the library. Please refer to the documentation for more details. - Bugfixes:
- Rescaling of decimal data during binary operations is no longer performed when reading. This change prevents buffer overflow errors that occurred when decimals exceeded the default size of the binary schema.
- Notable dependency updates:
- parquet-hadoop upgraded to 1.15.0
- hadoop-client (provided) to 3.4.1
- pekko to 1.1.3
- cats effect to 3.5.7