Skip to content

v2.21.0

Latest
Compare
Choose a tag to compare
@mjakubowski84 mjakubowski84 released this 16 Jan 08:55
· 1 commit to master since this release

This release introduces several substantial changes:

  1. Decimal Formats and Scaling Enhancements:
    Users of Parquet4s can now choose from multiple formats for decimal values. In addition to the default binary format, you can use int and long columns. Furthermore, you can customize the scale and precision of decimals. When reading decimals, you have the option to rescale the original source format or retain it as stored in the Parquet file. Please refer to the documentation for further details.
  2. Simplified Filtering Type Classes:
    The filtering type classes have undergone significant simplification. The FilterDecoder and, consequently, the FilterCodec are no longer required to define custom filters. From now on, only the FilterEncoder is needed.
    Over the course of Parquet4s's evolution, the UDP class remained the last feature requiring filter decoding. To simplify the filtering mechanism and eliminate this last dependency, UDP must now rely on Java column types instead of Scala ones. This is a breaking change, but it is a necessary tradeoff to enhance the overall simplicity of the library. Please refer to the documentation for more details.
  3. Bugfixes:
  • Rescaling of decimal data during binary operations is no longer performed when reading. This change prevents buffer overflow errors that occurred when decimals exceeded the default size of the binary schema.
  1. Notable dependency updates:
  • parquet-hadoop upgraded to 1.15.0
  • hadoop-client (provided) to 3.4.1
  • pekko to 1.1.3
  • cats effect to 3.5.7