parquet4s

Improvements:

• partitions and files are now sorted alphanumerically while reading a directory
• loosens schema validation when reading files without projections - when a field of the case class is optional (nullable or Option) and the corresponding column is missing in the source file then Parquet4s no longer throws an exception but sets the field as empty (null or None)

**Minor upgrad...

Improvements and bugfixes:

• Parquet4s no longer sets partition values in read records if the value is not present in projection schema. This removes redundant operations, which could highly affect computational and memory efficiency of the library in certain scenarios.
• Improved schema resolution of case classes. Removes repeated resolution of nested case classes, which could lead to sit...

This release introduces several substantial changes:

1. Decimal Formats and Scaling Enhancements: Users of Parquet4s can now choose from multiple formats for decimal values. In addition to the default binary format, you can use int and long columns. Furthermore, you can customize the scale and precision of decimals. When reading decimals, you have the option to rescale the original sou...

New features:

• New filters isNull and isNotNull to filter by missing values.
• custom function added to FS2's viaParquet to use custom Parquet builders, e.g. to write a stream of Avro files to Parquet.

Improvements and bug fixes:

• Improved file rotation by maxDuration in FS2's viaParquet by giving it a higher priority than regular writes. Thanks to that, rotation by time hap...

The release introduces several changes to viaParquet:

• ability to define a default partition value so that you can partition your data even when the partition column is nullable
• introduced a custom builder for viaParquet in Akka / Pekko so that you can stream any document format (supported by https://github.com/apache/parquet-java, e.g. Avro) and partition it.

Moreover, to keep compati...