New
0.10.0
DataFusion Comet 0.10.0 Changelog
This release consists of 183 commits from 26 contributors. See credits at the end of this changelog for more information.
Fixed bugs:
Unclaimed project
Are you a maintainer of datafusion-comet? Claim this project to take control of your public changelog and roadmap.
Changelog
Apache DataFusion Comet Spark Accelerator
♞ lichess.org: the forever free, adless and open source chess server ♞
💾 Database Tools incl. ORM, Migrations and Admin UI (Postgres, MySQL & MongoDB) [deprecated]
The Community Maintained High Velocity Web Framework For Java and Scala.
CMAK is a tool for managing Apache Kafka clusters
This release consists of 183 commits from 26 contributors. See credits at the end of this changelog for more information.
Fixed bugs:
spark.comet.batchSize instead of conf.arrowMaxRecordsPerBatch for data that is coming from Java #2196 (rluvaton)auto scan mode with Spark 4.0.0 #1975 (andygrove)CometScanRule #2252 (andygrove)native_comet for encrypted Parquet scans #2250 (andygrove)native_comet when object store not supported by native_iceberg_compat #2251 (andygrove)Performance related:
Implemented enhancements:
map_entries #2059 (comphead)Documentation updates:
sphinx-reredirects for redirects #2324 (andygrove)Other:
CopyExec with maintains_input_order, supports_limit_pushdown and cardinality_effect #1979 (rluvaton)QueryPlanSerde to CometExecRule #2015 (andygrove)IntegralDivide #2054 (coderfender)chr function from datafusion-spark #2080 (andygrove)stringDecode() to CommonStringExprs trait #2111 (peter-toth)CometExpressionSerde #2114 (peter-toth)QueryPlanSerde merge error #2127 (comphead)CometTestBase to stop setting the scan implementation to native_comet #2176 (andygrove)createPlan #2180 (andygrove)auto scan instead of native_comet #2178 (andygrove)CometBatchIterator and fix a potential issue #2168 (andygrove)array_intersect test #2246 (comphead)strict-warning profile for Scala #2254 (comphead)native_iceberg_compat #2253 (comphead)Cast serde to avoid code duplication #2242 (andygrove)hex/unhex SerDe to avoid code duplication #2287 (hsiang-c)arrow-rs #2279 (hsiang-c)alias and attributeReference #2290 (andygrove)Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.
75 Andy Grove
27 dependabot[bot]
11 Oleks V
9 Zhen Wang
7 hsiang-c
5 Artem Kupchinskiy
5 B Vadlamani
5 Kazantsev Maksim
5 Matt Butrovich
5 Parth Chandra
4 Rishab Joshi
3 Peter Toth
3 Tal Glanzman
2 Dharan Aditya
2 Huaxin Gao
2 KAZUYUKI TANIMURA
2 Leung Ming
2 Raz Luvaton
2 Yu-Chuan Hung
1 Asura7969
1 Emily Matheys
1 K.I. (Dennis) Jung
1 Kristin Cowalcijk
1 Peter Nguyen
1 codetyri0n
1 Šimon Lukašík
Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.