Apache Iceberg 1.9.0

## What's Changed * Build: Bump software.amazon.awssdk:bom from 2.30.11 to 2.30.16 by @dependabot in https://github.com/apache/iceberg/pull/12208 * Build: Bump com.google.cloud:libraries-bom from 26.53.0 to 26.54.0 by @dependabot in https://github.com/apache/iceberg/pull/12207 * Build: Bump org.xerial:sqlite-jdbc from 3.48.0.0 to 3.49.0.0 by @dependabot in https://github.com/apache/iceberg/pull/12206 * Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.4.1 to 5.4.2 by @dependabot in https://github.com/apache/iceberg/pull/12209 * Docs: Fix expire_snapshots output by @manuzhang in https://github.com/apache/iceberg/pull/12213 * Update LICENSE/NOTICE files with last dependency updates by @jbonofre in https://github.com/apache/iceberg/pull/12214 * Doc: update note about notebook in spark-quickstart by @xxchan in https://github.com/apache/iceberg/pull/11996 * Spec: Support geo type by @szehon-ho in https://github.com/apache/iceberg/pull/10981 * Build: skip scheduled docker image publish workflows on forks by @dramaticlly in https://github.com/apache/iceberg/pull/12218 * Docs: Add missing types to the spec v3 summary by @wgtmac in https://github.com/apache/iceberg/pull/12219 * Spec: Typo - missing be by @RussellSpitzer in https://github.com/apache/iceberg/pull/12229 * Docs: Site update for 1.8.0 release by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12242 * spec: Remove `source-ids` for `V{1,2}` tables by @Fokko in https://github.com/apache/iceberg/pull/12161 * Build: Update revAPI to compare against 1.8.0 by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12244 * Update release version to 1.8.0 in doap.rdf by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12247 * Infra: Update Iceberg bug report template for 1.8.0 by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12248 * Docs: Fix formatting of 1.8.0 release notes by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12249 * Core: Adjust Jackson settings to handle large metadata json by @bryanck in https://github.com/apache/iceberg/pull/12224 * Build: Bump Hive to 2.3.10 by @nastra in https://github.com/apache/iceberg/pull/12253 * Build: Clean up dependencies by @nastra in https://github.com/apache/iceberg/pull/12252 * Core: Add InternalData read and write builders by @rdblue in https://github.com/apache/iceberg/pull/12060 * API, Core: Support default values in UpdateSchema by @rdblue in https://github.com/apache/iceberg/pull/12211 * update site to include iceberg summit link by @danicafine in https://github.com/apache/iceberg/pull/12256 * Core: Fix divide by zero when adjust split size by @hantangwangd in https://github.com/apache/iceberg/pull/12201 * Spark: Remove unused PruneColumnsWithReordering class by @rdblue in https://github.com/apache/iceberg/pull/12258 * Spark: Fix assertion checks by @nastra in https://github.com/apache/iceberg/pull/12255 * API: Deprecate NestedType.of in favor of builder by @rdblue in https://github.com/apache/iceberg/pull/12227 * Docker: Pin QEMU version temporarily by @ebyhr in https://github.com/apache/iceberg/pull/12262 * OpenAPI: Add RemoveSchemas REST update type by @gaborkaszab in https://github.com/apache/iceberg/pull/12022 * Minor: update Learn More to point to spark quickstart by @danicafine in https://github.com/apache/iceberg/pull/12272 * Build: Bump datamodel-code-generator from 0.27.2 to 0.28.1 by @manuzhang in https://github.com/apache/iceberg/pull/12290 * Spark 3.5: Fix job description of RewriteTablePathSparkAction by @ebyhr in https://github.com/apache/iceberg/pull/12282 * Build: Bump io.netty:netty-buffer from 4.1.117.Final to 4.1.118.Final by @dependabot in https://github.com/apache/iceberg/pull/12287 * Build: Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.21 by @dependabot in https://github.com/apache/iceberg/pull/12286 * OpenAPI: Add overwrite option when registering an iceberg table by @dramaticlly in https://github.com/apache/iceberg/pull/12239 * Build: Bump mkdocs-material from 9.6.3 to 9.6.4 by @dependabot in https://github.com/apache/iceberg/pull/12284 * Core: Fix non-setting row-lineage from table properties on initial table creation by @tomtongue in https://github.com/apache/iceberg/pull/12307 * API: Reject unknown type for required fields and validate defaults by @rdblue in https://github.com/apache/iceberg/pull/12302 * API: Fix TestInclusiveMetricsEvaluator notStartsWith tests by @rdblue in https://github.com/apache/iceberg/pull/12303 * Core: add variant type support by @aihuaxu in https://github.com/apache/iceberg/pull/11831 * Fix CI: Update tests with `UnknownType` from `Required` to `Optional` by @Fokko in https://github.com/apache/iceberg/pull/12316 * Docs: Refactor site navigation bar by @manuzhang in https://github.com/apache/iceberg/pull/12289 * Parquet: Implement Variant readers by @rdblue in https://github.com/apache/iceberg/pull/12139 * Docs: Add rewrite-table-path in spark procedure by @dramaticlly in https://github.com/apache/iceberg/pull/12115 * Parquet: Fix errorprone warning by @ebyhr in https://github.com/apache/iceberg/pull/12324 * Docs: add apache amoro(incubating) with iceberg (#11965) by @czy006 in https://github.com/apache/iceberg/pull/11966 * Parquet: Fix performance regression in reader init by @bryanck in https://github.com/apache/iceberg/pull/12305 * Core: Fallback to GET requests for namespace/table/view exists checks by @nastra in https://github.com/apache/iceberg/pull/12314 * Fix: fix apache amoro ams doc pic ref by @czy006 in https://github.com/apache/iceberg/pull/12332 * Revert "Core: Serialize `null` when there is no current snapshot" by @Fokko in https://github.com/apache/iceberg/pull/12312 * [1.8.x] Parquet: Fix performance regression in reader init (#12305) by @nastra in https://github.com/apache/iceberg/pull/12329 * Checkstyle: Apply the same generic type naming rules to interfaces and classes by @pvary in https://github.com/apache/iceberg/pull/12333 * Kafka: Pin Kafka-Connect version to fix integration tests by @nastra in https://github.com/apache/iceberg/pull/12340 * Docs: Fix link of catalog in terms.md by @wangyinsheng in https://github.com/apache/iceberg/pull/12326 * Docs: Add documentation for Rate limiting in Spark Structured Streaming by @singhpk234 in https://github.com/apache/iceberg/pull/12217 * Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure by @RussellSpitzer in https://github.com/apache/iceberg/pull/12319 * Parquet: Remove deprecated VectorizedReader.setRowGroupInfo and ParquetValueReader.setPageSource by @ebyhr in https://github.com/apache/iceberg/pull/12321 * Bump versions in `{LICENSE,NOTICE}` by @Fokko in https://github.com/apache/iceberg/pull/12337 * Spark: Remove Spark 3.3 support by @manuzhang in https://github.com/apache/iceberg/pull/12279 * Core: Remove deprecated Util.blockLocations method and StructCopy class by @ebyhr in https://github.com/apache/iceberg/pull/12320 * Core: Handle partition evolution case in PartitionStatsUtil#computeStats by @deniskuzZ in https://github.com/apache/iceberg/pull/12137 * Core, Spark: Remove deprecated code for 1.9.0 by @nastra in https://github.com/apache/iceberg/pull/12336 * API: Move variant to API and add extract expression by @rdblue in https://github.com/apache/iceberg/pull/12304 * Core: Remove namespace/table/view HEAD endpoints from defaults by @nastra in https://github.com/apache/iceberg/pull/12351 * Nit: Remove additional 'Iceberg' in Puffin footer payload by @tomtongue in https://github.com/apache/iceberg/pull/12369 * API: Move Variant interfaces and serialized implementations to API by @rdblue in https://github.com/apache/iceberg/pull/12374 * Flink: Fix the comment error in SketchDataStatistics by @Guosmilesmile in https://github.com/apache/iceberg/pull/12375 * Core: Don't remove trailing slash from absolute paths by @adutra in https://github.com/apache/iceberg/pull/12389 * Build: Bump mkdocs-material from 9.6.4 to 9.6.5 by @dependabot in https://github.com/apache/iceberg/pull/12386 * Build: Bump org.xerial:sqlite-jdbc from 3.49.0.0 to 3.49.1.0 by @dependabot in https://github.com/apache/iceberg/pull/12385 * Build: Bump org.awaitility:awaitility from 4.2.2 to 4.3.0 by @dependabot in https://github.com/apache/iceberg/pull/12384 * Build: Bump nessie from 0.102.5 to 0.103.0 by @dependabot in https://github.com/apache/iceberg/pull/12383 * Build: Bump com.google.cloud:libraries-bom from 26.54.0 to 26.55.0 by @dependabot in https://github.com/apache/iceberg/pull/12382 * Build: Bump testcontainers from 1.20.4 to 1.20.5 by @dependabot in https://github.com/apache/iceberg/pull/12380 * Build: Bump software.amazon.awssdk:bom from 2.30.21 to 2.30.26 by @dependabot in https://github.com/apache/iceberg/pull/12379 * Spec: Allow Equality Deletes with Row Lineage and Define Behavior by @RussellSpitzer in https://github.com/apache/iceberg/pull/12230 * Core: Add "volatile" to HadoopFileIO#hadoopConf by @okumin in https://github.com/apache/iceberg/pull/12388 * Arrow, Parquet, Spark 3.5, Flink 1.20: Avoid deprecated method by @ebyhr in https://github.com/apache/iceberg/pull/11874 * Docs: Add Stackable to the Vendors page by @lfrancke in https://github.com/apache/iceberg/pull/12344 * Build: Upgrade to Gradle 8.13 by @jbonofre in https://github.com/apache/iceberg/pull/12398 * Kafka Connect: Add SMTs for Debezium and AWS DMS by @ismailsimsek in https://github.com/apache/iceberg/pull/11936 * API, Core: Update inclusive metrics evaluator for extract and transforms by @rdblue in https://github.com/apache/iceberg/pull/12311 * Build: remove Hadoop 2 dependency by @Kontinuation in https://github.com/apache/iceberg/pull/12348 * View Spec: Fix typo by @ajantha-bhat in https://github.com/apache/iceberg/pull/12405 * Fix grammar issues in descriptions about Hive environment in hive-quickstart.md by @wenxin-cn in https://github.com/apache/iceberg/pull/12402 * [hive]:Fix Hive table creation syntax errors by @qining-mj in https://github.com/apache/iceberg/pull/12394 * Core: Interface changes for separating rewrite planner and runner by @pvary in https://github.com/apache/iceberg/pull/12306 * Docs: Describe how to handle versioned docs/javadoc during a release by @nastra in https://github.com/apache/iceberg/pull/12413 * Spark: Bump Spark 3.5 to 3.5.5 by @manuzhang in https://github.com/apache/iceberg/pull/12396 * Docs: Remove Hive runtime jar link from latest release by @manuzhang in https://github.com/apache/iceberg/pull/12422 * Docs: Site updates for 1.8.1 by @nastra in https://github.com/apache/iceberg/pull/12410 * Infra: Update Bug report template for 1.8.1 by @nastra in https://github.com/apache/iceberg/pull/12409 * Update release version to 1.8.1 in doap.rdf by @nastra in https://github.com/apache/iceberg/pull/12408 * Build: Ignore docker folder in CI by @manuzhang in https://github.com/apache/iceberg/pull/12417 * Fix IndexOutOfBounds exception in FileFormat#fromFileName by @rshkv in https://github.com/apache/iceberg/pull/12301 * Core: Print un-pretty metadata files without whitespace by @istreeter in https://github.com/apache/iceberg/pull/12318 * Core: Code cleanup around TestTable and TestTableOperations by @gaborkaszab in https://github.com/apache/iceberg/pull/12419 * Data: Add partition stats writer and reader by @ajantha-bhat in https://github.com/apache/iceberg/pull/11216 * Azure: Move docker-based tests to integrationTest by @anuragmantri in https://github.com/apache/iceberg/pull/12274 * spec: fix the wrong example in Geo Types by @jiayuasu in https://github.com/apache/iceberg/pull/12421 * Docs: Fix link of ndv in spark-procedures.md by @wangyinsheng in https://github.com/apache/iceberg/pull/12425 * Build: Bump mkdocs-material from 9.6.5 to 9.6.6 by @dependabot in https://github.com/apache/iceberg/pull/12432 * Build: Ignore README.md/LICENSE/NOTICE in all paths on CI by @manuzhang in https://github.com/apache/iceberg/pull/12429 * Build: Bump org.openapitools:openapi-generator-gradle-plugin from 7.11.0 to 7.12.0 by @dependabot in https://github.com/apache/iceberg/pull/12435 * Build: Bump datamodel-code-generator from 0.28.1 to 0.28.2 by @dependabot in https://github.com/apache/iceberg/pull/12433 * Build: Bump org.mongodb:bson from 4.11.0 to 4.11.5 by @dependabot in https://github.com/apache/iceberg/pull/12438 * Build: Bump slf4j from 2.0.16 to 2.0.17 by @dependabot in https://github.com/apache/iceberg/pull/12436 * Build: Bump jackson-bom from 2.18.2 to 2.18.3 by @dependabot in https://github.com/apache/iceberg/pull/12434 * Fix typo in delete statement: change database name from "dbl" to "db" in spark-writes.md by @wenxin-cn in https://github.com/apache/iceberg/pull/12426 * Build: Bump io.netty:netty-buffer from 4.1.118.Final to 4.1.119.Final by @dependabot in https://github.com/apache/iceberg/pull/12440 * Build: Bump net.snowflake:snowflake-jdbc from 3.22.0 to 3.23.0 by @dependabot in https://github.com/apache/iceberg/pull/12437 * Docs: Deprecate data_file.distinct_counts in v3 by @jacobmarble in https://github.com/apache/iceberg/pull/12182 * Core: Change RemoveSnapshots to remove unused schemas by @gaborkaszab in https://github.com/apache/iceberg/pull/12089 * Core: Add Variant logical type for Avro by @aihuaxu in https://github.com/apache/iceberg/pull/12238 * Spark: Infer partition spec in ADD_FILES procedure for FileTables than taking latest table spec by @bharos in https://github.com/apache/iceberg/pull/12327 * Avro: Support timestamp(9) and unknown types by @rdblue in https://github.com/apache/iceberg/pull/12455 * Wrap variant in PrimitiveHoder so serialization can result same instance by @aihuaxu in https://github.com/apache/iceberg/pull/12317 * Core: Ensure current and newly added view versions are retained in ViewMetadata build by @lliangyu-lin in https://github.com/apache/iceberg/pull/12401 * support create table like in flink catalog by @swapna267 in https://github.com/apache/iceberg/pull/12199 * Parquet: Implement Variant writers by @rdblue in https://github.com/apache/iceberg/pull/12323 * Core: Write `null` for `current-snapshot-id` for V3+ by @Fokko in https://github.com/apache/iceberg/pull/12335 * Spec: Add implementation note on `current-snapshot-id` by @Fokko in https://github.com/apache/iceberg/pull/12334 * Core: Don't create empty RemovePartitionSpecs MetadataUpdate by @gaborkaszab in https://github.com/apache/iceberg/pull/12465 * Data: Expose snapshot-id instead of branch for computing partition stats by @ajantha-bhat in https://github.com/apache/iceberg/pull/12464 * Parquet: Support unknown and timestamp(9) in generics and internal model by @rdblue in https://github.com/apache/iceberg/pull/12463 * Core: Provide access to format-version of metadata table by @nastra in https://github.com/apache/iceberg/pull/12462 * Site: Fix Footer Link by @RussellSpitzer in https://github.com/apache/iceberg/pull/12478 * Docs: fix typo in rest-catalog-open-api.yaml. by @slfan1989 in https://github.com/apache/iceberg/pull/12480 * Build: Bump software.amazon.awssdk:bom from 2.30.26 to 2.30.31 by @dependabot in https://github.com/apache/iceberg/pull/12439 * Build: Bump mkdocs-material from 9.6.6 to 9.6.7 by @dependabot in https://github.com/apache/iceberg/pull/12483 * Build: Bump com.gorylenko.gradle-git-properties:gradle-git-properties from 2.4.2 to 2.5.0 by @dependabot in https://github.com/apache/iceberg/pull/12486 * Build: Rename versions.spark.hive3* to versions.spark3* by @pan3793 in https://github.com/apache/iceberg/pull/12489 * Build: Bump testcontainers from 1.20.5 to 1.20.6 by @dependabot in https://github.com/apache/iceberg/pull/12484 * Core: Apply correct metric configs in GenericAppenderFactory by @XBaith in https://github.com/apache/iceberg/pull/12366 * Add unit test for AddFilesProcedure to check invalid column in partition filter by @bharos in https://github.com/apache/iceberg/pull/12456 * Adds AWS to vendors page by @rbowen in https://github.com/apache/iceberg/pull/12468 * Spark: Rewrite V2 deletes to V3 DVs by @nastra in https://github.com/apache/iceberg/pull/12250 * Docs: Update Iceberg talks with recent Iceberg meetup sessions by @sida-shen in https://github.com/apache/iceberg/pull/12481 * AWS: Integrate S3 analytics accelerator library by @SanjayMarreddi in https://github.com/apache/iceberg/pull/12299 * AWS: Don't fetch credential from endpoint if properties contain a valid credential by @nastra in https://github.com/apache/iceberg/pull/12504 * Revert "OpenAPI: Handle NamespaceNotEmptyException when dropping a namespace" by @nastra in https://github.com/apache/iceberg/pull/12517 * AWS: Update S3 async client configurations and docs for analytics-accelerator-s3 by @SanjayMarreddi in https://github.com/apache/iceberg/pull/12503 * Make reporter() method public so that it can be accessed by Trino for BaseTable creation by @puchengy in https://github.com/apache/iceberg/pull/12519 * Migrate Spark 3.4 test base to JUnit5 by @tomtongue in https://github.com/apache/iceberg/pull/12501 * Core: Fix support for GenericManifestFile index projection by @danielcweeks in https://github.com/apache/iceberg/pull/12522 * Core: Set missing table-default property in RESTSessionCatalog by @ebyhr in https://github.com/apache/iceberg/pull/11646 * Core: Fix default and initial value handling on table creation by @pvary in https://github.com/apache/iceberg/pull/12520 * Core: Don't expose InMemoryViewOperations and RESTViewBuilder outside their visibility scope by @nastra in https://github.com/apache/iceberg/pull/12524 * Kafka Connect: Add config for transactional ID prefix by @thjaeckle in https://github.com/apache/iceberg/pull/11780 * Kafka Connect: Handle no coordinator and data loss in ICR mode by @kumarpritam863 in https://github.com/apache/iceberg/pull/12372 * Core: Use `buildKeepingLast` for table properties in REST table builder by @smaheshwar-pltr in https://github.com/apache/iceberg/pull/12526 * Flink 1.20: Support Avro and Parquet timestamp(9), unknown, and defaults by @rdblue in https://github.com/apache/iceberg/pull/12470 * Spark 3.4: Backport partition spec inference in spark ADD_FILES procedure by @bharos in https://github.com/apache/iceberg/pull/12508 * Flink 1.18, 1.19: Implement timestamp(9), unknown, and defaults by @rdblue in https://github.com/apache/iceberg/pull/12532 * Build: Bump mkdocs-material from 9.6.7 to 9.6.8 by @dependabot in https://github.com/apache/iceberg/pull/12542 * Build: Bump datamodel-code-generator from 0.28.2 to 0.28.4 by @dependabot in https://github.com/apache/iceberg/pull/12541 * Core: Add missing table-override property to REST catalog by @ebyhr in https://github.com/apache/iceberg/pull/12548 * Spark: Call configureTable in ScanTestBase to ensure proper table configuration by @drexler-sky in https://github.com/apache/iceberg/pull/12546 * AWS, Core, GCP: Auth Manager API enablement by @adutra in https://github.com/apache/iceberg/pull/12197 * Core: fix "Unclosed S3FileIO" warning in JdbcCatalog by @rcjverhoef in https://github.com/apache/iceberg/pull/12540 * Migrate Spark 3.4 TestBase related tests in spark and actions by @tomtongue in https://github.com/apache/iceberg/pull/12552 * API: Implement Variant#toString by @rdblue in https://github.com/apache/iceberg/pull/12531 * Avro: Add variant readers and writers by @rdblue in https://github.com/apache/iceberg/pull/12457 * Core: Add `view-override` catalog property by @ebyhr in https://github.com/apache/iceberg/pull/12534 * Kafka: Suppress warnings around java.util.Date usage / fix var names by @nastra in https://github.com/apache/iceberg/pull/12561 * REST: HTTPRequest.baseUri() should be nullable by @adutra in https://github.com/apache/iceberg/pull/12556 * Parquet, Core: Enable passing Variant tests by @rdblue in https://github.com/apache/iceberg/pull/12559 * Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled by @hsiang-c in https://github.com/apache/iceberg/pull/12511 * Core, Spark 3.5: Apply Ignore Residuals to Delete Filtering by @RussellSpitzer in https://github.com/apache/iceberg/pull/12479 * Core: Make totalRecordCount optional in PartitionStats by @ajantha-bhat in https://github.com/apache/iceberg/pull/12226 * Core: Replace withFailMessage() with as() by @nastra in https://github.com/apache/iceberg/pull/12570 * Spark: Improve assertions for better debuggability by @nastra in https://github.com/apache/iceberg/pull/12569 * Docs: Update statements mentioning Hive's alpha/beta versions by @okumin in https://github.com/apache/iceberg/pull/12430 * Infra: Update Bug report template for 1.7.2 by @nastra in https://github.com/apache/iceberg/pull/12574 * docs: update go impl status by @zeroshade in https://github.com/apache/iceberg/pull/12578 * Core: Use InternalData when reading manifests in FileCleanupStrategy by @nastra in https://github.com/apache/iceberg/pull/12575 * Docs: Site updates for 1.7.2 by @jbonofre in https://github.com/apache/iceberg/pull/12576 * ORC: Support timestamp(9), variant, and unknown in generics by @rdblue in https://github.com/apache/iceberg/pull/12567 * Flink: Support source watermark for flink sql windows by @swapna267 in https://github.com/apache/iceberg/pull/12191 * Spark: Detect dangling DVs properly by @nastra in https://github.com/apache/iceberg/pull/12270 * OpenAPI: Handle NamespaceNotEmptyException when dropping a namespace by @nastra in https://github.com/apache/iceberg/pull/12518 * Spark: Use correct statistics file in SparkScan::estimateStatistics(Snapshot) by @wypoon in https://github.com/apache/iceberg/pull/12482 * Flink: fix read config of connector.iceberg.max-allowed-planning-failures by @Guosmilesmile in https://github.com/apache/iceberg/pull/12585 * Flink: backport for fix read config of connector.iceberg.max-allowed-planning-failures to 1.18 and 1.19 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12589 * Core: Use InternalData with avro for readers. by @danielcweeks in https://github.com/apache/iceberg/pull/12476 * Core, Parquet, ORC: Fix missing data when writing unknown by @rdblue in https://github.com/apache/iceberg/pull/12581 * Core: Handle NamespaceNotEmptyException in NamespaceErrorHandler by @nastra in https://github.com/apache/iceberg/pull/12505 * Spark 3.4: Read DVs when reading from .position_deletes table / Include content offset/size by @nastra in https://github.com/apache/iceberg/pull/12599 * Migrate Spark 3.4 TestBase-related remaining tests in actions by @tomtongue in https://github.com/apache/iceberg/pull/12579 * Spark 3.4: Test metadata tables with format-version=v3 / add ExtensionsTestBase by @nastra in https://github.com/apache/iceberg/pull/12600 * Core: Bulk deletion in RemoveSnapshots by @gaborkaszab in https://github.com/apache/iceberg/pull/11837 * Spark: Backport Spark 3.5 DVs related part to Spark 3.4 by @tomtongue in https://github.com/apache/iceberg/pull/12603 * Parquet: Implement Variant metrics by @rdblue in https://github.com/apache/iceberg/pull/12496 * Spark 3.4: Rewrite data files with high delete ratio by @nastra in https://github.com/apache/iceberg/pull/12601 * Spark 3.4: Rewrite V2 deletes to V3 DVs / Detect dangling DVs properly by @nastra in https://github.com/apache/iceberg/pull/12606 * Build: Bump mkdocs-material from 9.6.8 to 9.6.9 by @dependabot in https://github.com/apache/iceberg/pull/12614 * Build: Bump nessie from 0.103.0 to 0.103.2 by @dependabot in https://github.com/apache/iceberg/pull/12615 * Spec: Geo spec simplifications by @szehon-ho in https://github.com/apache/iceberg/pull/12533 * Spark 3.5: Adjust repeated INFO logs to DEBUG in SparkWrite and SparkPositionDeltaWrite by @manuzhang in https://github.com/apache/iceberg/pull/12404 * Spark 3.4: Backport Spark actions changes in Spark rewrite_table_path procedure (#12006 #12172 #11929 #12282 #12569) by @slfan1989 in https://github.com/apache/iceberg/pull/12568 * Core: Add update event for rewrite manifests by @bryanck in https://github.com/apache/iceberg/pull/12627 * Build: Enforce error message check on Exception assertions by @lliangyu-lin in https://github.com/apache/iceberg/pull/12624 * Build: Bump com.google.errorprone:error_prone_annotations from 2.36.0 to 2.37.0 by @dependabot in https://github.com/apache/iceberg/pull/12622 * Core: Add commit metrics for rewriting manifests by @nastra in https://github.com/apache/iceberg/pull/12630 * Data: Refactor PartitionStatsHandler by @ajantha-bhat in https://github.com/apache/iceberg/pull/12550 * Spark 3.4: Propagate snapshot properties / Add max allowed failed commits by @nastra in https://github.com/apache/iceberg/pull/12632 * Docs: Fix lifecycle and versions in multi-engine-support by @manuzhang in https://github.com/apache/iceberg/pull/12370 * Build: Bump parquet from 1.15.0 to 1.15.1 by @dependabot in https://github.com/apache/iceberg/pull/12616 * Build: Bump calcite from 1.10.0 to 1.39.0 by @dependabot in https://github.com/apache/iceberg/pull/12617 * Added New Blog Post: Loading Data into Apache Iceberg by @SourabhEstuary in https://github.com/apache/iceberg/pull/12587 * Core: Add option to fallback to thread classloader by @bryanck in https://github.com/apache/iceberg/pull/12613 * Core: child HTTPClient should not close shared resources by @adutra in https://github.com/apache/iceberg/pull/12566 * API, Core: Add geometry and geography types support by @Kontinuation in https://github.com/apache/iceberg/pull/12346 * AWS: fix incorrect parent session when calling delegate auth manager by @adutra in https://github.com/apache/iceberg/pull/12582 * Docs: Update block spacing guideline in contribute.md by @ajantha-bhat in https://github.com/apache/iceberg/pull/12641 * Docs: Fix ASF sponsorship links by @manuzhang in https://github.com/apache/iceberg/pull/12646 * Spark 3.4: Migrate SparkRowLevelOperationsTestBase related tests to JUnit 5 by @tomtongue in https://github.com/apache/iceberg/pull/12656 * Spark 3.4 : Use correct statistics file in SparkScan::estimateStatistics(Snapshot) by @jeesou in https://github.com/apache/iceberg/pull/12647 * Core: Enhance TestRemoveSnapshots by @manuzhang in https://github.com/apache/iceberg/pull/12662 * update status page for pyiceberg as of 0.9.0 by @kevinjqliu in https://github.com/apache/iceberg/pull/12645 * Build: Bump jetty from 11.0.24 to 11.0.25 by @dependabot in https://github.com/apache/iceberg/pull/12618 * Docs: Fix Latest Iceberg Support version of Hive by @manuzhang in https://github.com/apache/iceberg/pull/12640 * Azure: Support vended credentials refresh in ADLSFileIO. by @ChaladiMohanVamsi in https://github.com/apache/iceberg/pull/11577 * Build: Revert AWS SDK from 2.30.31 to 2.29.52 by @ajantha-bhat in https://github.com/apache/iceberg/pull/12649 * AWS: Use assertThat instead of JUnit4 assertions by @nastra in https://github.com/apache/iceberg/pull/12668 * GCP: Use catalog endpoint as base when refreshing OAuth2 token by @nastra in https://github.com/apache/iceberg/pull/12638 * Core: Cleanup unit tests by @sullis in https://github.com/apache/iceberg/pull/12666 * Core: FileRewritePlanner implementation by @pvary in https://github.com/apache/iceberg/pull/12493 * Flink: backport support create table like in flink catalog by @swapna267 in https://github.com/apache/iceberg/pull/12679 * Doc: Update Instructions for rewrite_table_path. by @slfan1989 in https://github.com/apache/iceberg/pull/12628 * Spark, API: Enhance hashing efficiency by operating on raw UTF-8 bytes by @xiaoxuandev in https://github.com/apache/iceberg/pull/12657 * Build: Bump datamodel-code-generator from 0.28.4 to 0.28.5 by @dependabot in https://github.com/apache/iceberg/pull/12683 * Docs: Update link for User-Defined Tag Restrictions in AWS documentation by @XBaith in https://github.com/apache/iceberg/pull/12698 * Build: Bump com.palantir.gradle.gitversion:gradle-git-version from 3.1.0 to 3.2.0 by @dependabot in https://github.com/apache/iceberg/pull/12687 * Build: Bump com.google.cloud:libraries-bom from 26.55.0 to 26.58.0 by @dependabot in https://github.com/apache/iceberg/pull/12688 * AWS: Fix Catalog URI within VendedCredentialsProvider by @wolflex888 in https://github.com/apache/iceberg/pull/12612 * Core: Add MetricsReporter for SnapshotManager by @wangyinsheng in https://github.com/apache/iceberg/pull/12665 * Spark 3.5: Fix RewriteDataFiles with partial progress enabled and max-failed-commits larger than total-file-group by @manuzhang in https://github.com/apache/iceberg/pull/12120 * Use delimited column names in CreateChangelogViewProcedure by @andyglow in https://github.com/apache/iceberg/pull/12418 * Flink: Backport support source watermark for flink sql windows by @swapna267 in https://github.com/apache/iceberg/pull/12697 * Spark 3.4: Fix RewriteDataFiles with partial progress enabled and max-failed-commits larger than total-file-group by @manuzhang in https://github.com/apache/iceberg/pull/12701 * Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.4.2 to 5.4.3 by @dependabot in https://github.com/apache/iceberg/pull/12685 * Build: Bump guava from 33.4.0-jre to 33.4.6-jre by @dependabot in https://github.com/apache/iceberg/pull/12686 * Core: Pass storage credentials from LoadTableResponse to FileIO by @nastra in https://github.com/apache/iceberg/pull/12591 * AWS: Update the aws-bundle with latest dependencies by @SanjayMarreddi in https://github.com/apache/iceberg/pull/12553 * Core, Hive: Double check commit status in case of commit conflict for NoLock by @lirui-apache in https://github.com/apache/iceberg/pull/12637 * Spark 3.4: Fix NotSerializableException when migrating Spark tables by @manuzhang in https://github.com/apache/iceberg/pull/12705 * Doc: Remove Warning for resolved issue #11147. by @slfan1989 in https://github.com/apache/iceberg/pull/12694 * Flink: Backport avoid RANGE mode broken chain when write parallelism changes by @huyuanfeng2018 in https://github.com/apache/iceberg/pull/12080 * Updated vendors documentation to add SingleStore by @akoller in https://github.com/apache/iceberg/pull/12708 * Spec: update to reflect lineage is required by @danielcweeks in https://github.com/apache/iceberg/pull/12580 * Core: Enhance remove snapshots efficiency by executing them in bulk by @ricardopereira33 in https://github.com/apache/iceberg/pull/12670 * Core: Update deprecation msg by @nastra in https://github.com/apache/iceberg/pull/12720 * AWS: Add AWS integ tests to check task and enable tests based on required environment variables by @lliangyu-lin in https://github.com/apache/iceberg/pull/12671 * Core, Spark: Add row lineage metadata columns, and surface them in SparkTable metadata columns by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12596 * Build: Bump mkdocs-material from 9.6.9 to 9.6.11 by @dependabot in https://github.com/apache/iceberg/pull/12728 * Build: Bump io.delta:delta-standalone_2.12 from 3.3.0 to 3.3.1 by @dependabot in https://github.com/apache/iceberg/pull/12731 * Build: Bump net.snowflake:snowflake-jdbc from 3.23.0 to 3.23.2 by @dependabot in https://github.com/apache/iceberg/pull/12732 * Flink: Backport RowConverter to Flink 1.19 and 1.18 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12713 * Flink: Fix npe in SketchUtil when numPartitions bigger than length of samples by @Guosmilesmile in https://github.com/apache/iceberg/pull/12703 * Core: Lazy init workerPool in RemoveSnapshots and SnapshotProducer by @abstractdog in https://github.com/apache/iceberg/pull/12427 * Flink: Backport Typeinfo in RowConverter to Flink 1.19 and 1.18 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12739 * Hive: Refactor HMS table parameter setting to be able to reuse by @zratkai in https://github.com/apache/iceberg/pull/12461 * Flink:Backport fix npe in SketchUtil to Flink 1.19 and 1.18 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12741 * Core: Enable row lineage for all v3 tables by @rdblue in https://github.com/apache/iceberg/pull/12593 * Migrate Spark 3.4 ExtensionsTestBase-related tests by @tomtongue in https://github.com/apache/iceberg/pull/12744 * Build: Bump io.delta:delta-spark_2.12 from 3.3.0 to 3.3.1 by @dependabot in https://github.com/apache/iceberg/pull/12729 * Doc: Remove Hive 2.x/3.x related docs in hive.md by @jackylee-ch in https://github.com/apache/iceberg/pull/12700 * Core: Return this instead of null in enableRowLineage() by @nastra in https://github.com/apache/iceberg/pull/12747 * Core: Drop invalid function comment for HTTPClient.isSuccessful by @gaborkaszab in https://github.com/apache/iceberg/pull/12742 * CORE: Allow HTTPClient to parse headers from properties. by @wolflex888 in https://github.com/apache/iceberg/pull/12595 * Throw on `{write.folder-storage.path,write.object-storage.path}` properties by @Fokko in https://github.com/apache/iceberg/pull/12315 * Build: Bump io.netty:netty-buffer from 4.1.119.Final to 4.2.0.Final by @dependabot in https://github.com/apache/iceberg/pull/12730 * API: Deprecate CredentialSupplier by @nastra in https://github.com/apache/iceberg/pull/12763 * AWS: Add unit tests for AWS s3Async by @sullis in https://github.com/apache/iceberg/pull/12758 * Build: Bump com.google.cloud:libraries-bom from 26.58.0 to 26.59.0 by @dependabot in https://github.com/apache/iceberg/pull/12733 * Upgrades Flink Version to 1.19.2 and 1.20.1 by @rodmeneses in https://github.com/apache/iceberg/pull/12745 * Update hive-quickstart.md by @mollonado in https://github.com/apache/iceberg/pull/12746 * Flink: fix rateLimit argument check in TableMaintenance by @Guosmilesmile in https://github.com/apache/iceberg/pull/12773 * Flink: backport fix rateLimit argument check in TableMaintenance to Flink 1.19 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12776 * Docs: Add Estuary to docs and vendors by @aeluce in https://github.com/apache/iceberg/pull/12764 * Core: Update RewriteFiles tests to test against V3 by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12777 * Migrate Spark 3.4 ExtensionsTestBase-related tests for Partition, Schema and Branch/Tag by @tomtongue in https://github.com/apache/iceberg/pull/12766 * Build: Bump guava from 33.4.6-jre to 33.4.7-jre by @dependabot in https://github.com/apache/iceberg/pull/12789 * Build: Bump nessie from 0.103.2 to 0.103.3 by @dependabot in https://github.com/apache/iceberg/pull/12786 * Core: Use OutputFile.location(), InputFile.location() in Error Messages by @Jordano-Dremio in https://github.com/apache/iceberg/pull/12755 * Flink: Move unlock from MemoryLock open to TestCase Before by @Guosmilesmile in https://github.com/apache/iceberg/pull/12793 * Flink: backport #12793 to Flink 1.19 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12795 * Build: Bump junit to 5.12.2 by @iProdigy in https://github.com/apache/iceberg/pull/12391 * Flink: Fix TriggerManager to unlock task execution when previous job left an orphaned lock by @Guosmilesmile in https://github.com/apache/iceberg/pull/12794 * Flink: backport fix TriggerManager to unlock task execution when previous job left an orphaned lock for Flink 1.19 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12801 * Flink: fix parameters for test recovery by @Guosmilesmile in https://github.com/apache/iceberg/pull/12800 * Spark 3.4: Migrate integration test to JUnit5 by @nastra in https://github.com/apache/iceberg/pull/12796 * CatalogTests: Fix listNamespaces Check, Avoid Reserved Keyword, Allow Configurable Location by @talatuyarer in https://github.com/apache/iceberg/pull/12768 * Update the docs for working with Flink and REST catalog by @dttung2905 in https://github.com/apache/iceberg/pull/12726 * Migrate Spark 3.4 ExtensionsTestBase-related tests for Snapshot manipulation, ChangeLogView and Distribution/Ordering by @tomtongue in https://github.com/apache/iceberg/pull/12807 * Core: Fix deprecated FileSystem.isDirectory warning and remove redundant test code by @ebyhr in https://github.com/apache/iceberg/pull/12805 * Core: Test loading table/view with non-existing namespace by @nastra in https://github.com/apache/iceberg/pull/12812 * Doc: Fix typo in spark-procedures.md by @slfan1989 in https://github.com/apache/iceberg/pull/12814 * Spark 3.4: Migrate ExtensionsTestBase-related remaining tests by @tomtongue in https://github.com/apache/iceberg/pull/12813 * Docs: Add the recommended style for ArrayAssertions by @tomtongue in https://github.com/apache/iceberg/pull/12820 * Spark 3.5: Use ProcedureInput for SnapshotTableProcedure. by @slfan1989 in https://github.com/apache/iceberg/pull/12783 * Spark 3.5: Use ProcedureInput for MigrateTableProcedure. by @slfan1989 in https://github.com/apache/iceberg/pull/12782 * Fix versions in LICENSE and NOTICE by @jbonofre in https://github.com/apache/iceberg/pull/12831 * Spark3.4: Backport ProcedureInput for MigrateTableProcedure And SnapshotTableProcedure (#12782 #12783) by @slfan1989 in https://github.com/apache/iceberg/pull/12837 * Core: Support first-row-id for manifests and manifest lists by @rdblue in https://github.com/apache/iceberg/pull/12672 * Site: Remove Iceberg Summit Link from the Homepage by @RussellSpitzer in https://github.com/apache/iceberg/pull/12842 * Core: use ALL_VERSIONS constant in TestBase by @sullis in https://github.com/apache/iceberg/pull/12748 * Spec: Update row lineage requirements for upgrading tables by @rdblue in https://github.com/apache/iceberg/pull/12781 * Flink: Add StreamingStartingStrategy.INCREMENTAL_FROM_LATEST_SNAPSHOT_EXCLUSIVE by @morhidi in https://github.com/apache/iceberg/pull/12839 * Spark: Add _row_id and _last_updated_sequence_number readers by @rdblue in https://github.com/apache/iceberg/pull/12836 * spec: Variant lower/upper bounds by @aihuaxu in https://github.com/apache/iceberg/pull/12658 * API: Use normalized JSON path to identify Variant fields by @rdblue in https://github.com/apache/iceberg/pull/12835 * Core: Add test cases for row lineage metadata by @rdblue in https://github.com/apache/iceberg/pull/12843 * Spark 3.5: Add Parallelism Parameter Validation to AddFilesProcedure. by @slfan1989 in https://github.com/apache/iceberg/pull/12784 * Spec: Allow the use of `source-id` in V3 by @Fokko in https://github.com/apache/iceberg/pull/12644 * Core: Use assumeThat instead of assumeTrue by @slfan1989 in https://github.com/apache/iceberg/pull/12822 * Spark3.4: Migrate tests in spark, extensions and functions by @tomtongue in https://github.com/apache/iceberg/pull/12853 * API: Don't check underlying error msg on AIOOBE by @nastra in https://github.com/apache/iceberg/pull/12867 ## New Contributors * @danicafine made their first contribution in https://github.com/apache/iceberg/pull/12256 * @czy006 made their first contribution in https://github.com/apache/iceberg/pull/11966 * @lfrancke made their first contribution in https://github.com/apache/iceberg/pull/12344 * @wenxin-cn made their first contribution in https://github.com/apache/iceberg/pull/12402 * @qining-mj made their first contribution in https://github.com/apache/iceberg/pull/12394 * @istreeter made their first contribution in https://github.com/apache/iceberg/pull/12318 * @jiayuasu made their first contribution in https://github.com/apache/iceberg/pull/12421 * @bharos made their first contribution in https://github.com/apache/iceberg/pull/12327 * @swapna267 made their first contribution in https://github.com/apache/iceberg/pull/12199 * @XBaith made their first contribution in https://github.com/apache/iceberg/pull/12366 * @rbowen made their first contribution in https://github.com/apache/iceberg/pull/12468 * @sida-shen made their first contribution in https://github.com/apache/iceberg/pull/12481 * @SanjayMarreddi made their first contribution in https://github.com/apache/iceberg/pull/12299 * @thjaeckle made their first contribution in https://github.com/apache/iceberg/pull/11780 * @kumarpritam863 made their first contribution in https://github.com/apache/iceberg/pull/12372 * @drexler-sky made their first contribution in https://github.com/apache/iceberg/pull/12546 * @SourabhEstuary made their first contribution in https://github.com/apache/iceberg/pull/12587 * @ChaladiMohanVamsi made their first contribution in https://github.com/apache/iceberg/pull/11577 * @wolflex888 made their first contribution in https://github.com/apache/iceberg/pull/12612 * @andyglow made their first contribution in https://github.com/apache/iceberg/pull/12418 * @akoller made their first contribution in https://github.com/apache/iceberg/pull/12708 * @ricardopereira33 made their first contribution in https://github.com/apache/iceberg/pull/12670 * @abstractdog made their first contribution in https://github.com/apache/iceberg/pull/12427 * @zratkai made their first contribution in https://github.com/apache/iceberg/pull/12461 * @jackylee-ch made their first contribution in https://github.com/apache/iceberg/pull/12700 * @mollonado made their first contribution in https://github.com/apache/iceberg/pull/12746 * @aeluce made their first contribution in https://github.com/apache/iceberg/pull/12764 * @Jordano-Dremio made their first contribution in https://github.com/apache/iceberg/pull/12755 * @iProdigy made their first contribution in https://github.com/apache/iceberg/pull/12391 * @talatuyarer made their first contribution in https://github.com/apache/iceberg/pull/12768 * @dttung2905 made their first contribution in https://github.com/apache/iceberg/pull/12726 * @morhidi made their first contribution in https://github.com/apache/iceberg/pull/12839 **Full Changelog**: https://github.com/apache/iceberg/compare/apache-iceberg-1.8.0...apache-iceberg-1.9.0

iceberg

What's Changed

New Contributors

Related Projects

mapbox-navigation-android

ToastFish

barcodelib

haze