Unclaimed project
Are you a maintainer of glow? Claim this project to take control of your public changelog and roadmap.
Changelog
An open-source toolkit for large-scale genomic analysis
Last updated about 1 month ago
v1.2.1 bumps glow to Spark v3.2.1
This release includes Java/Scala artifacts in Maven Central , and Python artifacts in pypi. Docker containers projectglow/open-source-glow:1.2.1, projectglow/databricks-glow:1.2.1, projectglow/databricks-glow:10.4 and projectglow/databricks-hail:0.2.93 can be found in projectglow's dockerhub. The Glow notebook continous integration test now uses Databricks Runtime 10.4, which is on Spark 3.2.1 ()
♞ lichess.org: the forever free, adless and open source chess server ♞
💾 Database Tools incl. ORM, Migrations and Admin UI (Postgres, MySQL & MongoDB) [deprecated]
Scala 2 compiler and standard library. Scala 2 bugs at https://github.com/scala/bug; Scala 3 at https://github.com/scala/scala3
Glow leverages private catalyst APIs that have changed from Spark 3.1 to Spark 3.2. We wrote a Shim to maintain backwards compatibility. However, Spark 2 is end of life (EoL). Databricks, AWS EMR and Google Dataproc now depend on Hadoop 3.x, which is incompatible with Spark 2. So we are removing support for Spark 2, including the Spark 2 continuous integration tests (ci/cd) performed with circleci. Glow version 1.1.2 is the last release that supports Spark 2
The Spark 3 ci/cd tests depend on Hail, and these were failing since Hail does not yet support Spark 3.2, they are waiting on Google's Dataproc and AWS EMR to upgrade from Spark 3.1. So for now we expect the Spark 3 circleci tests to continue failing until we can resolve the hail tests. However, we moved forward with the new release as it is unclear when Dataproc or EMR will support Spark 3.2
Thanks to Alex Barreto, Jasser Abidi, Cameron Smith, Marcus Henry, Karen Feng, Joseph Bradley, and William Brandler for their contributions to this release
Full Changelog: https://github.com/projectglow/glow/compare/v1.1.2...v1.2.1