New
v0.54.0
Upgrade Notes
Rust Core Public API
RFC-6189: Remove Native Blocking Support
OpenDAL v0.54 implements RFC-6189, which removes all native blocking support in favor of using block_on from async runtimes.
The following breaking changes have been made:
blocking::Operatorcan no longer be used within async contexts- Using blocking APIs now requires an async runtime
- All
Blocking*types have been moved to theopendal::blockingmodule
To migrate:
- use opendal::BlockingOperator;
+ use opendal::blocking::Operator;
RFC-6213: Options Based API
OpenDAL v0.54 implements RFC-6213, which introduces options-based APIs for more structured and extensible operation configuration.
New APIs added:
read_options(path, ReadOptions)write_options(path, data, WriteOptions)list_options(path, ListOptions)stat_options(path, StatOptions)delete_options(path, DeleteOptions)
Example usage:
// Read with options
let options = ReadOptions::new()
.range(0..1024)
.if_match("etag");
let data = op.read_options("path/to/file", options).await?;
// Write with options
let options = WriteOptions::new()
.content_type("text/plain")
.cache_control("max-age=3600");
op.write_options("path/to/file", data, options).await?;
Remove stat_has_xxx and list_has_xxx APIs
All stat_has_* and list_has_* capability check APIs have been removed. Instead, check capabilities directly on the Capability struct:
- if op.info().full_capability().stat_has_content_length() {
+ if op.info().full_capability().stat.content_length {
// ...
}
Fix with_user_metadata signature
The signature of with_user_metadata has been changed. Please update your code accordingly if you use this method.
Services removed due to lack of maintainer
The following services have been removed due to lack of maintainers:
atomicservericloudnebula_graph
If you need these services, please consider maintaining them or use alternative services.
HttpClientLayer replaces update_http_client
The Operator::update_http_client() method has been replaced by HttpClientLayer:
- op.update_http_client(client);
+ op = op.layer(HttpClientLayer::new(client));
Expose presign_xxx_options API
New options-based presign APIs have been exposed:
let options = PresignOptions::new()
.expire(Duration::from_secs(3600));
let url = op.presign_read_options("path/to/file", options).await?;
Rust Core Raw API
Remove native blocking support
All native blocking implementations have been removed from the raw API. Services and layers no longer need to implement blocking-specific methods.
Bindings Java Breaking changes
Removed services
The following services have been removed:
- Chainsafe service has been removed (PR-5744) - The service has been sunset.
- libsql service has been removed (PR-5616) - Dead service removal.
Batch operations removed
PR-5393 removes the batch concept from OpenDAL. All batch-related operations and capabilities have been removed.
Capability changes
Options-based API
New options classes have been introduced for structured operation configuration:
ReadOptions- for read operationsWriteOptions- for write operationsListOptions- for list operationsStatOptions- for stat operations
Example usage:
// Read with options
ReadOptions options = ReadOptions.builder()
.range(0, 1024)
.ifMatch("etag")
.build();
byte[] data = operator.read("path/to/file", options);
// Write with options
WriteOptions options = WriteOptions.builder()
.contentType("text/plain")
.cacheControl("max-age=3600")
.build();
operator.write("path/to/file", data, options);
Bindings Python
Breaking change: Native blocking API removed
OpenDAL has removed the native blocking API in the core. The Python binding's blocking API now uses an async runtime internally. This is a transparent change and should not affect most users, but:
- The blocking API now requires an async runtime to be available
- Performance characteristics may be slightly different
- All blocking operations are now implemented as async operations running in a tokio runtime
Breaking change: Removed services
The following services have been removed due to lack of maintainers and users:
atomicserver- This service is no longer supportedicloud- This service is no longer supportednebula_graph- This service is no longer supported
If you were using any of these services, you'll need to migrate to an alternative storage backend.
Breaking change: Chainsafe service removed
The Chainsafe service has been sunset and is no longer available.
What's Changed
Added
- RFC-6213: Options API by @Xuanwo in https://github.com/apache/opendal/pull/6213
- feat(core): Expose xxx_options API by @Xuanwo in https://github.com/apache/opendal/pull/6215
- RFC-6209: Glob Support by @asukaminato0721 in https://github.com/apache/opendal/pull/6209
- feat(bindings/java): Add WriteOptions support for new options API by @geruh in https://github.com/apache/opendal/pull/6219
- feat(services/azdls): Support parsing Azure Storage configs from connection strings by @DerGut in https://github.com/apache/opendal/pull/6212
- feat(bindings/java): Add ListOptions support for new options API by @geruh in https://github.com/apache/opendal/pull/6246
- feat(bindings/python): Enhance Reader and Writer by @chitralverma in https://github.com/apache/opendal/pull/6086
- feat(bindings/java): Add StatOptions support for new options API by @geruh in https://github.com/apache/opendal/pull/6255
- feat(website): Auto-generate llms.txt and llms-full.txt by @kingsword09 in https://github.com/apache/opendal/pull/6247
- oli: support dropbox by @Kinchkun in https://github.com/apache/opendal/pull/6265
- feat(bindings/python): Enhance Stat, Lister, Metadata & Entry by @chitralverma in https://github.com/apache/opendal/pull/6232
- feat(layers): add fastmetrics layer by @koushiro in https://github.com/apache/opendal/pull/6269
- feat(bindings/haskell): add more api by @asukaminato0721 in https://github.com/apache/opendal/pull/6264
- feat(core): Expose presign_xxx_options API by @geruh in https://github.com/apache/opendal/pull/6273
- feat: Add HttpClientLayer to replace
Operator::update_http_client()by @Xuanwo in https://github.com/apache/opendal/pull/6290 - feat(bin/oli): support oli edit by @asukaminato0721 in https://github.com/apache/opendal/pull/6229
- feat(bindings/cpp): cpp async op && reader, lister by @asukaminato0721 in https://github.com/apache/opendal/pull/6228
- feat(services/moka): expose more moka configurations by @koushiro in https://github.com/apache/opendal/pull/6285
- feat(bindings/nodejs): Add StatOptions support for new options API by @kingsword09 in https://github.com/apache/opendal/pull/6282
- feat: implement --tree option for oli ls subcommand by @waynexia in https://github.com/apache/opendal/pull/6311
- feat(bindings/nodejs): Add ReadOptions and ReaderOptions support for new options API by @kingsword09 in https://github.com/apache/opendal/pull/6312
- feat(bindings/nodejs): Add ListOptions support for new options API by @kingsword09 in https://github.com/apache/opendal/pull/6320
- feat(bindings/go): add benchmark by @yuchanns in https://github.com/apache/opendal/pull/6341
- feat(services/azdls): Implement write returns metadata by @jonathanc-n in https://github.com/apache/opendal/pull/6368
Changed
- refactor(core/types)!: fix
with_user_metadatasignature by @meteorgan in https://github.com/apache/opendal/pull/5960 - refactor(!): Remove services lack of maintainers and users by @Xuanwo in https://github.com/apache/opendal/pull/6263
- refactor(services/moka)!: replace
sync::Cachewithfuture::Cacheby @koushiro in https://github.com/apache/opendal/pull/6270 - refactor(bindings/go): Restructure FFI system with type-safe wrapper by @yuchanns in https://github.com/apache/opendal/pull/6268
- refactor: Migrate redis from adapter::kv to Access instead by @Xuanwo in https://github.com/apache/opendal/pull/6291
- refactor: Migrate moka from adapter::typed_kv to Access instead by @Xuanwo in https://github.com/apache/opendal/pull/6300
- refactor: Migrate memory service to implment Access directly by @Xuanwo in https://github.com/apache/opendal/pull/6301
- refactor: Migrate services cacache to implement Access by @Xuanwo in https://github.com/apache/opendal/pull/6303
- refactor!: Remove stat_has_xxx and list_has_xxx by @Xuanwo in https://github.com/apache/opendal/pull/6313
- refactor(services/fs): extract implementation to core by @erickguan in https://github.com/apache/opendal/pull/6317
- refactor: Migrate mini_moka service to implement Access directly by @meteorgan in https://github.com/apache/opendal/pull/6316
- refactor: remove uuid dependency when creating a temp path by @erickguan in https://github.com/apache/opendal/pull/6324
- refactor(layers/logging): Don't trigger logigng in heavy IO path by @Xuanwo in https://github.com/apache/opendal/pull/6343
- refactor: Migrate dashmap service to implement Access directly by @meteorgan in https://github.com/apache/opendal/pull/6344
Fixed
- fix: java bug in list with delete option test by @geruh in https://github.com/apache/opendal/pull/6257
- fix(nodejs): esmodule and commonjs support by @kingsword09 in https://github.com/apache/opendal/pull/6266
- fix(gcs): headers missing in XML multipart API and incorrect x-goog-acl header values in XML API by @wlinna in https://github.com/apache/opendal/pull/6275
- fix(bindings/nodejs): update nodejs and deno bench by @kingsword09 in https://github.com/apache/opendal/pull/6286
- fix(hdfs): fix infinite loop in write for HDFS failure by @oven-yang in https://github.com/apache/opendal/pull/6295
- fix(nodejs): test stat with version by @kingsword09 in https://github.com/apache/opendal/pull/6307
- fix(bindings/python): Fix Writer doesn't throw correct error code by @Xuanwo in https://github.com/apache/opendal/pull/6315
- fix(python): correctly calculate end bound using offset + size instead of size directly by @kingsword09 in https://github.com/apache/opendal/pull/6314
- fix(fs/ftp/hdfs): correct tmp_path generation for append operations by @kingsword09 in https://github.com/apache/opendal/pull/6327
- fix(dav-server): Fix create_dir to create nested directories by @sqlpxc in https://github.com/apache/opendal/pull/6321
- fix(service/fs): handle if_not_exists flag to raise ConditionNotMatch error by @kingsword09 in https://github.com/apache/opendal/pull/6326
- fix(services/fs): Avoid creating partial files by @Xuanwo in https://github.com/apache/opendal/pull/6336
- fix(bindings/nodejs): ListOptions test list with deleted by @kingsword09 in https://github.com/apache/opendal/pull/6335
- fix(bindings/go): ffi calls use after free by @yuchanns in https://github.com/apache/opendal/pull/6380
- fix(services/azdls): Fix append not handled correctly while offset==0 by @Xuanwo in https://github.com/apache/opendal/pull/6393
Docs
- docs: Remove deprecated APIs and polish docs for public APIs by @Xuanwo in https://github.com/apache/opendal/pull/6220
- docs(services/hdfs_native): fix outdated capabilities and config option name by @kezhuw in https://github.com/apache/opendal/pull/6224
- docs: Add CLAUDE docs to make AI Agents happy by @Xuanwo in https://github.com/apache/opendal/pull/6299
- docs: Polish claude file after some experiments by @Xuanwo in https://github.com/apache/opendal/pull/6302
- docs: Add upgrade guide for opendal's 0.54 release by @Xuanwo in https://github.com/apache/opendal/pull/6382
CI
- ci: Use expression syntax to avoid VS Code warnings. by @kingsword09 in https://github.com/apache/opendal/pull/6284
- ci: Disable failed CI until #6305 been fixed by @Xuanwo in https://github.com/apache/opendal/pull/6306
- chore(ci): use mlugg/setup-zig instead of archvied action by @assignUser in https://github.com/apache/opendal/pull/6310
- ci(bindings/go): simplify and improve Go bindings test infrastructure by @yuchanns in https://github.com/apache/opendal/pull/6293
- ci(services/compfs): add integrtation tests for compfs service by @meteorgan in https://github.com/apache/opendal/pull/6319
- chore(dependabot): update CI for dependabot PRs by @erickguan in https://github.com/apache/opendal/pull/6411
Chore
- chore: add deepwiki badge into readme to enable auto-refresh by @koushiro in https://github.com/apache/opendal/pull/6200
- chore: upgrade opentelemetry to 0.30.0 by @tisonkun in https://github.com/apache/opendal/pull/6259
- chore: Update bb8 to version 0.9.0 by @cryptomilk in https://github.com/apache/opendal/pull/6127
- chore(deps): bump uuid from 1.16.0 to 1.17.0 in /bin/oli by @dependabot[bot] in https://github.com/apache/opendal/pull/6245
- chore(bindings/go): update Go dependencies by @yuchanns in https://github.com/apache/opendal/pull/6280
- chore(deps): bump tokio from 1.45.0 to 1.45.1 in /bin/ofs by @dependabot[bot] in https://github.com/apache/opendal/pull/6240
- chore(metrics): add more docs about global instance of PrometheusLayer and FastmetricsLayer by @koushiro in https://github.com/apache/opendal/pull/6308
- chore: Add npm to dependabot by @shaonianche in https://github.com/apache/opendal/pull/6318
- chore(deps): bump astral-sh/setup-uv from 5 to 6 by @dependabot[bot] in https://github.com/apache/opendal/pull/6241
- chore(deps): update datafusion requirement from 47.0.0 to 48.0.0 in /integrations/object_store by @dependabot[bot] in https://github.com/apache/opendal/pull/6332
- chore(deps): bump @docusaurus/core from 3.6.1 to 3.8.1 in /website by @dependabot[bot] in https://github.com/apache/opendal/pull/6334
- chore(deps): bump compio from 0.14.0 to 0.15.0 in /core by @dependabot[bot] in https://github.com/apache/opendal/pull/6331
- chore(deps): bump clap from 4.5.38 to 4.5.40 in /bin/ofs by @dependabot[bot] in https://github.com/apache/opendal/pull/6330
- chore(deps): bump dirs from 5.0.1 to 6.0.0 in /bin/oli by @dependabot[bot] in https://github.com/apache/opendal/pull/6329
- chore: fix clippy warnings when using rust 1.88 by @koushiro in https://github.com/apache/opendal/pull/6339
- chore(dav-server): Add a test for creating nested directories by @sqlpxc in https://github.com/apache/opendal/pull/6338
- chore: bump msrv to v1.82.0 by @MrCroxx in https://github.com/apache/opendal/pull/6348
- chore(deps): bump crate-ci/typos from 1.31.1 to 1.34.0 by @dependabot[bot] in https://github.com/apache/opendal/pull/6351
- chore(deps): bump tokio from 1.45.0 to 1.45.1 in /bin/oli by @dependabot[bot] in https://github.com/apache/opendal/pull/6353
- chore(deps): bump quick-xml from 0.36.2 to 0.37.5 in /bin/oay by @dependabot[bot] in https://github.com/apache/opendal/pull/6350
- chore(deps): bump actions/checkout from 3 to 4 by @dependabot[bot] in https://github.com/apache/opendal/pull/6356
- chore(deps): bump semver from 7.6.3 to 7.7.2 in /website by @dependabot[bot] in https://github.com/apache/opendal/pull/6352
- chore(deps): bump toml from 0.8.22 to 0.8.23 in /bin/oli by @dependabot[bot] in https://github.com/apache/opendal/pull/6361
New Contributors
- @kezhuw made their first contribution in https://github.com/apache/opendal/pull/6224
- @DerGut made their first contribution in https://github.com/apache/opendal/pull/6212
- @chitralverma made their first contribution in https://github.com/apache/opendal/pull/6086
- @Kinchkun made their first contribution in https://github.com/apache/opendal/pull/6265
- @oven-yang made their first contribution in https://github.com/apache/opendal/pull/6295
- @assignUser made their first contribution in https://github.com/apache/opendal/pull/6310
- @sqlpxc made their first contribution in https://github.com/apache/opendal/pull/6321
- @MrCroxx made their first contribution in https://github.com/apache/opendal/pull/6348
- @jonathanc-n made their first contribution in https://github.com/apache/opendal/pull/6368
Full Changelog: https://github.com/apache/opendal/compare/v0.53.3...v0.54.0