v1.3.0
This is the fourth major release of JuiceFS, and is an LTS version as well that will be maintained for 24 months. Since v1.2.0 it has 696 commits from 60 contributors, thanks to @zhijian-pro @jiefenghuang @polyrabbit @zhoucheng361 @davies @chenjie4255 @YunhuiChen @tangyoupeng @CodingPoeta @CaitinChen @Xuyuchao-juice @SandyXSD @anysql @xiaogaozi @yuhr123 @zxh326 @kyungwan-nam @Suave @timfeirg @zwwhdls @wenjianhn @winglq @cxljs @dsd2077 @ddrid @daimashusheng @SonglinLife @williamstein @mt5225 @lidaohang @JoyLiuc @Looka149 @YinhaoHu @justforlxz @yetone @wpleonardo @taadis @shenxingwuying @prOOrc @myron369 @menwenjun @hawkli-1994 @bitsark @chnliyong @austinhmh @wzhallright @201341 @servusdei2018 @mrdrivingduck @maxjustus @lianneli @davidzhangbj @sjp00556 @jerrycaimin @aucusaga @solracsf @fuyufjh @veinkr @DASWU @achetronic !
Highlights
- Python SDK (#5437,#5516,#5573,#5779,#5852,#5847,#5853,#5931,#5998...)
- Improved tools and compatibility on Windows (#5376,#5630,#5629,#5643,#5661,#5714,#5724...)
- Speedup backup using protobuf (#5344,#5348,#5352,#5356,#5733,#5379,#5383,#5404...)
- Support Apache Ranger (#5285,#5430,#6016)
New
- fuse: add
--async_diooption to enable asynchronous handling of direct I/O (#4943,#4986) - fuse: add
--max-fuse-iooption to config max write size for request (#5087,#6017) - fuse: add timeout for FUSE requests (#5670,#5681,#5690)
- fuse: enable kernel symlink cache (#5766)
- fuse: support /dev/fd/N mount point for
--fusemount(#5076,#5082,#5976) - cmd/mount: add
max-readaheadoption to set the max readahead buffer (#5225,#5229,#5704) - cmd/mount: add
--sort-diroption to sort entries by name within a directory (#5031) - cmd/mount: add
readdir-cacheoption to enable kernel caching readdir entries (#5462) - cmd/mount: add
negative-entry-cacheoption to allow kernel caching negative dir entry lookups (#5788,#6038) - cmd/mount: add
cache-large-writeoption to cache blocks after uploading (#5261) - cmd/mount: add
cache-itemsoption to limit number of cache items (#5489,#5495) - cmd/mount: add
check-storageoption to perform object storage test at startup (#5281) - cmd/mount: add
enable-capandenable-selinuxoptions to enable corresponding xattr requests (#5406,#5962) - cmd/mount: add
all-squashoption to map all users to a specified one (#5395) - cmd/mount: add
umaskoption to overwrite the one from app (#5756,#5942) - cmd/mount: add
disable-transparent-hugepageoption to disable transparent huge page (#6147) - cmd/mount: supervisor process pass signal to service process (#5238,#5245,#5246,#5278,#5624)
- vfs: support
O_TMPFILEflag (#5309,#5433) - meta: support readdir streaming (#5162,#5207,#5223,#5242,#5482,#5504,#5895)
- meta: add
fast-statfsoption to speed up statfs (#6168,#6171) - meta/tikv: support
open-tso-follower-proxyandmax-tso-batch-wait-intervaloptions (#6166) - meta/sql: support multiple volumes sharing a single db (#5791,#5804)
- cache: add env
JFS_DROP_OSCACHEto enable dropping page cache (#4981) - cmd/format: support TLS configuration via bucket URL parameters (#5241)
- cmd/sync: add
max-failureoption to abort on error threshold (#5142) - cmd/sync: add
start-timeandend-timeoptions to filter file (#6072,#6121) - cmd/sync: add
check-changeoption to verify file size (#6093) - cmd/quota: add
--createoption to create directory if not exists (#5196) - cmd/rmr: add
skip-trashoption to delete files directly (#5384,#5818) - cmd: add
threadsoption for rmr tool, webdav, gateway and libjfs to delete files (#5522,#5535) - cmd/gateway: add
object-metaoption to enable object metadata api (#5536) - cmd/sync: add
files-fromoption to specify the list of files or directories to sync (#5639,#5864,#5894) - cmd/objbench: add
shardsto support object storage shards (#5755) - cmd/dump&load: support zstd compression (#4921)
- cmd/webdav: support PROPPATCH method (#5041)
- sdk/hadoop: support subdir option in hadoop sdk (#6097,#6141)
Changed
- vfs: limit buffer usage for read requests (#5557)
- meta: improve statfs performance (#5161)
- meta/kv: improve mknod performance via batch get (#5056)
- meta: improve performance via inode prefetching. (#5130,#5481)
- meta: reduce
GetAttrcalls (#5540) - meta: optimizations for reducing transaction conflicts (#5308,#5330,#5574)
- meta: adjust the interval to one hour for deleted files cleanup (#5343)
- meta: cancel background tasks when closing session (#5486,#5613,#5943,#5986,#6108,#6060)
- meta/kv: improve scan performance and robustness (#5298,#5314,#5080)
- meta/kv: simplify transactions in point get scenarios to reduce one RPC call for TiKV (#6136,#6150)
- meta/sql: improve performance by simplifying transaction and reducing locks (#5377,#5460)
- meta/sql: set default transaction isolation level to repeatable-read (#5432,#5456)
- meta/sql: support more connection options (#5512,#5806)
- meta/sql: improve compatibility by updating exact required fields (#6153,#6158)
- meta/redis: support tls server name configuration (#5182)
- cache: inherit atime and track logic size when re-scan (#5692)
- cache: adjust cache capacity and max items by freeratio (#5970)
- cache: change the default value of
verify-cache-checksumoption toextend(#6103) - cmd: exit compact timely and log a message when long-running background tasks start (#6033)
- cmd: refuse more than expected positional arguments (#6154)
- cmd/mount: the
upload-hoursoption allow using-as a separator (#5494) - cmd/mount: set the process as an
IO_FLUSHERto avoid potential hang (#6144,#6157) - cmd/gateway: add one more hierarchy to temporary multipart dir to reduce transaction conflicts (#5549)
- cmd/gateway: improve the gateway performance in read-only mode (#6187)
- cmd/gateway: some performance optimizations (#5163,#5548,#5544)
- cmd/gateway: delete intermediate temporary files promptly to reduce inode usage (#6080)
- cmd/gateway: implement ListObjectVersions (#6173)
- cmd/sync: support more environment variables in cluster mode (#5364,#5422)
- cmd/sync: improve performance by calculating checksum in flight to reduce redundant reads from the source (#5524)
- cmd/warmup: accelerate big file warmup (#5713,#5886,#5897)
- cmd/warmup: improve performance by reducing file operations (#5408)
- cmd/warmup: show cache location info (#5559)
- cmd/info: add column for absolute file offset (#6037)
Bugfix
- vfs: remove object files if metadata commit fails with ENOENT (#5133)
- vfs: allow
ftruncateon an opened and deleted file (#5664) - meta: fix the issue that trash files may be dangling (#5414)
- meta: evict symlink cache periodically (#5131)
- meta: fix permission check of open and mknod (#4948)
- meta: fix volume stats update issues (#4979,#5221,#5272,#5280,#5303,#5305)
- meta: fix the issue that the username cannot contain the "@" character (#5761)
- meta: fix potential issues with context reuse (#5372,#5665)
- meta: exit
doCleanupSlicestimely (#6081) - meta: check for incorrect filenames (#6073)
- meta: fix stats for trash directories (#6099)
- meta: fix all-zero read response due to invalid openFile.chunks cache (#6192)
- meta: fix leaking objects when quota is full (#6200)
- meta/kv: return EINVAL for empty xattr value when using TiKV (#6104)
- meta/sql: fix nanosecond of link and readlink (#5496)
- meta/sql: fix sqlite3 "database is locked" issue (#5586)
- meta/sql: fix flush quotas and stats deadlock (#5706,#5712)
- meta/sql: fix database retry issues (#5440,#5485)
- meta/sql: fix concurrency issue in json backup dump (#5930)
- meta/redis: fix potential loss of attributes issue (#5709)
- meta/redis: check
MGeterror to avoid panic (#5773) - meta/redis: fix parsing options for redis cluster (#6129)
- chunk: remove unexpected read retries (#5199)
- chunk: deduplicate prefetch/partial-block/full-block requests to avoid read amplification (#4947,#5219)
- cache/stage: retry uploading stage blocks upon disk read failures (#5084)
- cache/stage: add stage write timeout with object storage fallback to prevent hangs (#5230,#5563,#5575)
- cache: disable writeback during compaction to prevent EIO from multi-client races (#5767)
- cache: maintain a margin based on the free-space ratio during cleanup to prevent client interference (#5216)
- cache: fix inoperative
--cache-expireoption in memory cache (#5313) - cache: fix cache duplication issue caused by concurrency (#5623)
- cache: fix potential high latency caused by locks in
getDiskUsage(#5857) - cache: fix sleep duration in cache expiration loop (#6165)
- cmd/mount: fix process management issues (#5064,#5172,#5641,#5881)
- cmd/mount: fix potential hangs during stack trace collection by adding timeout (#5325)
- cmd/mount: support using ":" to separate multiple in fstab (#5719)
Deps
- deps: bump
github.com/winfsp/cgofusefrom v1.5.0 to v1.6.0 (#5840) - deps: bump MySQL driver
github.com/go-sql-driver/mysqlfrom v1.7.1(2023-04-25) to v1.9.1(2025-03-20) (#5834) - deps: bump PostgreSQL driver
github.com/jackc/pgx/v5from v5.5.4 to v5.7.3 (#5835) - deps: bump sqlite3 driver
github.com/mattn/go-sqlite3from v1.14.16 to v1.14.24 (#5555) - deps: bump BadgerDB
github.com/dgraph-io/badger/v4from v4.2.0 to v4.5.1 (#5626) - deps: bump Redis client
github.com/redis/go-redis/v9from v9.0.2 to v9.7.3 (#5832) - deps: bump TiKV SDK from v2.0.1 to v2.0.7 (#5850)
- deps: bump baidu bce sdk from v0.9.200 to v0.9.221 (#5820)
- deps: bump tencent cos sdk to v0.7.45 to v0.7.63 (#5821)
- deps: bump volcengine tos sdk from v2.7.0 to v2.7.8 (#5838)
- deps: change aliyun oss sdk to
github.com/aliyun/alibabacloud-oss-go-sdk-v2v1.2.1 (#5830,#5878) - deps: bump ks3 sdk from v1.2.2 to v1.6.0 (#5829)
- deps: bump ibm cos sdk from v1.10.0 to v1.12.1 (#5828)
- deps: bump qiniu sdk from v7.15.0 to v7.25.2 (#5831)
- deps: bump gs sdk from v1.28.1(2022-12-02) to v1.48.0(2024-12-05) (#5849)
- deps: bump azure sdk from v1.0.0 to 1.3.1 (#5819)
- deps: bump
github.com/ncw/swift/v2from v2.0.1 to v2.0.3 (#5836) - deps: bump
github.com/colinmarc/hdfs/v2from v2.3.0 to v2.4.0 (#5827) - deps: bump
github.com/studio-b12/gowebdavfrom 3282f94193f2(2023-02-04) to v0.10.0 (#5839) - deps: bump prometheus from v1.15.1 to v1.21.1 (#5843)
- deps: bump
github.com/goccy/go-jsonfrom v0.10.2 to v0.10.5 (#5841) - deps: bump
github.com/grafana/pyroscope-gofrom v1.0.4 to v1.2.1 (#5842) - deps: bump
golang.org/x/cryptofrom 0.21.0 to 0.31.0 (#5374) - deps: bump
golang.org/x/netfrom v0.23.0 to v0.33.0 (#5412) - deps: bump
github.com/DataDog/zstdfrom v1.5.0 to v1.5.6 (#5741) - deps: bump
github.com/remyoudompheng/bigfftto latest version (#5747) - deps: bump
go.uber.org/automaxprocsfrom v1.5.2 to v1.6.0 (#5844)