Unclaimed project
Are you a maintainer of bacalhau? Claim this project to take control of your public changelog and roadmap.
Changelog
bacalhau
Community-driven, simple, yet powerful framework for fast, cost-effective distributed Compute over Data.
Back to changelogImproved
v1.7.0-rc1
Bacalhau v1.7.0 Release Notes
This release introduces enterprise support options, enhanced partitioning capabilities, improved networking, comprehensive authentication, and significant WebAssembly enhancements.
Enterprise Support through Expanso
- Enterprise support now available through Expanso while maintaining Bacalhau's open source core
- Free tier for everyone supporting up to 5 nodes with all core features
- Enterprise licensing option for organizations requiring professional support
Expanso Cloud Enhancements
- Comprehensive job submission capabilities through the Expanso Cloud web interface
- Library of pre-defined job templates for common use cases:
- Log processing workflows
- Data analysis with DuckDB
- Apache Iceberg operations
- And more
- Custom template creation capabilities with parameterization
- Simplified complex distributed computing tasks while maintaining flexibility
Enhanced Job Execution with Partitioned Workloads
- Distribution of work across compute nodes with built-in partition management
- Horizontal scaling and parallel processing for improved performance
- Granular failure recovery with automatic retry mechanisms for failed partitions
- Independent partition execution with context through environment variables
S3 Storage Input Partitioning
- Automatic data distribution across multiple executions with graceful failure handling
- Multiple partitioning strategies:
- Object-Based Distribution: Even distribution without specific grouping
- Date-Based Partitioning: Parallel processing for time-series analysis
- Regex Pattern Matching: Distribution by patterns in file paths
- Substring-Based Partitioning: Organization by customer segments
- Combined partitioned data with shared reference data support
S3 Publishing Enhancements
- Plain encoding support for direct access to job outputs without compression
- Individual file publishing rather than compressed archives
- Efficient access to individual files with no decompression overhead
- Default gzip encoding still available for complete result sets
Enhanced Networking Capabilities
- Networking enabled by default (shifted from opt-in to opt-out)
- Configuration option for restricting network access
- Experimental port mapping support
- Multiple network modes: bridge (default), host, and none
Comprehensive Authentication and Authorization
- Redesigned authentication system with multiple methods:
- Basic authentication
- API tokens
- Single Sign-On (SSO)
- Fine-grained authorization controls
- New CLI commands for authentication management
Host Environment Variable Forwarding
- Access to specified environment variables from the host system
- Secure credential passing through a controlled allowlist mechanism
- Reference host variables using
env: prefix in job specifications
Improved WebAssembly Support
- Refactored WASM job specification with better separation of concerns
- Remote modules defined as input sources for alignment with other executors
- HTTP networking support for WebAssembly through new host function implementation
- Configurable network access modes and security controls
- TinyGo-compatible client library for HTTP functionality
Docker DinD Integration
- Docker-in-Docker images for simplified containerized deployment
- Two image variants:
- Standard base image for client usage and orchestrator nodes
- DinD image for compute nodes running Docker workloads
- Pre-configured Docker daemon for simple integration with container ecosystems
Performance and Quality of Life Improvements
- Enhanced resource management and more efficient state tracking
count field now defaults to 1 instead of 0NetworkDefault type for automatic networking configuration
New Contributors
- @virajbhartiya made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4848
- @haixuanTao made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4857
- @Olg-K made their first contribution in https://github.com/bacalhau-project/bacalhau/pull/4873
Full Changelog: https://github.com/bacalhau-project/bacalhau/compare/v1.6.0...v1.7.0-rc1