New
v0.26.2
Added
pw.io.gdrive.readnow supports the"only_metadata"format. When this format is used, the table will contain only metadata updates for the tracked directory, without reading object contents.- Detailed metrics can now be exported to SQLite. Enable this feature using the environment variable
PATHWAY_DETAILED_METRICS_DIRor viapw.set_monitoring_config(). pw.io.kinesis.readandpw.io.kinesis.writemethods for reading from and writing to AWS Kinesis.
Fixed
- A bug leading to potentially unbounded memory consumption that could occur in
Table.forgetandTable.sortoperators during multi-worker runs has been fixed. - Improved memory efficiency during cold starts by compacting intermediary structures and reducing retained memory after backfilling.
Changed
- The frequency of background operator snapshot compression in data persistence is limited to the greater of the user-defined
snapshot_intervalor 30 minutes when S3 or Azure is used as the backend, in order to avoid frequent calls to potentially expensive operations. - The Google Drive input connector performance has been improved, especially when handling directories with many nested subdirectories.
- The MCP server
toolmethod now allows to pass the optional datatitle,output_schema,annotationsandmetato inform the LLM client. - Relaxed boto3 dependency to <2.0.0.