What's Changed

• Refactored StickyPartitionAssignmentStrategy and implemented task estimation logic in LoadBasedPartitionAssignmentStrategy by @jzakaryan in https://github.com/linkedin/brooklin/pull/835
• Fix flaky test testConsumeFromMultipleTopics by @vmaheshw in https://github.com/linkedin/brooklin/pull/838
• Refactor Optional parameters in the constructor of strategy by @vmaheshw in https://github.com/linkedin/brooklin/pull/836
• Implemented logic to prevent tasks from having more than specified number of partitions by @jzakaryan in https://github.com/linkedin/brooklin/pull/837
• Publishing artifacts to JFrog by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/839
• Make the Throughput based assignment and task estimation based on partition assignment configurable. by @vmaheshw in https://github.com/linkedin/brooklin/pull/841
• Implemented metrics for LoadBasedPartitionAssignmentStrategy by @jzakaryan in https://github.com/linkedin/brooklin/pull/840
• Clear the CallbackStatus entry from the map in FlushlessEventProducerHandler by @vmaheshw in https://github.com/linkedin/brooklin/pull/843
• Handle the new session after session expiry by @vmaheshw in https://github.com/linkedin/brooklin/pull/770
• Fixed issue with config key by @jzakaryan in https://github.com/linkedin/brooklin/pull/847
• Print count of orphaned tasks and orphaned task locks in the log message by @surajkn in https://github.com/linkedin/brooklin/pull/844
• Datastream stop transition redesign by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/842
• Fix flaky stopping simultaneous datastream test by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/849
• Update the DatabaseChunkedReader to take the Connection as input rather than the DataSource by @somandal in https://github.com/linkedin/brooklin/pull/850
• Handle leaking TP exception in handleAssignmentChange by @surajkn in https://github.com/linkedin/brooklin/pull/845
• Migrating from AdminUtils with AdminClient by @surajkn in https://github.com/linkedin/brooklin/pull/848
• Fix running task data structure logic in AbstractKafkaConnector for failing to stop task by @surajkn in https://github.com/linkedin/brooklin/pull/851
• Removing partial Helix ZkClient dependency by @surajkn in https://github.com/linkedin/brooklin/pull/852
• Stop the tasks in parallel in AbstractKafkaConnector by @vmaheshw in https://github.com/linkedin/brooklin/pull/853
• Add additional log in LoadBasedPartitionAssignmentStrategy by @vmaheshw in https://github.com/linkedin/brooklin/pull/856
• Fixing flaky test testCreateDatastreamHappyPathDefaultRetention by @surajkn in https://github.com/linkedin/brooklin/pull/854
• Added toString() override in PartitionThroughputInfo by @jzakaryan in https://github.com/linkedin/brooklin/pull/858
• Add Stats to DatastreamTaskImpl by @vmaheshw in https://github.com/linkedin/brooklin/pull/855
• Make PartitionThroughputProvider metricsAware by @vmaheshw in https://github.com/linkedin/brooklin/pull/859
• Added base strategy metric info in LoadBasedPartitionAssignmentStrategy by @jzakaryan in https://github.com/linkedin/brooklin/pull/857
• Add alert metrics to identify that elastic task configurations require adjustment by @vmaheshw in https://github.com/linkedin/brooklin/pull/860
• Fix restartDeadTask logic when the task thread has died by @surajkn in https://github.com/linkedin/brooklin/pull/861
• Fix metric infos in PartitionThroughputProvider by @vmaheshw in https://github.com/linkedin/brooklin/pull/862
• Fix the metrics deregistration in AbstractKafkaConnector when multiple stop are called by @vmaheshw in https://github.com/linkedin/brooklin/pull/865
• Fix logging in LoadBasedPartitionAssignmentStrategy by @vmaheshw in https://github.com/linkedin/brooklin/pull/866
• Make Default byte in rate and Msg in rate configurable by @vmaheshw in https://github.com/linkedin/brooklin/pull/864
• Metrics are getting emitted in LoadBasedPartitionAssignmentStrategy only when it needs adjustment by @vmaheshw in https://github.com/linkedin/brooklin/pull/867
• Use topic level throughput information when partition level information is unavailable. by @vmaheshw in https://github.com/linkedin/brooklin/pull/871
• Fix compilation error by @vmaheshw in https://github.com/linkedin/brooklin/pull/874
• Loadbased Partition assigner not using topic level metrics to recognize partitions by @vmaheshw in https://github.com/linkedin/brooklin/pull/876
• Flushless producer supporting both comparable and non comparable offsets by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/873
• LiveInstanceProvider subscription should be done only by the leader coordinator by @vmaheshw in https://github.com/linkedin/brooklin/pull/879
• Fixed issue with missing exception message during task initialization by @jzakaryan in https://github.com/linkedin/brooklin/pull/882
• Kafka upgrade by @srinagaraj in https://github.com/linkedin/brooklin/pull/881
• skipping onPartitionsRevoked during consumer.close() call by @srinagaraj in https://github.com/linkedin/brooklin/pull/886
• Scala 2.12 upgrade by @srinagaraj in https://github.com/linkedin/brooklin/pull/895
• Upgrade avro and move jackson from codehaus to fasterxml by @vmaheshw in https://github.com/linkedin/brooklin/pull/894
• Fix topic deletion when multiple duplicate streams expire at the same time by @surajkn in https://github.com/linkedin/brooklin/pull/897
• Use 2.4.1.57 kafka version by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/901
• tests for min/max partitions per task metrics and minor code quality improvements by @ryannedolan in https://github.com/linkedin/brooklin/pull/887
• fix rebalancing-tasks bug and added tests by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/900
• Refactor Stopping Tasks On Assignment Change of Tasks by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/868
• Change python dependency in commit-msg git hook by @surajkn in https://github.com/linkedin/brooklin/pull/904
• Remove Scala Dependencies by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/905
• Introduce broadcast API to TransportProvider by @surajkn in https://github.com/linkedin/brooklin/pull/903
• Dedupe tasks on LeaderDoAssignment by @jogrogan in https://github.com/linkedin/brooklin/pull/906
• Fix Stopping Logic and Maintain Stopping Latch Counter by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/877
• Fixing test OnAssignmentChangeMultipleReassignments by @shrinandthakkar in https://github.com/linkedin/brooklin/pull/908
• Update kafka version by @jogrogan in https://github.com/linkedin/brooklin/pull/910
• Replace 101tec ZkClient with Helix ZkClient by @surajkn in https://github.com/linkedin/brooklin/pull/909
• Add retry to query retention time for destination topic by @surajkn in https://github.com/linkedin/brooklin/pull/863
• Upgrade Zookeeper version to 3.6.3 by @jogrogan in https://github.com/linkedin/brooklin/pull/913

New Contributors

• @surajkn made their first contribution in https://github.com/linkedin/brooklin/pull/844
• @srinagaraj made their first contribution in https://github.com/linkedin/brooklin/pull/881
• @ryannedolan made their first contribution in https://github.com/linkedin/brooklin/pull/887
• @jogrogan made their first contribution in https://github.com/linkedin/brooklin/pull/906

Full Changelog: https://github.com/linkedin/brooklin/compare/1.1.0...4.1.0