trl

What's Changed

• Remove access to warnings_issued by @qgallouedec in #4960
• Fix SFTTrainer init logic: remove TrainingArguments.push_to_hub_token only for transformers < v5 by @albertvillanova in #4942
• Fix extra EOS appended in DPO preprocessing for conversational data by @qgallouedec in https://github.com/huggingface/trl/pull/4908

Full Changelog: https://github.com/huggingface...

What's Changed

• Fix: undefined current_gradient_accumulation_steps by @qgallouedec in https://github.com/huggingface/trl/pull/4852
• fix(DeepSeek OPSM): passing correct (vLLM) logprobs by @casinca in https://github.com/huggingface/trl/pull/4857
• Fix SFT training for prompt-completion type and transformers v5 by @qgallouedec in https://github.com/huggingface/trl/pull/4880
• Bugfix: Log...

Features

• Add vllm_group_port argument to GRPO, RLOO and OnlineDPO configuration by @pointerhacker in https://github.com/huggingface/trl/pull/4545
• Preserve truncated tokens in BFD packing by @qgallouedec in https://github.com/huggingface/trl/pull/4632
• Support async reward functions and parallelize call to reward functions. by @pramodith in https://github.com/huggingface/trl/pull/456...

What's Changed

• Overwrite model default generation config used by model.generate by @albertvillanova in https://github.com/huggingface/trl/pull/4647

Full Changelog: https://github.com/huggingface/trl/compare/v0.26.1...v0.26.2

What's Changed

• Fix vLLM error for tools usage not supported when running GRPO training by @apalmas-saifh in https://github.com/huggingface/trl/pull/4663
• Fix GRPO config validation in case num_generations_eval is specified and different than num_generations by @apalmas-saifh in https://github.com/huggingface/trl/pull/4682

New Contributors

• @apalmas-saifh made their first co...