WeClone

🎉 What's Changed

The key highlights of this update include an upgrade to Python 3.12 and optimization of the dataset pipeline.

Dependency and Environment Updates:

• Upgraded the required Python version to 3.12 in pyproject.toml and development settings, and updated the target version for linting and type checking to Python 3.12. [[1]](diffhunk://#diff-50c86b7ed8ac2cf95bd48334961...

🎉 What's Changed

Enable configurable thinking in offline cleaning, improve image and gif handling in QA processing, refactor configuration models for cleaner dataset naming, and bump versions and dependencies for release v0.3.02

New Features:

• Introduce enable_thinking flag in LLMCleanConfig to control offline cleaning behavior
• Supporting scoring and cleaning of datasets containing...

🎉 What's Changed

New Features:

• Added retry_on_http_error and retry_openai_api decorators with backoff strategy to implement automatic retry mechanism for online LLM calls
• feat(dataset): Enhance training data by including time-related options.

Enhancements:

• performance(PII): add batch PII detection to improve performance
• refactor: unifies combined content separator \n
• feat...

🎉 What's Changed

• Support fine-tuning of Telegram chat logs
• Utilize presidio for privacy filtering
• Added multilingual support configuration
• Optimized vllm inference and decoding parsing
• Optimized the logging system, hooked other dependency logging, and added log level configuration
• Translated log printing and code comments to English
• Migrated commentjson dependency to...

🥰 What's Changed

• Update torch version to 2.7.0 and vllm version to 0.9.1, switch offline inference to chat-style invocation
• Add test_model_args and vllm_args configuration items to allow custom test dataset files
• Add config file path option in CLI, support setting WECLONE_CONFIG_PATH environment variable
• Update max_new_tokens and enable_thinking parameters in data cleaning stra...