New
Unclaimed project
Are you a maintainer of tokenizers? Claim this project to take control of your public changelog and roadmap.
Changelog
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Full Changelog: https://github.com/huggingface/tokenizers/compare/v0.21.3...v0.21.4
No change, the 0.21.3 release failed, this is just a re-release.
https://github.com/huggingface/tokenizers/releases/tag/v0.21.3