New
Unclaimed project
Are you a maintainer of tokenizers? Claim this project to take control of your public changelog and roadmap.
Changelog
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Full Changelog: https://github.com/huggingface/tokenizers/compare/v0.21.2...v0.21.3