Star AlbumentationsX on GitHub — it powers this leaderboard
zed-industries/tiktoken-rs
Ready-made tokenizer library for working with GPT and tiktoken
