minbpe

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

10,337stars

Forks

1,012

Open issues

Watchers

10,337

Size

0.3 MB

PythonMIT License

Created: Feb 16, 2024

Updated: Feb 28, 2026

Last push: Jul 1, 2024