← Back to Leaderboard
Developer ToolsTOOL
About
OpenAI's fast BPE tokenizer — count tokens, encode/decode for GPT models, written in Rust with Python bindings.
Tags
tokenizerbpeopenaillmpython
Tech Stack
PythonRust
Comments
No comments yet.
About
OpenAI's fast BPE tokenizer — count tokens, encode/decode for GPT models, written in Rust with Python bindings.
Tags
Tech Stack
No comments yet.