Fishing For Magikarp Automatically Detecting Under Trained Tokens In Large Language Models 2024 09 24
Source
cohere.com
Author
Cohere
Date
Context
A LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition →'s vocabulary can contain text units it barely encountered during training. Cohere's research abstract identifies these under-trained tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → as a source of unwanted behavior. Being included in the tokenizer's vocabulary does not ensure that a model has learned how to handle a token reliably.
The authors report methods for finding such tokens through vocabulary analysis, indicators in model weights and prompting tests. This makes the mismatch a target for systematic inspection, rather than depending entirely on users stumbling across unusual inputs. The abstract reports detection methods, without establishing that every identified problem has been repaired.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
postCohere releases open-weights translation model North Small Translate
articleThe Data Provenance Initiative A Large Scale Audit Of Dataset Licensing And Attribution In Ai 2023 10 25
articleFishing For Magikarp Automatically Detecting Under Trained Tokens In Large Language Models 2024 05 08Cohere
articleBreaking Down Model Vocabulary Barriers With Tokenizer TransplantationArcee AI
articleOne Tokenizer To Rule Them All Emergent Language Plasticity Via Multilingual Tokenizers 2025 05 30Cohere