Ollama served my 40k-context model at 4k, silently
Source
github.com
Author
satoshiakiyama
Date
Why it matters
Two silent measurement failures that would quietly invalidate a local benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition →, one of them in configuration nobody set.
Terms in this piece · Glossary
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
articleBroken on Arrival: Silently Defective LLM Artifacts in Public Model Registries and How to Catch ThemAditi Patodiya
articleQwen 3.8 27B is excellent, but it defaults to wildly overthinking thingssimonwillison.net