inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Why it matters
If you build on Claude, this explains concretely how inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition →-stack issues — routing, precision, and compiler-level bugs — can silently degrade output quality without any model change, and why such regressions are so hard to detect with standard evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition →. It's a rare first-party account of production LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → serving failure modes you should account for in your own monitoring and evals.
Key quotes
“We never reduce model quality due to demand, time of day, or server load.”
“The problems our users reported were due to infrastructure bugs alone.”
“A small subset of users that asked a question in English might have seen "สวัสดี" in the middle of the response, for example.”
“Model quality is non-negotiable, so we accepted the minor efficiency impact.”
