inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Why it matters
Agentic workloads and code completion make opposite demands on an inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → stack, throughput versus latency, and that tension, not raw speed, is what shapes a serving codebase used for both training-time evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → and production.
