Poolside makes model evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → part of its training infrastructure so that testing can start when a new saved model version becomes available. Its Model Factory account describes automatically launching the model-serving jobs needed for those tests and sending results to a dashboard.
Each evaluation is a separately scheduled job. That lets the system allocate computing resources to individual tests and run comparisons across saved model versions without treating an entire evaluation suite as one large reservation. The useful design principle is to make evaluation repeatable infrastructure, reducing the manual coordination needed to learn whether a training experiment helped.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
