The Cheaper AI Model That Costs You More - Rayan Krishnan, Vals AI
Source
youtube.com
Author
AI Engineer
Date
Why it matters
Price per tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → is a poor proxy for cost per task: a cheaper model that uses more tokens can cost more and score lower. Building evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → from your own work history is how one team cut spend about 90%.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
videoYour Agent Just Paid for Nothing: 6 Attacks on x402 Payments - Ishween Kaur, SoFi
videoWhy Humans Can't Keep Up With AI-Written Code - Tariq Shaukat, Sonar
videoHow Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models - David Abitbol, Revolut
videoWe Trust AI Agents That Break Every Day - Shy Ruparel, Temporal
