Tokens too cheap to meter

• inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
• token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Reframes cost planning for AI-heavy products: if tokens keep getting cheaper faster than expected, architecture and pricing decisions built around per-token caution may become obsolete quickly.

• articleManaging a token budget? Think efficiency, not intelligenceWriter Team
• articleWhy the cheap model sometimes costs more than the expensive oneWRITER
• podcastWhy Smarter AI Models Could Drive Up Compute Prices 10xDwarkesh Patel

Checking sign-in…

Loading comments…