Tuhin Srivastava (Baseten) on Production Inference Economics

Baseten, a platform for running AI models, describes specialization as a process that starts with the customer's definition of a good result. In its founder's account, customers provide data, choose a base model and specify what training should optimize. Baseten supplies the training infrastructure and connects the resulting model to production serving.

The customer's contribution is knowledge of its product and users: the platform cannot decide which errors matter most. A menu-extraction case study from a different company's founder illustrates how business-specific corrections can become a training objective.

“there's language models, there's audio models, there's um there's actually I think three or four language models in the middle there and two audio models to make that happen and all of them run on base 10.”

Stanford Online

“Um and today about 90% or 95% of spend on inference is going to frontier models and about 5% is going to custom models”

Stanford Online

“open source models about 90 90 days behind um Frontier models and you can run them about 70 to 70 to 90% cheaper.”

Stanford Online

“the leading coding companies that are not the frontier model companies themselves are, you know, still rumored to be negative gross margin and so I imagine is existential for them to to to be a viable business”

Stanford Online

“we put out a post yesterday about the world of many models. We think intelligence shouldn't be owned by two people.”

Stanford Online