Baseten's Mammoth-1 and Elephant-1 now available on AI Gateway for free

Vercel has put two models from Baseten, the 27-billion-parameter Mammoth-1 and the smaller 9-billion-parameter Elephant-1, on its AI Gateway free for one week, through October 27. Neither is a chatbot. They are decision models: instead of generating text, they answer questions you define about a piece of input, and return the answer together with a probability for each possible option. Vercel's documentation describes three question types. A Boolean question returns the probability that something is true, a Choice question picks one option from a named set, such as which support team should get a ticket, and a Score question rates the input along an ordered scale. Several questions about the same input can go in one request, and both models have a 16k-token context window.

The probabilities are the practical point. Because each answer comes with a confidence figure, an application can set its own threshold: act automatically when the model is sure, and send the case to a person when it isn't. The gateway can also be set up to rerun a decision on another model when the first answer's confidence is low, an opt-in feature that bills both calls. That suits high-volume sorting jobs, like routing support tickets or checking whether a message asks for a refund. As Artificial Analysis notes, a general-purpose language model can answer the same questions with structured output, but it names one option without a probability for each, which is the gap decision models are built to fill.

Vercel says Mammoth-1 is first on the Jev Decision Index and Elephant-1 is tied for first on the Artificial Analysis Decision Index, but the post gives no scores or dates for either ranking. Artificial Analysis built its index, explained in its launch article, from four tests covering knowledge and hallucination, multiple-choice reasoning, deciding when to call a tool, and safety classification. That is a reasonable spread, but it is not your workload, and the free week is best used to check the models against your own labeled examples. The post does not say what the models will cost after October 27.