AI Gateway adds confidence-based decision fallbacks

Vercel's AI Gateway can now send a decision request to a fallback model when the first model's answer is not confident enough. According to the changelog, a developer adds a condition to the gateway's list of models, such as escalating to a backup when the answer to a named question has a confidence below 0.6. Its example uses an OpenAI model as the backup.

This is for decision requests, where the model answers a defined question with a choice, a score or a true-or-false probability instead of text. Confidence conditions work on choice and score questions, while true-or-false questions escalate on a probability range. Leaving out the question name applies the condition to every question of that type. Existing fallbacks that cover outright errors keep working unchanged.

The practical tradeoff is cost: a triggered fallback runs a second decision and bills both stages. The feature is in beta, and Vercel's post gives no data on how often escalation happens or how much accuracy it adds.