AI Gateway adds confidence-based decision fallbacks
Source
vercel.com
Author
Rohan Taneja
Date
Why it matters
You can route low-confidence decisions to a stronger model through one providerOptions entry, instead of only falling back on errors. Escalations bill both stages, so cost planning matters.
Key takeaways · AI-distilled
Existing fallbacks keep working: plain model names already listed in models still catch outright errors, and requests without the new conditional object keep their prior behavior.
Confidence conditions apply to Choice and Score questions; Boolean questions instead escalate on a probability range instead.
A fallback is one object in providerOptions.gateway.models with a model plus a when clause, such as question "intent" and confidenceBelow 0.6; omit question and the threshold applies to every Choice or Score answer.
