Vibeleaderboard
← All Intel
Intel / article

Service tiers now available on AI Gateway

Source
vercel.com
Author
Walter Korman
Date

Context

Vercel's AI Gateway service tiers let an application request different processing tradeoffs for different jobs. The announcement describes priority processing at a higher price and flex processing at a lower price with potentially longer waits. That gives interactive requests and background work different options on supported models.

A requested tier is best effort. If it cannot be applied, the request uses default processing and default pricing. The response reports the tier actually used, so applications can compare observed speed and cost against the service delivered.

Key quotes

“Service tiers let you optimize for latency, throughput, and cost per request to match your use case.”

“Pick a faster tier for interactive workloads (less queueing, higher token throughput), or a lower cost tier for background jobs that can tolerate more latency.”

“Service tier is serviced on a best-effort basis: if a tier can't be applied, the request runs on the default tier at the default rate, and only an invalid service tier value fails the request.”

“If a priority request gets downgraded to default capacity, billing reflects the default rate, not the priority rate.”

More from Walter Korman
Recommended reads
Comments

Checking sign-in…

Loading comments…