Serverless inference is convenient for experiments and bursty traffic because idle capacity is the provider's problem. Billing is usually tied to requests, tokens, or execution time.

The tradeoffs can include cold starts, shared rate limits, less predictable tail latency, and less control over the exact hardware or runtime.