Which AI inference provider should I use?
Use a router while models are unsettled, an inference cloud for open or custom-model performance, your existing cloud for governance, and a direct lab API when native features matter more than portability.
Surveyed 22 August 2026
Start here
Pick the layer first
- 01
Compare and switch models
- 02
Optimize open or custom models
- 03
Stay inside cloud controls
- 04
Get native features first
New here? Dotted terms open a plain-English definition. Also see , , and .
Reviewed catalog
Provider field guide
53 reviewed / 19 tracked
01
Routers
One account and request shape across multiple model labs or execution providers.
OpenRouterOne integration with broad model choice, explicit provider routing, and application-level fallbacks.
- Watch for
- A model name can resolve to several execution providers; pin routing and data policies when reproducibility matters.
- Model families
- Claude · GPT · Gemini · Llama · Qwen · DeepSeek · Mistral · Kimi
- Interface
- OpenAI-compatible
- Deployment
- Managed multi-provider routing
Hugging Face Inference ProvidersDiscovering and running open models across text, image, video, audio, and embedding tasks with one token.
- Watch for
- The OpenAI-compatible endpoint is chat-focused; non-chat tasks use Hugging Face's task-specific clients.
- Model families
- Llama · Qwen · DeepSeek · Gemma · FLUX · gpt-oss
- Interface
- OpenAI-compatible for chat
- Deployment
- Managed multi-provider routing
Vercel AI GatewayOne API for text, image, video, and audio models with routing, fallbacks, budgets, and AI SDK integration.
- Watch for
- Its strongest advantages sit inside the Vercel and AI SDK workflow; compare portability if that is not your stack.
- Model families
- Claude · GPT · Gemini · Grok · Llama · FLUX · Veo
- Interface
- OpenAI-compatible
- Deployment
- Managed multi-provider routing
02
Inference clouds
Specialists that run open or custom models.
Together AIOpen-model applications that may grow from serverless calls into fine-tuning or dedicated deployments.
- Watch for
- Its broad platform is more than a thin inference endpoint; compare only the pieces your workload will actually use.
- Model families
- Llama · Qwen · DeepSeek · Kimi · FLUX
- Interface
- OpenAI-compatible
- Deployment
- Serverless, dedicated, and custom
Fireworks AIProduction open models, model customization, and teams that need both serverless and dedicated serving.
- Watch for
- Performance depends on the exact model and deployment shape; a platform-wide speed claim is not a useful comparison.
- Model families
- Llama · Qwen · DeepSeek · Kimi · FLUX
- Interface
- OpenAI-compatible
- Deployment
- Serverless, on-demand, and dedicated
GroqInteractive text and speech workloads where low and predictable generation latency is the deciding constraint.
- Watch for
- The catalog is deliberately narrower than a marketplace; confirm the exact model and context requirements first.
- Model families
- Llama · Qwen · gpt-oss · Whisper
- Interface
- OpenAI-compatible
- Deployment
- Managed accelerator cloud
CerebrasHigh-throughput open-model text generation when time-to-first-token and generation speed dominate the decision.
- Watch for
- A focused accelerator catalog is not a replacement for a broad multimodal provider.
- Model families
- Llama · Qwen · gpt-oss
- Interface
- OpenAI-compatible
- Deployment
- Managed accelerator cloud
DeepInfraA broad hosted open-model catalog spanning language, embeddings, reranking, image, and audio workloads.
- Watch for
- Breadth is not uniformity; capabilities and parameters vary substantially between model endpoints.
- Model families
- Llama · Qwen · DeepSeek · Gemma · FLUX · Whisper
- Interface
- OpenAI-compatible for selected models
- Deployment
- Serverless and dedicated
BasetenDeploying custom or fine-tuned models with explicit control over runtimes, autoscaling, and production operations.
- Watch for
- It is deployment infrastructure first, not the simplest route to sampling hundreds of third-party APIs.
- Model families
- Custom weights · Llama · Qwen · DeepSeek · FLUX
- Interface
- Provider-specific APIs
- Deployment
- Serverless, chains, and dedicated
ReplicateTrying and shipping versioned community models, especially image, video, audio, and other non-chat workloads.
- Watch for
- Model-owned interfaces vary; portability is weaker than on a uniform chat-completions catalog.
- Model families
- FLUX · Stable Diffusion · Whisper · Llama · Community models
- Interface
- Provider-specific APIs
- Deployment
- Public models and private deployments
falGenerative image, video, and audio pipelines where media model breadth and queueing matter more than chat APIs.
- Watch for
- It belongs in a media-inference comparison, not as a default general-purpose LLM provider.
- Model families
- FLUX · Stable Diffusion · Kling · Veo · Wan
- Interface
- Provider-specific APIs
- Deployment
- Serverless media inference
ModalOwning the inference code and runtime while keeping GPU infrastructure serverless and Python-native.
- Watch for
- You operate the serving code; it is not a one-key model marketplace or a direct lab API.
- Model families
- Custom weights · vLLM · SGLang · Diffusers
- Interface
- Provider-specific APIs
- Deployment
- Serverless custom containers
Nebius Token FactoryHosted open models with a path from token API usage to fine-tuning and dedicated endpoints.
- Watch for
- Model flavors, endpoint configuration, and regional availability should be evaluated together.
- Model families
- Llama · Qwen · DeepSeek · Mistral · FLUX
- Interface
- OpenAI-compatible
- Deployment
- Serverless and dedicated
Novita AIA mixed text, image, video, and GPU platform when one vendor must cover several generative modalities.
- Watch for
- Compare the service and data path per modality; a broad catalog does not imply one consistent runtime.
- Model families
- Llama · Qwen · DeepSeek · FLUX · Video models
- Interface
- OpenAI-compatible for selected models
- Deployment
- Serverless APIs and GPU instances
ChutesOpen-model experimentation and deployments that value an open, distributed inference marketplace.
- Watch for
- Treat hardware provenance, reliability, and data handling as first-class evaluation questions.
- Model families
- Llama · Qwen · DeepSeek · Community models
- Interface
- OpenAI-compatible for selected models
- Deployment
- Distributed serverless inference
12 more inference clouds
NVIDIA API CatalogTrying NVIDIA-hosted APIs with a path to self-hosting the same optimized NIM containers.
- Watch for
- Hosted catalog access and production NIM deployment are distinct products with different operational commitments.
- Model families
- Llama · Qwen · Nemotron · Mistral · Embedding · Reranking
- Interface
- OpenAI-compatible for selected models
- Deployment
- Hosted APIs and self-hosted NIM
SambaNova CloudFast hosted open-model inference on SambaNova's purpose-built dataflow hardware.
- Watch for
- The available catalog is narrower than a general marketplace and can change independently of open-weight releases.
- Model families
- Llama · DeepSeek · Qwen · gpt-oss
- Interface
- OpenAI-compatible
- Deployment
- Managed accelerator cloud
Featherless AISampling a long tail of open text models through a simple subscription-oriented API.
- Watch for
- Catalog breadth includes niche models with uneven tool support, latency, and production readiness.
- Model families
- Llama · Qwen · DeepSeek · Mistral · Community models
- Interface
- OpenAI-compatible
- Deployment
- Serverless open-model inference
HyperbolicHosted open-model inference plus on-demand GPU access in the same compute marketplace.
- Watch for
- Evaluate the managed inference surface separately from raw GPU rentals and community compute supply.
- Model families
- Llama · Qwen · DeepSeek · FLUX
- Interface
- OpenAI-compatible for selected models
- Deployment
- Serverless inference and GPU marketplace
NscaleEuropean AI infrastructure spanning hosted inference, fine-tuning, and dedicated GPU capacity.
- Watch for
- Check regional and product availability because serverless endpoints and dedicated infrastructure differ.
- Model families
- Llama · Qwen · DeepSeek · FLUX
- Interface
- OpenAI-compatible for selected models
- Deployment
- Serverless and dedicated GPU cloud
OVHcloud AI EndpointsEuropean-hosted open models with OVHcloud billing, infrastructure, and data-location options.
- Watch for
- The model catalog and endpoint features are smaller than the largest global inference marketplaces.
- Model families
- Llama · Mistral · Qwen · Embedding · Speech
- Interface
- OpenAI-compatible for selected models
- Deployment
- Managed European cloud endpoints
Scaleway Generative APIsEuropean-hosted language and embedding APIs with straightforward cloud integration.
- Watch for
- Its focused catalog trades breadth for regional infrastructure and simpler governance.
- Model families
- Llama · Mistral · Qwen · Embedding
- Interface
- OpenAI-compatible
- Deployment
- Managed European cloud endpoints
WaveSpeedAIImage and video generation APIs where media-model choice and generation latency matter.
- Watch for
- This is a specialist media provider, not a substitute for a general-purpose text inference layer.
- Model families
- FLUX · Kling · Wan · Hunyuan Video · Image models
- Interface
- Provider-specific APIs
- Deployment
- Serverless media inference
FriendliAIServing open or custom language models through managed endpoints and optimized dedicated deployments.
- Watch for
- Confirm which model families are available serverlessly versus only through custom endpoints.
- Model families
- Llama · Qwen · DeepSeek · Custom weights
- Interface
- OpenAI-compatible for selected models
- Deployment
- Serverless and dedicated endpoints
SiliconFlowBroad access to Chinese and global open models through regional API surfaces.
- Watch for
- Regions, model availability, billing, and data-handling terms must be checked for the exact endpoint.
- Model families
- Qwen · DeepSeek · GLM · Llama · FLUX
- Interface
- OpenAI-compatible for selected models
- Deployment
- Managed regional inference
ClarifaiComposing hosted, third-party, and custom models into governed multimodal AI workflows.
- Watch for
- Its application and workflow platform is broader than inference, so compare the serving path you will actually use.
- Model families
- Llama · Qwen · Vision · Audio · Custom models
- Interface
- Provider-specific APIs
- Deployment
- Managed, dedicated, and on-premises
Public AI Inference UtilityLow-cost access to open public-interest models from initiatives such as Swiss AI and AI Singapore.
- Watch for
- The nonprofit catalog is intentionally selective and should be evaluated for capacity and production support needs.
- Model families
- Apertus · SEA-LION · OLMo · Public open models
- Interface
- OpenAI-compatible
- Deployment
- Nonprofit managed inference utility
03
Cloud catalogs
Model access inside a larger cloud security and procurement boundary.
Cloudflare Workers AIAI calls embedded in Workers applications, with edge delivery, gateway controls, and Cloudflare-native operations.
- Watch for
- The selected catalog and platform constraints matter more than raw access to the newest frontier model.
- Model families
- Llama · Qwen · Mistral · Whisper · Embedding models
- Interface
- OpenAI-compatible for selected models
- Deployment
- Cloudflare-managed edge platform
Amazon BedrockOrganizations that need multiple model labs inside AWS identity, networking, governance, and procurement.
- Watch for
- Model availability, features, and data residency are region-specific; the AWS boundary is part of the choice.
- Model families
- Claude · Llama · Mistral · Amazon Nova · Cohere · DeepSeek
- Interface
- Provider-native API
- Deployment
- Managed and provisioned throughput
Google Vertex AIGemini and partner or open models governed alongside data and applications already running in Google Cloud.
- Watch for
- AI Studio and Vertex AI are different developer surfaces; production governance usually points to Vertex.
- Model families
- Gemini · Gemma · Claude · Llama · Mistral
- Interface
- OpenAI-compatible for selected models
- Deployment
- Managed APIs and dedicated endpoints
Azure AI FoundryMicrosoft-cloud teams that want Azure OpenAI and a wider model catalog under enterprise identity and controls.
- Watch for
- APIs, deployment types, and regional availability vary by model publisher; verify the exact route before standardizing.
- Model families
- GPT · Phi · Llama · Mistral · Cohere · DeepSeek
- Interface
- Provider-specific APIs
- Deployment
- Managed and provisioned deployments
5 more cloud catalogs
Alibaba Cloud Model StudioQwen and partner models inside Alibaba Cloud, especially for workloads serving Asian regions.
- Watch for
- International and mainland-China regions can differ in endpoints, catalogs, billing, and governance.
- Model families
- Qwen · DeepSeek · Kimi · Embedding · Image and video
- Interface
- OpenAI-compatible for selected models
- Deployment
- Managed and dedicated cloud endpoints
Databricks Mosaic AIServing foundation, fine-tuned, and custom models next to governed enterprise data in Databricks.
- Watch for
- Its value depends heavily on adopting the wider Databricks data and governance boundary.
- Model families
- Llama · DBRX · Qwen · Embedding · Custom models
- Interface
- OpenAI-compatible for selected models
- Deployment
- Pay-per-token and provisioned endpoints
IBM watsonx.aiRegulated enterprises standardizing models, governance, and deployment through IBM's AI platform.
- Watch for
- The enterprise platform and contract are the differentiator; its public model catalog is not the broadest.
- Model families
- Granite · Llama · Mistral · Embedding
- Interface
- Provider-native API
- Deployment
- Managed and dedicated enterprise deployments
OCI Generative AIManaged and dedicated generative models inside Oracle Cloud networking, identity, and procurement.
- Watch for
- Model and dedicated-cluster availability is region-specific and narrower than multi-cloud marketplaces.
- Model families
- Cohere Command · Llama · Embedding
- Interface
- Provider-native API
- Deployment
- On-demand and dedicated AI clusters
Snowflake Cortex AIRunning governed language and embedding functions directly against data already in Snowflake.
- Watch for
- It is optimized for in-platform data workflows, not as a universal external application inference API.
- Model families
- Claude · Llama · Mistral · Snowflake Arctic · Embedding
- Interface
- Provider-specific APIs
- Deployment
- Snowflake-managed SQL and REST functions
04
Direct lab APIs
The model maker is also the API provider.
OpenAI APINative access to OpenAI's models, tools, and Responses platform without a third-party routing layer.
- Watch for
- Build an adapter boundary if the application may need another lab; native features can deepen lock-in quickly.
- Model families
- GPT · o-series · gpt-oss · Image · Audio · Embedding
- Interface
- Provider-native API
- Deployment
- Direct managed API
Anthropic APINative Claude features, long-running agent work, and direct access to Anthropic's model and tool surface.
- Watch for
- Its Messages API is not an OpenAI clone; use the native contract deliberately or isolate it behind your own interface.
- Model families
- Claude Opus · Claude Sonnet · Claude Haiku
- Interface
- Provider-native API
- Deployment
- Direct managed API
Google AI StudioThe shortest developer path to Gemini's native multimodal and generative-media capabilities.
- Watch for
- Move to Vertex AI when cloud governance, private networking, or enterprise controls become requirements.
- Model families
- Gemini · Imagen · Veo · Embedding
- Interface
- OpenAI-compatible for selected models
- Deployment
- Direct managed API
Mistral AIDirect access to Mistral's open and commercial model families, including multilingual and code-focused work.
- Watch for
- Separate what is open-weight from what is API-only; the licensing and deployment options are model-specific.
- Model families
- Mistral · Mixtral · Codestral · Ministral · Pixtral
- Interface
- OpenAI-compatible
- Deployment
- Direct API and deployable weights
xAI APINative Grok access and workloads that specifically need xAI's model or real-time product surface.
- Watch for
- Do not treat access to current information as a substitute for explicit source retrieval and citations.
- Model families
- Grok
- Interface
- OpenAI-compatible
- Deployment
- Direct managed API
CohereEnterprise retrieval systems that want generation, embeddings, and reranking designed as one stack.
- Watch for
- Its edge is retrieval infrastructure, not winning every general-purpose frontier-model comparison.
- Model families
- Command · Embed · Rerank
- Interface
- Provider-native API
- Deployment
- Direct API and private deployments
DeepSeek APIDirect access to DeepSeek's reasoning and chat models without an intermediary host.
- Watch for
- Availability, policy, and operational requirements may differ from third-party hosts serving the same open weights.
- Model families
- DeepSeek Chat · DeepSeek Reasoner
- Interface
- OpenAI-compatible
- Deployment
- Direct API and open weights
Z.aiDirect GLM access, particularly for coding, agent, and multilingual workloads centered on that family.
- Watch for
- Product names, regions, and billing surfaces can differ; verify which endpoint and organization contract you are using.
- Model families
- GLM
- Interface
- OpenAI-compatible for selected models
- Deployment
- Direct managed API and open weights
9 more direct lab apis
Moonshot AIDirect access to Kimi models, including long-context, reasoning, and agent-oriented releases.
- Watch for
- Regional product surfaces and model availability differ; verify whether you are using the global or China platform.
- Model families
- Kimi
- Interface
- OpenAI-compatible
- Deployment
- Direct managed API and open weights
MiniMaxOne lab's native text, speech, music, image, and video generation APIs.
- Watch for
- Capabilities, pricing, and data paths differ substantially across modalities and regional endpoints.
- Model families
- MiniMax M-series · Speech · Music · Image · Video
- Interface
- OpenAI-compatible for selected models
- Deployment
- Direct managed multimodal APIs
Perplexity APISearch-grounded answers with citations through Perplexity's Sonar models and search APIs.
- Watch for
- This is a retrieval product as much as a model API; evaluate source quality and citation coverage, not just prose quality.
- Model families
- Sonar · Search API · Embeddings
- Interface
- OpenAI-compatible for selected models
- Deployment
- Direct managed search and answer APIs
AI21Direct access to AI21's Jamba family and enterprise language-model deployment options.
- Watch for
- Its focused catalog makes sense when Jamba is the reason for choosing the provider.
- Model families
- Jamba
- Interface
- Provider-native API
- Deployment
- Direct API and enterprise deployments
Voyage AISpecialized embeddings and rerankers for retrieval, code, finance, legal, and multilingual corpora.
- Watch for
- It is a retrieval-model specialist, not a general text-generation provider.
- Model families
- Voyage Embed · Voyage Rerank
- Interface
- Provider-native API
- Deployment
- Direct managed API
Black Forest LabsNative access to FLUX image generation and editing models from their creator.
- Watch for
- This is an image-specialist API; workflows needing video, speech, or language models need another provider.
- Model families
- FLUX
- Interface
- Provider-specific APIs
- Deployment
- Direct managed image API
DeepgramReal-time speech recognition, text-to-speech, and voice-agent audio infrastructure.
- Watch for
- It solves the voice layer, not the reasoning-model layer behind a complete voice agent.
- Model families
- Nova · Aura · Flux voice agents
- Interface
- Provider-specific APIs
- Deployment
- Direct cloud API and self-hosted enterprise
ElevenLabsExpressive speech generation, dubbing, transcription, music, and conversational voice agents.
- Watch for
- Voice quality, licensing, latency, and consent controls need separate evaluation for each production use.
- Model families
- Eleven TTS · Scribe · Music · Conversational AI
- Interface
- Provider-specific APIs
- Deployment
- Direct managed audio APIs
Stability AINative access to Stable Diffusion image generation and editing APIs from the model lab.
- Watch for
- Licensing and deployable-weight terms vary by model version and use case.
- Model families
- Stable Diffusion · Stable Image
- Interface
- Provider-specific APIs
- Deployment
- Direct API and deployable weights
How providers make the list +
Coverage combines major routers, inference clouds, cloud catalogs, and model labs with a documented direct API. We do not create a separate entry for every reseller, regional alias, or invisible backend in a routing network. Inclusion is not an endorsement; prices, model versions, benchmarks, and availability require a separate dated comparison.
Building with an agent? Use the stable, machine-readable provider registry (JSON).
How to compare providers +
- 01Which layer are you buying: model optionality, execution performance, cloud governance, or direct access to a lab's native features?
- 02Does the exact model support every parameter your application relies on — tools, structured output, multimodal input, caching, and the required context window?
- 03Can you pin the execution provider, region, data-retention policy, and fallback order when reproducibility or compliance matters?
- 04What do p50 and p95 time-to-first-token and generation speed look like on your prompts, not on the provider's showcase benchmark?
- 05Can your application fail over without changing semantics, or have provider-specific features leaked through the abstraction?
Common questions +
- Should I use a router or call a model lab directly?
- Use a router while you are comparing models or need cross-provider fallbacks. Call a lab directly when its native features, support boundary, or release-day access are worth the tighter dependency.
- Is an OpenAI-compatible API actually portable?
- Only at the basic request-shape level. Tool schemas, structured output, reasoning controls, caching, media inputs, error behavior, and streaming events still vary, so test the features your application uses.
- What should I benchmark before choosing a provider?
- Run your own prompts and record success rate, p50 and p95 latency, output tokens per second, total cost per successful task, rate-limit behavior, and recovery when the preferred endpoint fails.
More in Understand the AI landscape
- Find AI benchmarksBrowse the evaluations used to compare models and agents — coding, tool use, cost, and the domain-specific long tail.
- Evaluate an LLM applicationBuild test sets, score outputs, and catch quality regressions.
- Observe an LLM applicationTrace calls, inspect failures, and monitor latency, quality, and spend.
- Run models locallyUse local inference runtimes and model managers on your own hardware.
- Add vector searchStore embeddings and retrieve relevant context for AI applications.