Vibeleaderboard
Index / tool

Inference Stack

www.inferencestack.io
Visit www.inferencestack.io
Category
Developer Tools
Rank
No. 4544Tools index
Pricing
Open Source
Type
TOOL
Date

About

Inference Stack is an MIT-licensed bundle that deploys a self-hosted LLM serving setup with one install command, using Docker Compose on one machine or Helm on Kubernetes. It combines vLLM for running models on NVIDIA GPUs, LiteLLM for a single API with keys, teams, and budgets, llm-d for choosing a replica per request from cache and queue data, and Grafana dashboards for health and traffic. A simulation mode returns canned responses for testing without a GPU.

Intel on Inference Stack

More in Intel

Tags

llminferencevllmlitellmkubernetesself-hosted

Media

Inference Stack

Comments (0)

No comments yet

Editorially curated, with community endorsements as a secondary signal. Corrections welcome.