InferPilot
github.com/poojithdevan4d/inferpilot- Category
- Developer Tools
- Rank
- No. 2580Tools index
- Pricing
- Open Source
- Type
- TOOL
- Use case
- Models: Train & Run · Deployment & Operations
- GitHub
- 2 stars
- Date
About
InferPilot reads a running vLLM server's live Prometheus metrics and diagnoses the real inference bottleneck, including whether a change such as fp8 KV cache will help or do nothing. It diagnoses only when aligned evidence supports a conclusion and otherwise abstains. It is MIT-licensed with no telemetry.
Tags
vllmllm-inferenceprometheusdiagnosticsfp8optimization
Tech Stack
Python
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.