- Category
- Developer Tools
- Rank
- No. 1770Tools index
- Pricing
- Open Source
- Type
- TOOL
- GitHub
- 10 stars
- Latest release
- v0.3.0
- Date
About
rigtune is a Python tool that benchmarks LLM serving settings on your own GPU and prompts, then serves the fastest configuration behind an OpenAI-compatible API. It tries engines (vLLM, SGLang, llama.cpp), weight precisions, KV-cache types and speculative decoding, and checks answer quality on GSM8K. The author reports speedups of 1.5x to 2.26x over stock settings on three models, with stated caveats.
Tags
llminferencevllmsglanggpubenchmarking
Tech Stack
PythonDocker
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.