llm-gauze
github.com/ggeorgovassilis/llm-gauze- Category
- Developer Tools
- Rank
- No. 2216Tools index
Previous survey · No. 2110 ·
- Pricing
- Open Source
- Type
- TOOL
- Use case
- Models: Train & Run
- Interfaces
- API
- GitHub
- 3 stars
- Latest release
- v0.12
- Date
About
An HTTP gateway that sits in front of a local LLM served through an OpenAI-compatible API and works around its failures. It handles transient errors without retries, hung or looping models, empty or sloppy responses, context-window overflows, runaway reasoning and malformed tool calls. It logs every exchange and fixes what it can before the client sees the response.
At a glance
llm-gauze sits between an OpenAI-compatible client and its model endpoint. It is worth evaluating if failures such as malformed tool calls or empty replies keep interrupting a local model workflow.
Good fit
Developers with a test model endpoint who can compare direct and proxied responses.
Consider something else when
Keep a direct connection if you cannot retain request logs or need every upstream failure returned unchanged.
Start here
Point a disposable client at a test upstream that returns an empty response. Check what the gateway returns, what it retries and what appears in its logs.
Cost and requirements
The MIT-licensed gateway documents a Docker Compose setup and needs a compatible upstream endpoint. Provider usage still costs whatever the upstream charges.
Limits to know
The gateway logs exchanges and can forward authorization headers. Review its logging and access settings before routing credentials or private prompts through it.
Sources checked 2026-11-05. Source-based assessment.
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.