Vibeleaderboard
Index / tool
Visit github.com
Category
Developer Tools
Rank
No. 2216Tools index

Previous survey · No. 2110 ·

Pricing
Open Source
Type
TOOL
Use case
Models: Train & Run
Interfaces
API
GitHub
3 stars
Latest release
v0.12
Date

About

An HTTP gateway that sits in front of a local LLM served through an OpenAI-compatible API and works around its failures. It handles transient errors without retries, hung or looping models, empty or sloppy responses, context-window overflows, runaway reasoning and malformed tool calls. It logs every exchange and fixes what it can before the client sees the response.

At a glance

llm-gauze sits between an OpenAI-compatible client and its model endpoint. It is worth evaluating if failures such as malformed tool calls or empty replies keep interrupting a local model workflow.

Good fit

Developers with a test model endpoint who can compare direct and proxied responses.

Consider something else when

Keep a direct connection if you cannot retain request logs or need every upstream failure returned unchanged.

Start here

Point a disposable client at a test upstream that returns an empty response. Check what the gateway returns, what it retries and what appears in its logs.

Cost and requirements

The MIT-licensed gateway documents a Docker Compose setup and needs a compatible upstream endpoint. Provider usage still costs whatever the upstream charges.

Limits to know

The gateway logs exchanges and can forward authorization headers. Review its logging and access settings before routing credentials or private prompts through it.

Tags

llmgatewayopen-weightopenai-compatibleproxylocal-llm

Tech Stack

PythonDocker

Comments (0)

No comments yet

Editorially curated, with community endorsements as a secondary signal. Corrections welcome.