GPT-Load

Why it's ranked here

MIT licensed, shipped as a versioned container image with a three-command Compose setup, and wide on both sides of the gateway: nine client protocol families in, roughly thirty provider channels out. The engineering choices that show up in the code are the careful kind. It binds to loopback unless told otherwise, encrypts stored credentials with a local key, caps the size of catalogue downloads, and drains in-flight requests on shutdown with a forced exit on a second signal. The cost is a heavy dependency tree and a 2.0 release that cannot read 1.x data, so this is a strong pick for new installs and a migration project for old ones.

What's good

Safe defaults that hold up. The service and its login callback ports publish on 127.0.0.1 unless you change one setting, and the crash handler deliberately avoids logging request bodies or credentials. Session stickiness is a bounded memory cache of 10,000 entries with a one hour lifetime that wipes itself whenever the routing config changes, so stale rules cannot linger. The model catalogue syncs from Models.dev with conditional requests and a 32 MiB size cap. An optional automatic model picker asks a model to choose among presets you define and falls back cleanly if the answer is malformed, off-list or carries an out-of-range confidence.

Tradeoffs

The encryption key that protects stored credentials cannot be rotated in this version, and the key rotation command prints that it will arrive in a later release. Lose that key and every saved credential is unrecoverable. Version 2.0 cannot open or migrate 1.x data in place. The subscription logins use fixed callback ports, so only one default Compose instance fits on a host. Session stickiness lives in process memory, so separate replicas do not share it. The module pulls in a large dependency set, including a full WebRTC stack for Codex voice and an embedded third-party proxy wired in through a local replace.

How to use it well

Fits a small team or a single developer who juggles several provider keys and subscription seats and wants one endpoint, per-key access control and a usage ledger without writing glue. Start with the default SQLite store, back up the data volume as one unit since the database and encryption key only work together, and keep it on loopback or behind a proxy you trust, setting the trusted proxy list if you rely on forwarded client IPs. Point clients at it in their native protocol rather than translating. It is not a prompt manager, an evaluation tool or a multi-node load balancer.

Observed

License

MIT (README badge)

Language

Go, module declares go 1.27.2 (go.mod)

Distribution

Container image ghcr.io/tbphp/gpt-load:2 with Docker Compose (docker-compose.yml)

Storage

SQLite by default; MySQL or PostgreSQL via GORM drivers (go.mod, .env.example)

Client interfaces

OpenAI Chat, Responses, Images, Embeddings; Rerank; Mistral OCR and audio; Anthropic Messages; Gemini (README.md)

Default network exposure

Binds 127.0.0.1 unless HOST is changed (.env.example, docker-compose.yml)

Key rotation

Not supported; migrate-keys command is deferred (main.go, README.md)

Upgrade path

2.0 cannot open, import or migrate 1.x data in place (README.md)

Platform support

Windows service commands built into the binary (main.go)