ShanHaiGateway

About

ShanHaiGateway is a Go gateway that combines several upstream LLM providers into a single OpenAI-compatible endpoint. It routes requests with smooth weighted round-robin, puts failing backends into exponential cooldown, and pins requests from the same job to the same backend so provider-side prefix caches keep getting reused. The README describes mixing free quotas, vouchers and paid channels to reduce token spend, but those cost and cache-hit figures are the author's own production claims.