LMCache
github.com/lmcache/lmcache- Category
- AI Tools
- Pricing
- Open Source
- Platform
- cli
- Type
- TOOL
- GitHub
- 11.0k stars
- Added
- Jun 15, 2026
About
A KV cache management layer that accelerates LLM inference by turning temporary cache into reusable knowledge that persists across sessions. Reduces time-to-first-token and improves throughput for long-context and multi-turn conversations.
Why it made the leaderboard
LMCache addresses a fundamental infrastructure challenge in LLM inference by making KV caches persistent and reusable across engines, which is genuinely differentiated from existing tools. The vendor-neutral approach and strong industry adoption (PyTorch Foundation, NVIDIA integration, 8k+ stars) demonstrates real production value beyond typical AI wrappers.
Tags
llminferencecacheperformancepytorchoptimizationai-infrastructure
Tech Stack
Python
Comments (0)
No comments yet
Indexed by a proprietary survey. Corrections welcome.