- Category
- AI Tools
- Pricing
- Open Source
- Type
- TOOL
- Builder
- microsoft
- GitHub
- 6.5k stars
- Added
- May 22, 2026
About
Microsoft's prompt and KV-cache compression for LLMs — up to 20x compression with minimal accuracy loss for cheaper, faster inference.
Why it made the leaderboard
Microsoft's prompt and KV-cache compression for LLMs — up to 20x compression with minimal accuracy loss, cutting cost and latency on long-context inference.
Tags
prompt-compressionllmmicrosoftinferenceefficiency
Tech Stack
Python
Comments (0)
No comments yet
Indexed by a proprietary survey. Corrections welcome.