capy-flash
github.com/valeriodolci/capy-flash- Category
- Developer Tools
- Rank
- No. 1316Tools index
- Pricing
- Open Source
- Type
- TOOL
- Use case
- Models: Train & Run
- Interfaces
- API
- GitHub
- 40 stars
- Latest release
- v0.5.4
- Date
About
capy-flash is an inference engine that runs the 125B mixture-of-experts Qwen3.8-Flash-Next model in NVFP4 on two 16 GB consumer GPUs. It uses tensor parallelism across both cards and keeps experts that do not fit in VRAM in system RAM, computed on the CPU. It exposes OpenAI- and Anthropic-compatible APIs, and the README reports faster prefill and time to first token than a compared engine on the same hardware.
Tags
llm-inferencemoenvfp4cudaqwentensor-parallelism
Tech Stack
Docker
Media
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.