- Category
- Developer Tools
- Rank
- No. 2952Tools index
Previous survey · No. 2811 ·
- Listed in
- #1 Interface with your agents
- Pricing
- Open Source
- Type
- TOOL
- Use case
- Models: Train & Run
- GitHub
- 2.5k stars
- Latest release
- v0.8.1
- Date
About
WARP (Weight-Aware Runtime and Paging, formerly WASTE) is a dependency-free, embeddable C inference engine that keeps a model's trunk in memory while streaming selected expert weights directly from NVMe disk, using remaining RAM as a bounded expert cache. This lets it run large mixture-of-experts models such as Kimi K3, DeepSeek V4.1 Flash, and GLM-5.3-Flash on hardware with far less RAM than the full model would normally require.
What it does
A local runtime for huge sparse language models whose weights sit mostly on a fast SSD. For each token only a small slice of a mixture-of-experts model is active, so the engine holds the always-used layers in memory and reads the chosen experts off disk with one aligned read each, overlapping those reads with arithmetic. A predictor guesses the next layer's experts and starts fetching them early without changing the output. It ships as a C library, a command line built on that library, and an HTTP server that speaks the OpenAI chat API.
Why it's ranked here
It does something few local runtimes attempt: the README reports the full 2.78-trillion-parameter Kimi K3 decoding at roughly 0.6 tokens per second on a 64 GB MacBook Pro, and DeepSeek-V4.1-Flash and GLM-5.3-Flash at close to 4. Every layer is checked against a PyTorch reference, with final logits agreeing within 3.6e-06. The documentation is unusually honest about measurement: it publishes failed experiments, keeps superseded numbers beside their corrections, and declines to claim a speedup it has not measured. Apache 2.0, from SQLite Cloud.
What's good
Memory is a hard ceiling, not a hint: a budget below the model's floor fails at open with a pointer to the planning command instead of swapping the machine, and the automatic budget reads Linux cgroup limits rather than host RAM. The command line has no private access, so anything it does a host program can do from C. Prompts are tokenized so that user text cannot forge control tokens or open a fake system turn. The server stops generating the moment a client disconnects, and per-record checksums are available on request for copied or downloaded containers.
Tradeoffs
Speed is bound by storage: a cold K3 token reads about 17 GB, and a tested USB enclosure managed 0.94 GB/s against 12.78 for the internal SSD. Giving the process more memory can make it eight times slower once the machine starts paging. Models must first be converted into its own container format, a step that needs torch and safetensors, and the K3 container is 982 GB. The server serializes generations behind one lock. CUDA and BLAS backends are not implemented, Metal is a build option, and the code itself is written by LLMs under human direction.
How to use it well
Start with Kimi-Linear, which the README names as the small model to begin with, then move up. Keep the converted container on internal NVMe storage, and run the planning command before choosing a budget, since the default deliberately leaves a quarter of usable RAM to the operating system. Turn on record verification for any container you did not convert yourself. Suited to experimenting with frontier-scale models on one workstation or serving a single local client; it is not a throughput server for many concurrent users.
Technical notes+
src/backend.c registers a portable CPU baseline, then overwrites kernel slots with NEON, AVX2 or AVX-512 chosen from CPUID or hwcaps at runtime; WASTE_BACKEND=cpu forces the baseline, and Metal, CUDA and BLAS are compile-time registrations. The Makefile builds the waste CLI, libwaste.a, a shared libwaste for ctypes, and libwastevq for the converter, and errors out early when WASTE_ENABLE_CUDA or WASTE_ENABLE_BLAS is set because their sources do not exist. docs/FORMAT.md defines the container directory, 4 KiB-aligned expert records, VQ3R residual vector quantization, and an optional crc32 check via waste_cfg.verify_records. serve/server.py is a stdlib ThreadingHTTPServer with one engine lock and a 64 MiB body cap. docs/ENGINE.md covers the budget ladder and cgroup reading.
Observed
- License
- Apache 2.0 (SPDX headers in serve/server.py, src/backend.c, cli/main.c)
- Language
- C engine and CLI; Python server and converter
- Runtime dependencies
- C plus libc, libm and pthreads; server is Python standard library only
- Interfaces
- C library, CLI (run, chat, bench, plan, info, version), OpenAI-compatible HTTP API
- CPU backends
- Portable baseline, NEON, AVX2, AVX-512 selected at runtime (src/backend.c)
- Accelerators
- Metal as a build option; CUDA and BLAS not implemented (Makefile)
- Model format
- Requires conversion into its own container directory (docs/FORMAT.md)
- Build
- make, producing CLI, static and shared libraries (Makefile)
Read from README.md, Makefile, docs/TECHNICAL.md, docs/ENGINE.md, docs/SERVE.md, docs/FORMAT.md, serve/server.py, src/backend.c, cli/main.c.
Intel on WARP
- Software Engineering Is Becoming Factory Engineering — Zach Lloyd, Warp
- How Software Factories Improve Themselves — Suraj Gupta, Warp
- How to Use NVIDIA Warp and MjWarp to Accelerate Robotics Simulation and Learning Workflows
- The Agent Behind the Curtain: Building the Oz Cloud Agent Platform — Safia Abdalla, Warp
- Clinch – Local-first Warp fork built for agent session management
Tags
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.