DwarfStar (ds4)
github.com/antirez/ds4- Category
- Developer Tools
- Rank
- No. 734Tools index
- Listed in
- #8 Run models locally
- Pricing
- Open Source
- Platform
- cli · desktop
- Type
- TOOL
- Builder
- antirez
- GitHub
- 22.6k stars
- Date
About
A narrow, purpose-built local inference engine for DeepSeek V4 Flash/PRO and GLM 5.2 models, supporting Metal, CUDA, and ROCm backends with SSD streaming, multi-GPU tensor/pipeline parallelism, and a built-in HTTP server and coding agent. It targets high-RAM consumer and workstation hardware (e.g., 96GB+ Macs, DGX Spark, Framework Desktop) and ships aggressive routed-expert quantization tuned specifically for these models.
What it does
DwarfStar treats local model execution as one integrated system. It handles model loading, chat formatting, tool use, conversation state, serving, evaluation, and agent work within a shared native engine. Its command-line client can run one prompt or retain state across interactive turns.
Why it's ranked here
The narrow scope produces unusually coherent engineering: inference, state management, serving, benchmarking, and quality checks evolve together. That focus also makes the project fragile as a general dependency. The maintainers call it beta, warn of possible instability, and may remove models when better replacements appear.
What's good
The repository pairs runtime code with practical validation tools. It includes real-model regression evaluation, official continuation fixtures, speed benchmarks, quantization tooling, and a documented release test matrix. Interactive sessions reuse live cache state, while distributed execution supports both tensor and pipeline strategies. Public model downloads can resume after interruption.
Tradeoffs
Only listed DeepSeek and GLM GGUF layouts work. Arbitrary GGUF files are unsupported, and execution modes differ by quantization. GLM lacks some features available to DeepSeek, including directional steering and external speculative-model files. Speculative decoding remains experimental and can slow unsuitable prompts. The project changes quickly and explicitly permits model removal.
How to use it well
Choose it when you control substantial local hardware and want a hackable, end-to-end runtime for its supported models. Start with a verified model package, use the supplied evaluation and benchmark tools, then tune or modify the engine for your hardware. Do not choose it as a stable general-purpose GGUF compatibility layer.
Technical notes+
The Makefile builds five C executables: ds4, ds4-server, ds4-bench, ds4-eval, and ds4-agent, with Metal, CUDA, ROCm, and CPU targets. ds4_cli.c implements one-shot and stateful interactive modes plus backend and generation configuration. ds4_tp.c defines a framed lockstep tensor-parallel protocol, using blocking TCP control and either Thunderbolt RDMA SEND/RECV or TCP for gate traffic. ds4_ssd.c plans resident expert caching and manages chunked memory locking. ds4_web.c implements local Chrome DevTools Protocol transport for the agent. ds4_eval.c loads the real model and grades embedded benchmark cases. rax.c supplies the radix tree used by server-side state, while README.md documents constrained GGUF layouts and the broader QA toolchain.
Observed
- Primary language
- C, with Objective-C build flags, Metal kernels, CUDA sources, and ROCm sources
- Packaging and install surface
- Source build through Make targets; model weights are downloaded separately
- Command-line interfaces
- Build produces inference, server, benchmark, evaluation, and coding-agent executables
- Server interface
- Includes a built-in HTTP server with persistent session state
- Platform support
- Metal on macOS, CUDA and ROCm on Linux, plus a CPU diagnostics build
- Model format scope
- Accepts only explicitly supported DeepSeek V4 and GLM 5.2 GGUF layouts, not arbitrary GGUF files
- Repository structure
- Includes a tests directory, benchmark tooling, quality-testing tooling, and a release QA matrix
Read from README.md, Makefile, rax.c, ds4_tp.c, ds4_cli.c, ds4_ssd.c, ds4_web.c, ds4_eval.c.
What it can do
Run local inference on DeepSeek V4 Flash/PRO and GLM 5.2 models
Model weights and prompt → Generated text
Serve inference requests via a built-in HTTP server
HTTP requests → Model responses
Run a built-in coding agent
Coding task or prompt → Generated code
Quantize models for the supported architectures
Full-precision model weights → Quantized model files
Tags
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.