Damai Dai coauthored DeepSeekMoE and the DeepSeek-V3 and DeepSeek-R1 technical reports. His published work concerns efficient language-model architectures and training methods that develop reasoning through reinforcement learning.[1][2][3]
Start with DeepSeekMoE →
Contributions
Experts that can specialize
Dai is the first-listed author of DeepSeekMoE. A mixture-of-experts model activates a subset of its components for each token, allowing a large model to use less computation per token than a comparably sized dense model. DeepSeekMoE divides experts more finely and separates shared experts from routed experts. Shared components can hold common knowledge while routed components specialize, reducing redundant learning across experts. The paper evaluates this design against conventional mixture-of-experts configurations.[1]
Scaling an efficient language model
The DeepSeek-V3 report lists Dai among its authors. It describes a mixture-of-experts language model that combines multi-head latent attention with an auxiliary-loss-free approach to balancing routed expert workloads. Load balancing matters because overloaded experts can slow distributed execution while other hardware waits. The report concerns a team-built training system and model; the author list does not allocate individual responsibility for each component.[2]
Reasoning learned through reinforcement
Dai also coauthored DeepSeek-R1. Its R1-Zero experiment applies reinforcement learning without a preliminary supervised fine-tuning stage and reports reasoning behaviors emerging during training, alongside readability and language-mixing problems. The final R1 pipeline adds cold-start data and further training stages to address those limitations. The team also distilled reasoning into smaller models. The comparison separates an exploratory training result from the additional work needed to make the resulting model more useful.[3]
Tools, writing & talks
Writing, research & talks
Start here
DeepSeekMoEarxiv.orgIntroduces fine-grained expert segmentation and shared experts for efficient language models.
Begin with the architecture diagrams and the comparison between shared and routed experts. Basic familiarity with neural networks is useful; read this before the larger V3 report to understand the expert-specialization problem it builds on.
First-listed author[1]- DeepSeek-V3 Technical Reportarxiv.org
Describes the architecture and training of DeepSeek’s large mixture-of-experts model.
Coauthor[2] - DeepSeek-R1arxiv.org
Studies reinforcement learning for reasoning and a pipeline that improves usability and distills smaller models.
Coauthor[3]