Xinyu Zhou is a Moonshot AI researcher whose published work includes scaling the Muon optimizer, sparse block attention for long contexts and Kimi-Audio. The Kimi-Audio report identifies him as a project lead and core contributor.[1][2][3]
Start with MoBA: Mixture of Block Attention for Long-Context LLMs →
Links
MoBA code Contributions
Make optimizer research usable at larger scale
Zhou coauthored the Moonlight report, which investigates how to scale the existing Muon optimizer to larger language models. Its changes include weight decay and adjustment of the update scale. The team releases a distributed implementation and model checkpoints so other researchers can examine the method and training results.[1]
Choose relevant blocks of a long context
MoBA divides a sequence into blocks and uses a parameter-free top-k gate to choose which key-value blocks a query attends to. Zhou is a coauthor and corresponding author. The approach preserves a path between sparse and full attention instead of committing every layer to one fixed local pattern. The implementation includes both a simpler reference version and an optimized version. Its documentation says an existing model needs further training to obtain MoBA’s benefits.[2][4]
Lead work on an audio foundation model
The Kimi-Audio report marks Zhou and Xu Tan as project leads and lists Zhou among the core contributors. The model combines continuous audio features as input with discrete tokens as output and supports audio understanding, generation and conversation. The report describes its data preparation, training, streaming audio generation and evaluation, while releasing code, checkpoints and evaluation tools. This is a team project with a documented contribution list.[3]
Tools, writing & talks
Writing, research & talks
Start here
Sparse attention selects relevant context blocks using a parameter-free gate.
Start with the block-selection diagram and introductory explanation of sparse attention. Knowledge of transformer attention helps; the repository’s reference implementation is easier to follow before the optimized kernel.
Coauthor and corresponding author[2]- Muon is Scalable for LLM Trainingarxiv.org
The Moonlight report adapts Muon for larger models and releases distributed training resources.
Coauthor[1] - Kimi-Audio Technical Reportraw.githubusercontent.com
Describes an open audio model’s architecture, training, inference and evaluation.
Project lead and core contributor[3]