Shen Nie researches generative models at Renmin University of China. He co-authored the original LLaDA work, which trains a language model to reconstruct masked text instead of requiring every word to be generated strictly from left to right.[1][2]
Start with Large Language Diffusion Models →
Education
PhD studies in Artificial Intelligence, in progress
2022 – end date unverified
[1]Links
Website
GitHub Google Scholar Top-seed research intern · ByteDance
Career
Nie earned a bachelor’s degree in computer science at Xi’an Jiaotong University and began doctoral studies in artificial intelligence at Renmin in 2022. His public biography lists a ByteDance research internship beginning in March 2025, after internships at Ant Group, Sea AI Lab, Kuaishou and Shengshu.[1]
Contributions
Generate text by filling in missing tokens
LLaDA is trained on text with randomly masked tokens. A transformer learns to predict the missing content, and generation reverses the masking process over several steps. This tests whether scaling and instruction-following require a strictly left-to-right model. Nie is an equal-contribution author on the original paper. The reported comparisons concern the evaluated models and tasks; they do not establish that diffusion is universally better than autoregressive generation.[2]
Reduce noise in preference alignment
Nie also co-authored LLaDA 1.5. Preference training needs estimates of how likely a model is to produce competing answers, but those estimates can vary substantially in masked diffusion models. The paper develops variance-reduction methods so the training signal is less noisy, then evaluates the resulting model on mathematics, coding and instruction-following tasks.[4]
Tools, writing & talks
Writing, research & talks
Start here
Large Language Diffusion Modelsarxiv.orgThe original LLaDA paper, explaining masked-token training and iterative text generation.
Start with the masking-and-generation diagram, then compare it with the paper’s autoregressive baselines. Basic transformer and probability knowledge help. Use the accompanying repository for code; the paper is the clearer introduction before running an 8B checkpoint.
Equal-contribution co-author[2]
Tools
- LLaDA code and modelsgithub.com
The research implementation and released checkpoints accompanying the LLaDA papers.
Research collaborator[3]