Barret Zoph is a machine-learning researcher whose work includes neural architecture search and sparsely activated language models. His current public account identifies him as vice president of research at Google DeepMind, following research and leadership roles at OpenAI and Thinking Machines Lab.[1][2]
Start with Neural Architecture Search with Reinforcement Learning →
Education
Links
Website
X LinkedIn Google Scholar Vice president of research, Google DeepMind · Google DeepMind
Career
Zoph worked on statistical machine translation at the Information Sciences Institute before joining Google Brain. There he researched automated model design and large sparse models. He later led post-training research at OpenAI, covering alignment, tool use, evaluation and models delivered through ChatGPT and the API. His public account lists subsequent roles as Thinking Machines Lab’s cofounder and CTO and OpenAI’s VP and GM for Enterprise before his return to Google DeepMind.[1][2]
Contributions
Neural Architecture Search, coauthored with Quoc Le, turns model design into a learning problem. A controller generates candidate network descriptions, the candidates are trained and evaluated, and validation accuracy provides the reward for improving the controller. The reported image and language experiments show that this process can find competitive designs. They do not establish a universally best architecture for every task.[3]
Scaling with sparse experts
Switch Transformers, coauthored with William Fedus and Noam Shazeer, routes different inputs to different expert subnetworks. Only part of the model is activated for a given input, separating total parameter capacity from the computation used on that input. The research addresses routing complexity and training instability, important obstacles to making sparse models usable.[4]
Tools, writing & talks
Writing, research & talks
Start here
A controller learns to propose neural-network designs using validation accuracy as its reward.
Read the controller-and-child-network loop first, then the experiments. Familiarity with neural-network training and reinforcement learning helps; the result is a model-design method, not a ready-to-use coding agent.
Co-author[3]- Switch Transformersarxiv.org
Sparse expert routing and training methods for increasing model capacity without activating every parameter.
Co-author[4]