John Schulman is a reinforcement-learning researcher whom Berkeley identified in December 2025 as chief scientist at Thinking Machines Lab. A founding member of OpenAI, he led its reinforcement-learning team and co-authored algorithms and training methods that connect learning from rewards with language-model behavior.[1][3][4][5][6]
Start with LoRA Without Regret →
GitHub Background
Schulman came to machine learning through physics and robotics. A contemporaneous Berkeley speaker biography says he graduated from Caltech with a physics degree in 2010 and studied robot motion planning and manipulation with Pieter Abbeel. Berkeley’s later alumni biography records his computer-science Ph.D. in 2016. Robotics supplies a concrete setting for reinforcement learning: a system takes actions, observes their consequences and changes its behavior to obtain better outcomes. His early policy-optimization papers tested methods on simulated movement as well as games.[1][2][4][5]
Career
OpenAI’s December 2015 introduction lists Schulman among its founding members. Berkeley credits him with leading the reinforcement-learning team that created ChatGPT. OpenAI’s own later filing states that he resigned on August 5, 2024 to join Anthropic. In 2025, Berkeley identified him as co-founder and chief scientist at Thinking Machines Lab, and he published research there on efficient model adaptation.[1][3][10][11][7]
Contributions
Improving a policy without destabilizing it
A policy is the rule a learning system uses to choose actions. Trust Region Policy Optimization studies how to improve that rule while limiting how much it changes in one update. Schulman and his co-authors derived a procedure with a theoretical improvement guarantee, then introduced approximations to make it practical for large neural networks. Their experiments covered simulated robotic movement and Atari games. The distinction between theory and implementation matters: the practical algorithm approximates the guaranteed procedure, so its experimental performance should not be described as an unconditional guarantee.[4]
Practical reinforcement learning and human feedback
Proximal Policy Optimization, or PPO, simplifies policy optimization and lets a training process reuse collected experience through multiple updates. Schulman is the paper’s first author. He also co-authored the InstructGPT paper, which combines human-written demonstrations, rankings of model outputs and reinforcement learning to make a language model better follow instructions. This connects a general optimization method with a specific training goal. Human preferences provide a signal about desired answers, but the study reports remaining mistakes and harmful outputs; the method is not evidence that alignment is solved.[5][6]
Adapting models with fewer trainable parameters
In LoRA Without Regret, Schulman and Thinking Machines colleagues investigate when low-rank adaptation can match full fine-tuning. LoRA trains relatively small additions to a model’s weight matrices instead of changing all the original weights. Their experiments find comparable performance when adapters cover the relevant layers and have enough capacity for the training data. The article also studies learning-rate choices. Its value is the conditions it identifies: efficient adaptation needs appropriate placement, capacity and tuning, rather than an assumption that every small adapter works equally well.[7]
Tools, writing & talks
Writing, research & talks
Start here
LoRA Without Regretthinkingmachines.aiA practical investigation of low-rank adaptation versus full fine-tuning, with conditions for comparable learning and guidance on learning rates. Start here if you already understand model training and want to customize a model.
A practical route into Schulman’s current research for builders who know basic fine-tuning; the PPO paper is the more mathematical alternative.
Named author, in collaboration with Thinking Machines colleagues[7]Introduces PPO and evaluates its balance of implementation simplicity, sample use and training time. Some knowledge of reinforcement learning and gradient optimization is needed for the equations.
First author[5]- Trust Region Policy Optimizationarxiv.org
Explains the constrained-update approach behind TRPO and its practical approximations. Read it for the link between an optimization argument and empirical behavior in robotics and games.
First author[4] The InstructGPT paper explains the progression from demonstrations to preference comparisons to reinforcement learning, including the limitations of the resulting models.
Co-author[6]