Jim Fan

Fan completed a computer-science PhD at Stanford, advised by Fei-Fei Li, after graduating from Columbia in 2016. His personal chronology dates the PhD from September 2016 to September 2021 and records research roles at Baidu, Mila, OpenAI, Google Cloud AI and NVIDIA. These earlier projects included speech recognition, distributed robot learning and World of Bits, which studied agents using a browser through pixels, keyboard input and mouse actions.[1]

Fan joined NVIDIA in December 2021, according to his personal chronology. NVIDIA’s July 2026 RAISE Summit page identifies him as a Director and Distinguished Research Scientist. His research goal is to build agents that can operate across physical environments, games and software interfaces. The projects below approach that goal through reusable code, learned rewards and models that turn observations and instructions into actions.[2][1][3]

Learning reusable skills in Minecraft

Voyager uses a language model to explore Minecraft, write programs that perform tasks and retain useful programs in a skill library. It improves those programs using environment feedback, execution errors and checks of whether the task succeeded. The project lists Fan as a co-author and equal adviser. Its important distinction is that capabilities accumulate through reusable code and interaction, without updating the underlying language model’s weights. The demonstrations provide a concrete starting point for understanding an agent that learns from experience.[4]

Using code to teach robot behavior

Eureka uses a language model to generate reward functions: code that tells a learning system which behavior should receive a higher score. Training results then help improve that code. The project lists Fan as a co-author and equal adviser, and demonstrates tasks including simulated robotic pen spinning. This is a way to automate part of the training setup, with results measured in specified environments. It should not be read as evidence that a language model directly controls every motor movement or that all demonstrations transfer unchanged to real robots.[5]

From perception to physical action

NVIDIA’s GR00T N1 work combines images, language and action for humanoid robots, training on human video, real and simulated robot trajectories, and synthetic data. Fan is a named co-author. The 2026 Vesta project, which credits him as a joint adviser, addresses embodied reasoning and memory, pairing a planner with a lower-level robot policy. Together these projects illustrate different parts of the problem: choosing what should happen and producing the actions that carry it out. Read Voyager first, then these robot demonstrations to compare the settings.[6][7][4]