Robot-Use Agents: Why General-Purpose Models May Win in Robotics
Source
youtube.com
Author
Y Combinator
Date
Context
Robot-use agents explore whether a general-purpose model can turn its coding abilities into instructions for physical machines. The episode description summarizes an essay by MIT professor Philip Isola arguing that such models could control different robots, write policies (rules for choosing actions), and learn new physical tasks with little or no robot-specific training. The surrounding control software and real-world evaluations are part of that research direction: generated instructions must become robot behavior that can be tested. The description presents this as an argument and a line of research, with no success rates or demonstrated capabilities.
Key takeaways · AI-distilled
The episode builds on MIT professor Philip Isola's essay arguing we may be entering an era of robot-use agents: general-purpose models that control different robots and learn new physical tasks with little or no robot-specific training.
Guests are the founders of Waddle Labs and RoboCurve, whose demos of general-purpose models controlling robots helped drive that idea.
The discussion covers code-as-policies and vision-language-action models, in-context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → learning versus model training, and how to turn robot actions into reusable skills.
Terms in this piece · Glossary
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.