Runway open-sources Praxis-1 world action model for robots

Runway, best known for AI video generation, has released Praxis-1, a model that controls real robots, with open weights under the Apache 2.0 license, which allows commercial use and fine-tuning on a company's own hardware (Runway's post). Runway calls it a world action model. According to its technical report, each time the robot acts, Praxis-1 predicts both what the scene will look like next and the motion that gets it there, returning a short chunk of predicted video with 1.6 seconds of motor commands. The robot carries out that chunk, sends back new camera frames and asks for the next one.

The bet behind it is that ordinary video can make up for scarce robot data. Robot control models are usually trained on demonstrations recorded by people operating the robot, which are slow and expensive to collect. Praxis-1 starts from Runway's video generation model. In the mid-training that follows, about 58% of the video is first-person footage of people and about 18% is real-robot video; actions are then added and the model is fine-tuned for a specific robot. When Runway previewed the model in September, it reported that pretraining on web video gave about the same final placement error after fine-tuning as pretraining on robot teleoperation video, 16.1 cm against 16.0 cm.

Runway's results put Praxis-1 ahead of comparable open models in simulation. On NVIDIA's RoboLab-120 leaderboard it solved 555 of 1,200 episodes, 46.2%, the top score among world action models and open-source robot foundation models, though four entries scored higher, including two systems built on language-model agents at 86.0% and 57.1%. Fine-tuned for Google's RT-1 robot, which was not in its training data, it reached 80.0% on the SimplerEnv simulation against 52.4% for RT-1's own policy. These are Runway's figures and have not been independently reproduced.

For a robotics team, the release is a generalist starting point it can adapt to its own machine rather than collecting all its training data from scratch. The full model has 27.2 billion parameters and runs on one 80 GB GPU. The report is direct about what still fails: the robot most often reaches for an object that resembles the target, instructions that do not name colour or size are harder to follow, and chaining several steps from one prompt remains difficult.