MolmoMotion: Language-guided 3D motion forecasting

MolmoMotion predicts how an object should move by forecasting the paths of points attached to it. Given visual observations, initial 3D point locations and an action description, Ai2's model outputs future trajectories. Those compact paths can guide a robot or a video generator without first generating a complete video. The release describes the representation.

In a simulated pick-and-place test, a control policy initialized from MolmoMotion reached 76.3% success, compared with 56.0% when initialized from Molmo 2. The policies otherwise used the same action head and training episodes. This supports the value of learned motion forecasting for that control task; the model's sparse points still limit its representation of complex deformation.