We present a research preview of Self-Flow: a scalable approach for training multi-modal generative models.
Multi-modal generation requires end-to-end learning across modalities: image, video, audio, text - without being limited by external models for representation learning. Self-Flow addresses this with self-supervised flow matching that scales efficiently across modalities.
Results:
• Up to 2.8x faster convergence across modalities.
• Improved temporal consistency in video
• Sharper text rendering and typography
This is foundational research for our path towards multimodal visual intelligence.
Self-Flow improves temporal consistency in video generation.
4B parameter multi-modal model trained on 6M videos.
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
Why it matters
Dropping the external representation model from multimodalA model that works with more than text — reading images, audio, or video, and sometimes generating them too.Full definition → training reportedly cuts convergence time substantially and yields one model spanning video, audio and action prediction, which is the architecture behind current world-model claims.
Read the full thread on X ↗
