Why FLUX 3 Trains on Images, Video and Sound — Andreas Blattmann & Dustin Podell, Black Forest Labs
- Source
- latent.space
- Author
- Latent.Space
- Date

Context
Black Forest Labs, the lab behind the FLUX image models, trains FLUX 3 on images, video and audio together because it treats each as an incomplete record of the same physical world (BFL's FLUX 3 announcement, also in Intel). A still image shows where things are at one moment, video adds how they move, and sound carries evidence about events such as an impact that the camera may have missed. BFL's claim is that learning all three at once lets them check each other: the sound has to match the impact and the motion has to fit the object. In this Latent Space episode, host swyx asks two BFL researchers to defend that bet: Andreas Blattmann, co-inventor of the latent diffusion method behind Stable Diffusion, and Dustin Podell, who led the FLUX Kontext image-editing model (episode page).
Blattmann's clearest example, as Latent Space's write-up of the episode describes it, is a camera panning around a person. At some point the model has to produce the back of their head, and he argues that ordinary video holds a great deal of evidence about how objects look from other angles. That makes a stricter test than whether a single frame looks convincing, though the write-up notes it does not settle whether a model understands physics, a point the conversation goes back and forth on. The kind of knowledge he means is closer to catching a tossed ball: your hand starts moving before the ball arrives, without your being able to write down its path.
The idea also has a plain engineering payoff. BFL derives a video upscaler from the FLUX 3 backbone, and the write-up explains why it cannot simply sharpen each frame on its own: the upscaler would make a slightly different decision for every frame, so the added detail shimmers when the clip plays. On the wider architecture question, swyx asked whether autoregressive models, which generate output one after another, were taking over image generation. Blattmann gives them their due: they can learn from every token of a training example at once, while diffusion models can be from a long sampling process down to a handful of steps. Which advantage matters depends on whether the cost falls on training the model or on waiting for an image.
The evidence behind the bet is still early. BFL describes its comparisons against rival video models as preliminary results from a model in development, and its robotics claims rest largely on partner work such as FLUX-mimic, where the video backbone is fine-tuned to predict a robot's actions. This explanation draws on Latent Space's written episode page and BFL's announcement, not on the recording or a full reading of its transcript.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
- multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
articleBuilding Legal Agents When You Can't See the Data - Jacob Lauritzen, CTO of Legora
articleHow Anthropic ships the #1 LLM in the World safely: Separating the Brain from the Hands
articleSupabase’s Plan for a Billion AI-Built Appsarticle[AINews] AI Engineer NYC: Gemini 4 Argon, Vals AI Benchmarks
Checking sign-in…
Loading comments…


