Biao Gong researches multimodal generative models. He is a co-author of Ming-Omni and Ming-Lite-Uni, which combine visual understanding with image generation, and Framer, a system for creating motion between supplied images.[4][5]
Start with Framer →
GitHub Contributions
Connect understanding and generation
Ming-Lite-Uni connects a fixed multimodal language model with a learnable diffusion generator. Multi-scale tokens and representation alignment let language-and-image inputs guide both image generation and instruction-based editing. Gong is a named co-author; the work is a team contribution rather than a model built by one person.[3]
Handle several modalities in one model
Ming-Omni extends the problem to text, images, audio and video. Dedicated encoders feed a mixture-of-experts model with modality-specific routing. Separate output components support speech and image generation. The architecture matters because processing different input types and generating different output types require more than adding an image encoder to a text model.[4]
Control motion between images
Framer synthesizes intermediate video frames between input images. Users can guide the motion with point trajectories, rather than relying entirely on an automatically chosen transition. Its project page provides examples and original code, making the effect of those controls visible.[5]
Tools, writing & talks
Tools
Start here
Frameraim-uofa.github.ioInteractive frame interpolation with point trajectories that guide motion.
Watch the controlled-motion examples before opening the implementation. Compare a trajectory-guided transition with an unconstrained one. The demonstrations need no setup; running the code requires a configured model environment and suitable GPU resources.
Co-author[5]
Writing, research & talks
- Ming-Lite-Uniarxiv.org
A unified multimodal framework for image generation and instruction-guided editing.
Co-author[3]