Vibeleaderboard
← All Intel
Intel / video

Stanford CS25: Transformers United V6 I From Language Models to Native Multimodal Intelligence

Source
youtube.com
Author
Stanford Online
Date

Context

A model that understands images does not necessarily generate images. This CS25 lecture distinguishes systems that accept several kinds of input but produce text from systems trained to generate multiple kinds of output.

The shared building idea is to convert different inputs into sequences the model can process. Images can become encoded patches, and video can become sequences of encoded frames. What the model is trained to predict then determines its output capabilities. The speaker treats input support and output support as separate questions, noting that little work shows training a model to generate images improves its image understanding.

Terms in this piece · Glossary
  • multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
  • LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Read the source www.youtube.com
More from Stanford Online
Recommended reads
Comments

Checking sign-in…

Loading comments…