Modality Misalignment and Originality Attribution in Short-Form Video — Aditya Gautam, Meta

• Aditya Gautam says cross-modality mismatch in a video is largely solved with embeddings, but drift within a single modality over time (six seconds of sports, then half a second of politics) is not, so Meta splits it across three agents.
• The pipeline: a perceiver segments video where temporal change occurs rather than at a fixed frame rate, a reviewer runs temporal analysis over its JSON plus live comments, and a retriever indexes into inverted, vector, and graph stores.
• Each agent runs on a small specialized VLM rather than a frontier model, with the vision encoder pretrained on in-house images because user-generated content looks unlike clean web data, then instruction-tuned to a JSON schema.
• Models improve via a DPO loop on daily production samples judged by an in-house LLM judge and a human queue that traces failures to the specific tool call or retrieval step.
• To keep most videos out of the pipeline, the talk describes compressing similar frames, caching verdicts on viral content, and pruning by metadata so trusted creators skip analysis.

• AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
• multi-agent — Using several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.
• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
• embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.

Describes a concrete, production-scale multi-agent architecture, including training details like DPO loops with in-house LLM judges, for a hard, ambiguous classification problem operating at internet scale.

• videoFrom Ingestion to Agents: How AI Teams Build on Document Intelligence — Adit Abraham, Reducto
• videoYou’re Not Thinking Big Enough: Rebuilding Food Systems with AI Agents — Cody Menefee, Firecrawl
• videoFrom VLM/VLA's to Embodied Agents — Armen Aghajanyan, Perceptron AI
• videoSkill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging Face

• videoVideo Has No Memory. Here's How We Built One. — James Le, TwelveLabsAI Engineer
• videoEvaling Video Slop — Maor Bril, Character.aiAI Engineer
• videoSOTA Generative Media Panel — Dumitru Erhan, Shane Gu & Nicole Brichtova, Google DeepMindAI Engineer

Checking sign-in…

Loading comments…