From VLM/VLA's to Embodied Agents — Armen Aghajanyan, Perceptron AI

• Armen Aghajanyan notes that an hour of video yields about a million visual tokens, yet supervision lands on roughly 2% of them; Perceptron's response is a perceptive objective that learns which percepts matter instead of predicting every pixel equally.
• To handle context bloat from always-on cameras, Perceptron uses a data-sparse mixture of experts whose router decides per layer which tokens to read and which to skip, beyond the ~10x compression patch averaging offers.
• Perceptron treats detection as an agentic task: the model tiles the image, raises contrast, and proposes boxes iteratively until it finds the target.
• The headline claim is a scaling law: training jointly on perception, reasoning, and control lets 10x more video pretraining substitute for 10x less teleoperation data, which he puts at about $100 per hour.

• mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
• reasoning model — A model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.
• pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
• context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

Introduces a specific scaling-law claim: joint training on perception, reasoning, and control lets 10x more video pretraining substitute for 10x less expensive teleoperation data, a concrete lever for cutting embodied-AI data costs.

• videoFrom Ingestion to Agents: How AI Teams Build on Document Intelligence — Adit Abraham, Reducto
• videoYou’re Not Thinking Big Enough: Rebuilding Food Systems with AI Agents — Cody Menefee, Firecrawl
• videoSkill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging Face
• videoFrom Scratch to SOTA: Training a 3B State-Space Vision Model — Krishna Prasad Srinivasan, Sarvam

• videoSkill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging FaceAI Engineer
• videoThe Best Models Still Reason Like Toddlers — Andrew Dai, ElorianAI Engineer
• videoVoice agents with Realtime Video — Sidney Primas, LemonSliceAI Engineer

Checking sign-in…

Loading comments…