Intel / video

Skill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging Face

Source: youtube.com
Author: AI Engineer
Date: Published Sep 23, 2026

Context

Large image-and-text models can help prepare a smaller object detector before it processes a live camera feed. The training workflow described in this talk starts with unlabeled images, uses one model to propose object labels and asks two smaller models to judge the marked images before training the detector.

Automating those steps still requires task knowledge. The talk reports coding agents making inappropriate training transformations, including flipping traffic signs and changing traffic-light colors. Humans also approve the judging instructions. The workflow shifts model-assisted work into dataset preparation, while those checks help keep automated edits from changing what an image means.

Key takeaways · AI-distilled

Merve Noyan argues against calling a VLM at runtime for vision tasks: a small task-specific detector can run at around 40 frames per second on weak hardware and still beat the VLM.
Her 'vibe training' flow labels an unlabeled dataset with a 9B open VLM, sends the overlaid boxes to two smaller VLM judges, merges on minimum agreement rather than consensus, and trains an RF-DETR detector, for about $3-4 end to end.
Minimum agreement mattered because one judge rejected far more than the other; requiring consensus would have left too few training examples. She also says the judge prompts still need human approval.
She warns that even strong coding agents make domain mistakes, such as horizontally flipping traffic signs or color-jittering traffic lights during augmentation, until explicitly told not to.
She also flags licensing: teams often deploy a popular detector without noticing its copyleft license, which is why her toolkit hands coding agents Apache 2.0 models.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

Read the source www.youtube.com