← All IntelClip / AI ToolsSpecificity and code-to-commentary linking as quality signals
From Ending AI Slop — Thais Castello Branco, Taste Labs · ≈12:46
“The specificity of their language and of how precise they're being able to be with how they're doing that description is what will determine that data quality.”
“we know that models have a tricky time kind of actually connecting the piece of the code to the visual”
What’s in it
- Reveals how to judge if expert-labeled training data is actually high-quality
- Shows a trick to tie code snippets directly to expert critique
- Explains why specificity in expert language predicts data signal strength
Clip transcript
lot of what we call essentially like QA on this data and I think there's two ways to do that is understanding what are properties about that data point that correlate with being it being rich and high signal. So for example, specificity when you're trying to ask an expert to define is this good, is this bad or put reasoning behind it or create a whole observation system around uh how they would judge an asset. The specificity of their language and of how precise they're being able to be with how they're doing that description is what will determine that data quality. Or for example, if you can tie their commentary with actually which piece in the code does this relate to. Like let's say you have an expert that's judging a landing page. Uh they might be able to just like write a paragraph describing this. But we know that models have a tricky time kind of actually connecting the piece of the code to the visual. And so if you can find for example a method to tie that exact code component to the commentary of the expert, suddenly you have data that is way less noisy and way more clear. So these are some things that we can do
Comments
Sign in to comment.
Loading comments…