← All IntelClip / AI ToolsData quality as a compute multiplier, stated precisely
From Data Quality Is the Compute Multiplier — Ari Morcos, DatologyAI · ≈2:03
Frames better data as steepening the performance-vs-scale curve, so the same budget buys results that would otherwise require roughly 10x the compute.
What’s in it
- Frames better data as steepening the performance-vs-scale curve, so the same budget buys results that would otherwise require roughly 10x the compute.
Clip transcript
is increasingly scarce and you need to make models better. Well, what do you do? Well, we work on data. Um I you're going to hear me say this over and over again. Data quality is a compute multiplier because what it does is it it makes the learning curve steeper. So this is just a very simple schematic of performance on the y-axis as a function of data on the x-axis. Note that this x-axis, you could swap it out for data, for compute, for time, for dollars. They're all the same x-axis fundamentally. Um, and if you can make data quality better, you can turn this gray curve um into this blue curve. And that means that now you can get dramatically better performance for the same compute budget as if you had trained with far more compute. Um and similarly you can get the same performance for a much smaller compute budget. Um which is exactly showing you how you can get performance as if you had spent 10 times this 10 times as much on compute and really shows this compute
Comments
Checking sign-in…
Loading comments…