Data Quality Is the Compute Multiplier — Ari Morcos, DatologyAI

DatologyAI's approach to synthetic training data starts with selecting informative documents, then presenting their content in different forms. In its presentation on data quality, the data-curation company gives the example of turning a document about a corporate takeover into true-or-false questions. The intended benefit is to expose a model to useful information through varied learning tasks. Document selection remains central: the company reports that rephrasing randomly chosen material works less well than rephrasing high-quality material. It also emphasizes matching the dataset to the model's intended tasks. This makes curation a decision about what the model should learn and how to present it, rather than simply how much text to generate.

“Data quality is a compute multiplier because what it does is it it makes the learning curve steeper.”

Ari Morcos

“How do you do more? You make data better and now it's as if you had 100 times um the compute.”

Ari Morcos

“The internet is an extremely biased view of the world that does not represent the world uniformly at all.”

Ari Morcos

“data quality remains the single most underleveraged compute multiplier”

Ari Morcos

“it's almost always better to repeat highquality data than it is to show lowquality data at a certain point”

Ari Morcos