NeMo Data Designer: An Extensible Framework for Multimodal Synthetic Data Generation
Source
Johnny Greco, Nabin Mulepati, Andre Manoel, Eric Tramel, Kirit Thadaka, Mike Knepper, Dhruv Nathawani, Dane Corneil, Yev Meyer, Alex Watson, Maarten Van Segbroeck
Author
Johnny Greco, Nabin Mulepati, Andre Manoel, Eric Tramel, Kirit Thadaka, Mike Knepper, Dhruv Nathawani, Dane Corneil, Yev Meyer, Alex Watson, Maarten Van Segbroeck
Date
Key takeaways · AI-distilled
Dataset columns can be defined with statistical samplers explicitly configured to steer dataset diversity, alongside text, code, structured-output, image, and embeddingA list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.Full definition → column types — diversity control is a first-class part of the configuration, not an afterthought.
The full dataset specification is an inspectable, shareable config artifact, and at runtime NDD resolves column dependencies, schedules calls to user-provided model endpoints, and automatically retries failed requests.
NDD was used to build datasets for NVIDIA's own Nemotron model development as well as production enterprise deployments, not just as a research prototype.
Terms in this piece · Glossary
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
Why it matters
Gives teams building training or evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → datasets a reusable, inspectable config format for generating text, code, structured, and image data instead of one-off scripts.
