How Generative Recommenders Are Redefining RecSys at Scale
- Source
- developer.nvidia.com
- Author
- Elizabeth Goodman
- Date

Context
Generative recommenders predict a person's next action or item from their interaction history. Scaling them involves storing representations of constantly arriving users and products, which can exceed a GPU's memory. NVIDIA's technical article describes tools aimed at that storage problem.
During training, DynamicEmb allocates a representation when an identifier is encountered instead of reserving a fixed table for every possible identifier. For serving recommendations, nv--cache places frequently accessed representations in GPU memory and uses CPU memory and remote storage for other entries. The implementation account explains how allocating and moving these representations helps accommodate catalogs larger than one GPU can hold.
- embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
- transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
articleDiagnose Distributed AI Workload Failures with NVIDIA Attribution
articleKeep Distributed Training Running with Automated Workload Recovery
articleMemory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4
articleHow NVIDIA OpenShell Reference Architecture Secures and Governs AI Agent Workloads at Scale
Checking sign-in…
Loading comments…
