Vibeleaderboard
← All Intel
Intel / video

Why AI needs a new kind of supercomputer network — the OpenAI Podcast Ep. 18

Source
youtube.com
Author
OpenAI
Date

Context

a large AI has thousands of GPUs working on one computation, so the network is part of the computation and the slowest link sets the pace. On the OpenAI Podcast, OpenAI engineers Mark Handley, also a University College London professor, and Greg Steinbrecher said internet-style networking fits this poorly: its traffic averages out across independent users, while a training job moves in lockstep and one slow or failed link leaves other GPUs idle.

Their answer, Multipath Reliable Connection (MRC), sprays packets across many paths, trims an overflowing packet down to its header so the receiver can request a resend immediately, and lets each endpoint stop using a failed within milliseconds, which let them run switches with static routing. They said it ran in new clusters built with Microsoft and that a specification is going out as an open standard. They gave no numbers.

Terms in this piece · Glossary
  • Training — The process that adjusts a model using examples and a measure of how well it performs.
  • Model — A computational system that uses learned patterns to produce an output from information supplied to it.
  • Path — A description of a file or folder location, written using directory names and separators.
  • throughput — How much model output a provider can generate over time, commonly measured in output tokens per second.
Read the source www.youtube.com
More from OpenAI
Recommended reads
Comments

Checking sign-in…

Loading comments…