Vibeleaderboard
← All Intel
Intel / video

Stanford CS153 Frontier Systems | The Discipline of Delivering Value per Gigawatt

Source
youtube.com
Author
Stanford Online
Date

Context

In a Stanford CS153 lecture, Amin Vahdat, introduced as the head of internal infrastructure at Google, argues that gigawatts of capacity is the wrong headline number. A gigawatt costs roughly $40 to $50 billion by figures discussed in the room, but what matters, Vahdat says, is the value delivered per dollar, such as users served. If chips fail or cannot be scheduled, useful , which Vahdat calls goodput, falls well below what was paid for, and idle capacity is a bug.

runs synchronously across many chips, so one failed node can stall the whole job. Historically services aimed for five nines of availability, which requires duplicate power feeds that leave about half the capacity unused. Vahdat says frontier labs would now often take double the capacity with a few days of downtime a year, while noting that this is not true of everyone. Vahdat makes related goodput points in a separate Sequoia interview.

Terms in this piece · Glossary
  • throughput — How much model output a provider can generate over time, commonly measured in output tokens per second.
  • Training — The process that adjusts a model using examples and a measure of how well it performs.
  • Model — A computational system that uses learned patterns to produce an output from information supplied to it.
Read the source www.youtube.com
More from Stanford Online
Recommended reads
Comments

Checking sign-in…

Loading comments…