Vibeleaderboard
← All Intel
Intel / video

The Shape of Compute (Chris Lattner of Modular)

Source
youtube.com
Author
Latent Space
Date

Context

Chris Lattner, whose company Modular builds software for running AI models across different chips, says in a Latent Space interview that Modular spent its first three years in research mode with one test: serve a Llama 3 model on NVIDIA GPUs at speeds matching or beating NVIDIA's best, without NVIDIA's CUDA programming platform. Lattner argues an alternative AI stack that trailed the state of the art would be judged against the leader, so Modular held back wide release until it met that bar.

According to Lattner, a release at the end of December met the goal, but only on one GPU type, the A100, and one model, with ugly code and technical debt. After that the work became ordinary engineering: a release every six weeks, H100 support, and about 500 models, with AMD MI300 support next, and the code was open sourced. Lattner says it is still an early system with bugs. All of this is Lattner's own account of Modular's progress.

Terms in this piece · Glossary
  • tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
Read the source www.youtube.com
More from Latent Space
Recommended reads
Comments

Checking sign-in…

Loading comments…