Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO
Source
latent.space
Author
Basil Chatha
Date
Why it matters
Fixed GPU contracts form rectangles while real workloads spike and fall. This explains why idle capacity drives inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → cost and how serverless providers pool demand, which affects how you budget and buy compute.
Key takeaways · AI-distilled
Modal reports passing $1 billion in annualized run rate and raising a $750 million Series D at a $16B valuation. CEO Erik Bernhardsson says sandboxes, which the team once discussed shutting down, now bring about a third of revenue and functions about two thirds.
Bernhardsson's capacity argument: most GPUs are sold as fixed multi-year blocks starting months out, while real workloads grow, spike and dip. Pooling thousands of customers smooths aggregate demand, so Modal can offer the capacity it secures in elastic slices.
Modal rents its GPUs, typically on three-year deals, across almost 25 clouds. Bernhardsson says those reservations leave Modal effectively long GPU prices, so he worries less about rising prices than about a collapse that would hurt anyone who locked in supply at the top.
He says the median Modal sandboxAn isolated environment where AI-generated code or agent actions run without being able to touch anything real.Full definition → runs about eight seconds and expects sandboxes to get more ephemeral, with snapshot, fork and resume, even one per command. Cheap start and stop would let CI run each test in its own cloned sandbox instead of repeating setup per shard.
Because agents learn old API syntax from training data, Modal now deprecates features more conservatively and updates public examples. It also runs internal tests asking agents to deploy a model with any tool to see whether they pick Modal.
Terms in this piece · Glossary
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
sandbox — An isolated environment where AI-generated code or agent actions run without being able to touch anything real.
