How Ai2 runs model inference across a continent in a day
- Source
- x.com
- Date
The organizations best positioned to use Earth-observation models – those working in conservation, food security, & disaster response – often can't run these models at the scale they need. That's an infrastructure problem. So we built the OlmoEarth Platform to solve it. 🧵
The OlmoEarth Platform takes geospatial models from fine-tuning all the way through large-scale inference. Today it can run inference across a continent in roughly a day, processing dozens of TB of imagery at a cost of fractions of a penny per km².
At its core is OlmoEarth Run, the engine that takes the area an OlmoEarth Platform job covers, splits it into partitions, then into smaller windows the models process. Because each window is independent, the same work can run across thousands of machines at once.
Each OlmoEarth Platform job runs in three hardware-matched stages – prep (CPU), inference (GPU), postprocess (CPU) – inside runners that spin up, do one task, & shut down. Every task is idempotent, so a stalled provider or crashed job is just retried or rerouted.
Context
Ai2's OlmoEarth Platform makes a continent-sized mapping job manageable by dividing the geographic area into independent pieces that many machines can process simultaneously. Its execution system assigns smaller image windows to workers, while a maintained imagery index helps locate the required satellite data. The engineering account explains how this avoids making each part of the map wait for the others.
For a North American wildfire-risk mapping run, Ai2 reports about 30.5 hours of elapsed time against an estimated 4,737 hours of serial computation. The quoted 155-fold speedup is that elapsed-time comparison, achieved using extensive parallel hardware.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Checking sign-in…
Loading comments…