ARGODRIVE

About

ARGODRIVE is a layout, balancer, and instrumentation toolkit for running mixture-of-experts language models directly from SSDs on memory-constrained machines, such as a 518GB model on a 128GB laptop. It found that the standard prefill sweep reads every expert of every routed layer even though a 512-token chunk only routes to about half of them, and fixing that gap alone yields a measured 1.73x speedup in prompt processing on one internal SSD without adding replicas.