MoEShift

About

MoEShift is an inference runtime for Mixture-of-Experts language models on Apple Silicon Macs. It streams expert weights from SSD so large models run with a small peak process memory, which leaves RAM free for other apps. The README reports a 125 billion parameter model running on a 24 GB Mac mini at up to about 11 tokens per second with under 10 GB peak memory, and compares it with another local runtime on the same hardware. It is labeled alpha.