Alex Cheema co-founded EXO Labs, which builds software for running AI models across clusters of ordinary computers. His background combines physics at Oxford with distributed systems and trading infrastructure.[2][1]
Start with exo →
X · alexocheema Background
Cheema studied physics at the University of Oxford. Canonical, an investor in EXO, describes his earlier work as CTO of GamingForGood and as a builder of decentralized-finance trading infrastructure at Portofino. Beacon’s founder biography independently identifies the Portofino work. Those experiences put the emphasis on software that coordinates machines and moves data, which also underlies EXO’s approach to local AI.[1][2]
Contributions
The exo project joins devices into a cluster so a model can use more memory than a single computer provides. Devices discover each other automatically, and the software chooses how to divide the model using available resources and network connections. Its current implementation uses Apple’s MLX inference system and supports direct memory transfer over Thunderbolt 5. That connection matters because communication between machines can otherwise erase the benefit of distributing a model. Cheema is an EXO co-founder; exo is maintained by the company and its contributors.[3]
Ideas
EXO’s practical proposition is that model execution can happen on hardware the user controls. The repository exposes interfaces compatible with OpenAI, Claude and Ollama clients, letting existing applications communicate with a local cluster. Its dashboard also provides a way to chat with and manage models. Capacity and speed still depend on the machines, their network and the chosen model; the repository supplies installation prerequisites and benchmarks for particular configurations.[3]
Tools, writing & talks
Tools
Start here
exo
github.comDistributed model inference with device discovery, model placement and a local dashboard.
Read the README for hardware requirements and cluster setup. Trying exo requires compatible machines and its documented dependencies.
Writing, research & talks
Cheema questions why more than 10 specialized LLM inference engines have shipped in a month when vLLM and sgLang already exist, drawing replies debating whether this is healthy competition or harmful ecosystem fragmentation.
Catalog workOriginal sourceApple is now marketing four M5 Ultra Mac Studios clustered over Thunderbolt 5 RDMA as a local frontier-model rig — 2TB of unified memory at 4.8TB/s, pitched to run trillion-parameter models on-prem — with exo's software as the clustering layer and the setup featured on Apple's own product pages.
Catalog workOriginal sourceExo Labs' founder marks the M5 Ultra Mac Studio as the moment clustered consumer Macs reach frontier-class local inference — roughly 1,000x the two-MacBook Llama 405B run, compounding a 10x hardware step with RDMA over Thunderbolt, better kernels and denser models.
Catalog workOriginal sourceArgues that agents able to benchmark their own kernels are quietly ending CUDA lock in.
Catalog workOriginal sourceA case for splitting inference across mismatched local hardware: prefill is compute bound and runs faster on a DGX Spark, decode is memory bandwidth bound and runs faster on an M5 Max, so run each phase where it is fastest and ship the KV cache between them.
Catalog workOriginal sourceAlex Cheema runs the local-inference maths on a freshly published 2.4T-parameter, 95B-active open model: 4.89TB of weights, no 4-bit quant from the lab, and four 512GB M3 Ultras as the only viable local 4-bit rig — with an NVFP4 quant from NVIDIA, Unsloth or Red Hat the missing piece.
Catalog workOriginal sourceThe full vLLM configuration for running Nemotron 3.5 Lightning on a DGX Spark with DSpark, covering the mixture-of-experts backend, FP8 KV cache, 262K context length and Mamba cache settings.
Catalog workOriginal sourceAlex Cheema compares Nemotron 3.5 Lightning against Qwen 3.6 35B A3B on DGX Spark: Qwen decodes faster per token but is 45% more verbose, so tasks take 42% longer overall.
Catalog workOriginal sourceAlex Cheema measures Nemotron 3.5 Lightning completing tasks 5.36x faster than the most intelligent DGX Spark-capable model while retaining 78% of its intelligence, with the speedup coming almost entirely from per-token decode speed.
Catalog workOriginal sourceAlex Cheema places Nemotron 3.5 Lightning firmly on the intelligence-versus-speed Pareto frontier, displacing Holo 3.1 35B A3B, the post-trained Qwen variant that previously held that region.
Catalog workOriginal sourceA working note on multi-token prediction and speculative decoding as near-free throughput for local models: acceptance rates climb with structured tokens so coding gains most, MLX has been slow to add MTP, and MTPLX already ships it on Mac ahead of Qwen 3.8 27B's weights.
Catalog workOriginal sourceAlex Cheema previews Qwen 3.8 27B's weight release: close to GLM-5.2-level intelligence running locally on a $4k 36GB MacBook at roughly 25 tokens per second.
Catalog workOriginal sourceAlex Cheema sketches running Kimi K3 locally: four 512GB M3 Ultra Mac Studios, with an expected 30-plus tokens per second using MTP and tensor parallelism over Thunderbolt 5 RDMA.
Catalog workOriginal source
