Shows how fitting a 120B model's roughly 70GB of weights in memory drives local agent speed: 128GB unified memory held it, while a 24GB GPU offloaded most layers and slowed down.
Key takeaways · AI-distilled
In Cline's kill race, three Cline agents on gpt-oss 120B had to read a PID file, write a bash script sending kill -9 to rivals, and run it; a cloud-backed Ollama variant on a Mac won in 1.04 seconds, which Cline credits to faster TTFT.
In a direct Ollama API test with the same coding prompt and no AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →, Cline measured DGX Spark at 42.9 tok/s (21.83s wall), an RTX 4090 at 8.7 tok/s (93.89s), and the cloud-backed model at 5.11s wall time.
The DGX Spark was slowed by network topology: every call from the Mac control node crossed a Tailscale VPN. Cline argues running the agent on the model's machine would remove that penalty, but did not test that setup.
Cline's tradeoff framing: TTFT decides short sequential tasks, while sustained throughput matters for agent sessions lasting minutes or hours; the cloud model is metered and sends code off-machine, while local inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → can run offline.
Terms in this piece · Glossary
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
