• token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
• speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.

Explains why local tok/sec roughly doubles with multi-token prediction and where support actually exists on Apple silicon today — which changes what hardware you need for a usable local coding model.

• postApple markets a four-Mac cluster for trillion-parameter local inference
• postLocal inference crosses over: the M5 Ultra and the 1,000x claim
• postCheap agents are closing the CUDA portability gap
• postExcited to collab with @ashxhart on this!

• postLocal inference crosses over: the M5 Ultra and the 1,000x claimAlex Cheema
• postQwen 3.8 27B weights are coming out next week. It's a big deal.Alex Cheema
• postRunning Kimi K3 on my desk?Alex Cheema

Checking sign-in…

Loading comments…