• inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

A 2.4T-parameter open release is not locally runnable as shipped: 4.89TB of weights with no 4-bit quant from the lab, and even 4-bit needs roughly four 512GB machines.

• post@Jason @Lons @eisokant @ape Correction: Should actually be closer to 50 tok/sec
• postApple markets a four-Mac cluster for trillion-parameter local inference
• postLocal inference crosses over: the M5 Ultra and the 1,000x claim
• postCheap agents are closing the CUDA portability gap

• articleServe Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72Michelle Horton
• clipMixed 2/4/8-bit quantization puts a 2B model at ~2.9 bits per weightAI Engineer
• postNVIDIA has just released the first Nemotron 3.5 model: Nemotron 3.5 Lightning,…Artificial Analysis

Checking sign-in…

Loading comments…