Analog's precision cost is steep: since thermal noise scales as the square root of kT/C, each extra bit of dot-product precision requires quadrupling capacitance and thus energy, so analog's edge over digital mainly shows up below about 4 bits.
Even at record efficiency the gap is smaller than hoped: 4-bit digital in-memory-compute columns hit over 100 TOPS/W (10 fJ/op) in the latest FinFET nodes, while the best analog implementations to date reach only a few hundred TOPS/W.
The author argues TOPS/W is nearly meaningless at the system level because it ignores write energy and memory-access costs; total energy per inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → is what matters, and a block with record fJ/op can still lose if it needs a larger model for iso-accuracy.
In one hand-crafted tinyML classifier (Doshi, 2024), the lowest-energy design came not from optimizing compute-block TOPS/W but from balancing compute against data-movement cost by interspersing small analog memory arrays with nearby bit-serial digital logic.
Terms in this piece · Glossary
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
articleHow to improve AI energy efficiency by 1000xPeter McMahon
articleSpeaking the Language of Physics: Introducing the First Instruction Set Architecture (ISA) for Dynamical System HardwareAng Li