An efficient arithmetic circuit does not automatically make an efficient AI system. An Unconventional AIengineering essay explains how analog computing gains can be offset by moving data, converting between analog and digital signals, or needing a larger model to compensate for noise.
The relevant comparison is therefore the energy required to produce a result at the required quality. A circuit-level operations-per-watt figure leaves out how many operations and memory accesses that result needs. The essay argues for choosing analog and digital components together around the complete workload, so that improvements in one part survive the costs of the surrounding system.
Key takeaways · AI-distilled
Analog's precision cost is steep: since thermal noise scales as the square root of kT/C, each extra bit of dot-product precision requires quadrupling capacitance and thus energy, so analog's edge over digital mainly shows up below about 4 bits.
Even at record efficiency the gap is smaller than hoped: 4-bit digital in-memory-compute columns hit over 100 TOPS/W (10 fJ/op) in the latest FinFET nodes, while the best analog implementations to date reach only a few hundred TOPS/W.
The author argues TOPS/W is nearly meaningless at the system level because it ignores write energy and memory-access costs; total energy per inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → is what matters, and a block with record fJ/op can still lose if it needs a larger model for iso-accuracy.
In one hand-crafted tinyML classifier (Doshi, 2024), the lowest-energy design came not from optimizing compute-block TOPS/W but from balancing compute against data-movement cost by interspersing small analog memory arrays with nearby bit-serial digital logic.
Terms in this piece · Glossary
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
articleNeural co-evolution, the inevitability of hardware and software co-evolution for AICarlos H. Morales
articleHow to improve AI energy efficiency by 1000xPeter McMahon
articleSpeaking the Language of Physics: Introducing the First Instruction Set Architecture (ISA) for Dynamical System HardwareAng Li