Vibeleaderboard
← All Intel
Intel / article

Neural co-evolution, the inevitability of hardware and software co-evolution for AI

Source
unconv.ai
Author
Carlos H. Morales
Date

Context

Unconventional AI proposes designing learning algorithms and chips together, allowing each to change what the other needs to do. Its neural co-evolution essay argues that starting with fixed mathematical operations can constrain hardware designers before they explore more energy-efficient physical behaviors.

The company's proposed alternative uses circuits whose changing physical state performs part of the computation. Models would be trained around those dynamics, so hardware behavior becomes part of the model design. The essay illustrates the idea with a trained simulation. Its contribution is a design strategy for pursuing efficiency, with the energy advantage still a goal rather than an established result.

Key takeaways · AI-distilled
  • Cerebras addressed memory-bandwidth limits with its Wafer-Scale Engine: hundreds of thousands of cores and tens of gigabytes of on-chip SRAM on one piece of silicon, at the cost of high energy consumption.
  • Groq's compute-near-memory design distributes a network across chips with local SRAM and uses a deterministic dataflow architecture to strip out the non-deterministic control-flow overhead general-purpose processors carry, prioritizing latency.
  • Nervana (2014) pioneered a low-precision FlexPoint numerical format for distributed matrix multiplication in neural-network chips; GPU makers validated the specialization by adopting similar approaches by 2017.
  • Google's TPU pairs massive interconnected Matrix Multiplier Units arranged as a systolic array with high-bandwidth memory to cut instruction overhead, built specifically to serve Google's own in-house models rather than as a general accelerator.
  • As a working example of its 'time as a first-class citizen' framing, the piece shows a steerable, trainable system built from a simulation of a physical dynamical system, trained to follow a target state-space trajectory over time.
More from Carlos H. Morales
Recommended reads
Comments

Checking sign-in…

Loading comments…