Intel / article

Training 1000-layer networks without backpropagation

Source: unknown
Author: Sakana AI
Date: Published Sep 13, 2026

Context

Sakana AI's PC-ALM explores how a deep network can learn when each layer communicates only with its neighbors. Conventional backpropagation passes error information backward through the network. PC-ALM instead gives each layer a feedback process combining its current prediction error with accumulated error, helping the learning signal reach distant layers. The research article explains this local mechanism.

The team reports training narrow networks up to 1,000 layers on handwritten-digit classification (MNIST), with accuracy within about two percentage points of backpropagation in that experiment. The depth result addresses a weakness of earlier predictive-coding methods, whose learning signals fade in deep, narrow networks. Potential use in specialized hardware remains a research direction; the reported demonstration concerns learning behavior on image-classification tasks.

Key takeaways · AI-distilled

Sakana AI says standard predictive coding struggles with depth because credit signals at the far ends of a network fail to diffuse into internal layers, which is the scaling problem PC-ALM targets.
PC-ALM adds dual neurons (Lagrange multipliers) to each layer's local dynamics, so every layer behaves like a PI feedback controller that drives its own prediction error down.
The authors report PC-ALM propagates learning signals to seemingly arbitrary depth, with the clearest gains in deep, narrow networks where standard predictive coding struggles to learn.
Sakana frames the work as a model of how brains might assign credit with only local coupling, and suggests it may suit neuromorphic hardware, where running dynamics is cheaper than on GPUs.

Source link unavailable.unknown