Thinking Machines explores how restricting a network's learned numerical weights could make training updates easier to control. Its Modular Manifolds article develops update rules that respect chosen geometric constraints, then explains how to combine constrained layers into a network. The article ties each layer's learning rate to how sensitive the network's output is to changes in that layer's weights, and proposes this as a way to budget learning rates across layers. It presents a mathematical framework and a small sanity-check experiment; efficient implementation at large scale remains a research direction.
