[AINews] Agents disprove Polysemanticity; greatly improve Interpretability & Alignment

This is a daily AINews roundup, now a section of the Latent Space newsletter run by swyx, covering October 14 and 15, 2026 across many items (latent.space). Its title story is an essay by Nick Cammarata at Goodfire, Neural Networks Are Already Clean, and the rest is separate items not explained here. The later part of the issue is behind a paywall.

The essay takes on polysemanticity, the observation that individual neurons seem to respond to several unrelated things, like a famous image-model neuron that fired on both cat faces and car fronts. The leading explanation, superposition, held that a network short of neurons folds unrelated concepts together, so researchers must first train a second model such as a sparse autoencoder to untangle it before studying anything. Cammarata, who co-authored early interpretability work at OpenAI, now argues that superposition was mostly the wrong explanation (goodfire.com). He reports that ordinary architectural improvements, with no change to model size or data, made 99-100% of neurons in the last layer of the old vision model InceptionV1 easily interpretable, where almost none were before. In language models, a 2023 Anthropic study gave the median neuron of a small model a 0 on its 14-point rubric, and the OpenAI project he started found about 1.7% interpretable; in preliminary audits of Qwen3-32B his team estimates roughly 90% of neuron poles have one main job, about 7% have two related jobs, and only 2-4% are not yet understood, with similar results in models up to trillions of parameters such as Kimi K3.

What remains, he argues, is mostly recursive generalization: a neuron sprouts related offshoots, as one that detected iy grew to detect diy and then do it yourself. Agents did much of the work: they reviewed 100,000 attribution graphs and tested explanations. In one check on 200 random neurons, an agent read only each explanation and wrote new examples and near misses; for the 183 neurons with positive-side explanations, new examples fired them strongly 76% of the time and near misses 11%, while texts on the most distant concept the agent could invent fired them strongly about 1% of the time. He adds that the one-job shares depend on where thresholds are set and are a broad pointer, not a precise score.

The newsletter's headline, that agents disproved polysemanticity, is stronger than the source. The essay calls itself an informal research essay and a working hypothesis, and its technical companion paper had not been published. The roundup records quick pushback: Neel Nanda said it did not change his cruxes and he still expects representational superposition, Leo Gao argued explanations should be judged by how well they simulate a neuron, Jake Mendel flagged the neglected tail of weaker activations, and Pfau noted the paper was not yet out.