Intel / article

[AINews] LLMs can read & write neural network weights: "Weight Space Programming" cuts Autoresearch out of the RSI loop

Source: latent.space
Author: swyx .ai
Date: Published Nov 3, 2026

Context

Researchers report that a frontier language model can edit the weights of a much smaller neural network directly, changing what it does without a training run. Normally a model is changed by training, which, as swyx's AINews issue on Latent Space puts it, means writing training code, running compute and watching loss curves. In the work the issue calls Weight-Space Programming, a large model instead inspects the raw weight tensors and proposes edits toward a goal stated in plain language. The newsletter compares this to reading and patching a small compiled program's ones and zeros instead of its source code.

The reported demonstrations are small. According to the summaries collected in the issue, the model identified the behavior of small networks from their raw weights, wrote small game-playing networks, repaired a failed car-control policy by editing 3.5% of its weights, and, in the headline result, raised a 0.8-billion-parameter Qwen model, Qwen3.5-0.8B, from 7.9% to 48.1% on the MATH-500 math benchmark by changing 131,072 weights, about 0.016% of the total. The summaries name the editing model as GPT-6 Astra and the lead author as Rikiya Takehi, and the newsletter says its former guest Alex Zhang is a coauthor.

Commenters whose critiques the issue summarizes argue that the math jump mostly repairs a failure rather than adding . At the 7.9% baseline, 93% of answers reportedly hit the token limit, which critics attribute to the small model looping under poor sampling settings; after the edit that share fell to 33%. On that reading the edit removed a looping behavior, similar to earlier work that removed a single refusal direction, and the same approach reportedly failed on Gemma 3 270M, a model without the looping problem. Another critique says interpreting dense middle layers failed until the model was allowed to build activation probes, which makes parts of the work closer to activation-based interpretability than to reading weights natively.

The newsletter calls the paper a blockbuster but also concedes that the method is a variant of mechanistic interpretability and unlikely for now to scale beyond simple games and reinforcement-learning policies. This explanation rests on the newsletter and the Reddit discussions it summarizes. The paper was not read, so its methods and full results are not checked here.

Key takeaways · AI-distilled

Researchers report that a frontier model directly edited 0.016% of Qwen3.5-0.8B's weights, with no gradient training, and raised its MATH-500 score from 7.9% to 48.1%. Critics note 93% of baseline answers hit the token limit, so the edit mainly removed a looping failure.
The same work reports a frontier model identifying small networks' behavior from raw weights and repairing a failed control policy by editing 3.5% of its weights. Critics say some results resemble known interpretability methods, and one test needed activation probes.
SemiAnalysis's new GDPx benchmark spans 206 agentic tasks in 11 economically valuable domains, including chip design, finance and biology. The best model, Opus 5.5, scores only 12%, with Gemini 4 Carbon close behind at 10.9% for roughly four times lower cost.
DeepSeek open-sourced a 'super-agent' prototype where one long-lived router session hands work to isolated parallel workers. In DeepSeek's own test, p95 completion time fell from 29.5 to 15.7 minutes versus a single session using subagents.
A Strata patch for local DeepSeek V4.1 Flash agent sessions spliced for a stable and cut new-session startup from about 100 s to 5 s. Commenters noted that agent tools injecting dates, paths or IDs into system prompts cause these prefix-cache misses.

Terms in this piece · Glossary

agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.

Read the source latent.space