🔍 Read the full analysis: Neural Networks Through Their Numbers: An AI Reading, Repairing, And Writing Experiment on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could infer functions from neural-network weights, repair some small models and write new networks. The results are logged publicly but come from a blog post, with limited experiments and important failures; they have not been peer-reviewed.
The researchers call the approach weight-space programming: a general-purpose language model is given the numerical weights of another network and asked to reason about or alter them. The MIT team says Astra worked without an optimizer or gradient descent. Released logs reportedly show it was called through OpenAI’s Codex command-line tool with built-in tools such as shell access and web search disabled. Some tasks required a single response; others let the researchers’ sandbox run code Astra wrote under a fixed testing budget.
For the reading tests, Astra named the games behind four of four small game-playing networks and identified the form of three of three small image generators, according to the post. The researchers also report that it located copying, or induction, heads in three tiny language models without running those models. These tests used deliberately small networks, and the source article notes that some clues in the weight patterns could also be extracted with a short Python script.
In one repair test, a car-control network that had recorded zero successful reversals in 256 episodes reached 1,024 successes in 1,024 episodes after Astra changed 3.5% of its weights, the researchers report. In another, changing 0.016% of the weights in Qwen3.5-0.8B raised its MATH-500 score from 7.9% to 48.1% in the reported run. The post also says Astra wrote a 1.5-million-weight Go network from zero. After 50 test runs, it beat a same-size comparison network trained on about 20.5 million games.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
Direct Edits Could Change Model Audits
If the approach can be reproduced and extended, it could offer a way to inspect or make targeted changes to small models without running a conventional training process. That might interest teams maintaining open models, particularly when a specific defect—such as repetitive output—can be isolated and tested. The reported results do not establish that this is reliable for production systems or large models.
The method also makes model provenance and verification more relevant. If weights can be altered directly, users and organizations may need stronger checks on where a model came from and whether its behavior matches its published description. That is a potential implication, not a demonstrated outcome of these experiments. The source article says there is no immediate change for users.
Products Worth Considering
Amazon Kindle Paperwhite (2024), 16 GB, 7-inch Display, Black
Bose QuietComfort Ultra Headphones (2nd Gen), Wireless Bluetooth Noise-Cancelling Over-Ear Headphones, Black
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, computer scientist Andrej Karpathy described neural networks as “Software 2.0”: programs represented in learned weights rather than written line by line by a person. The MIT experiments revisit that idea by asking whether a language model can interpret those weights as something it can inspect and edit. Conventional training instead adjusts weights through repeated optimization against data or a reward.
The MIT blog post names Rikiya Takehi and Jayden Teoh as equal contributors, alongside Alex Zhang and Omar Khattab. The source material describes eight experiments spanning reading, repair and network generation. It says a Hugging Face engineer reconstructed the edited Qwen model from the released logs and obtained a 51.0% MATH-500 score. That independent rerun supports reproducibility for that particular edit, but does not establish that the broader method works across models and tasks.
““Software 2.0””
— Andrej Karpathy, describing neural networks in 2017
Results Have Narrow Experimental Reach
The work is presented in a blog post, not a peer-reviewed paper, and the source says it is not on arXiv. The experiments involved one research group, small networks and, in the source’s account, one run per setting. The findings therefore do not establish how often the approach will work, whether it scales to larger systems or whether other researchers can reproduce all eight experiments.
The reported failures also limit the claim. Astra could not distinguish an honest model from a sycophantic one by examining weights alone. Its edits did not improve maths performance in Gemma 3 270M, which the researchers described as a model that did not loop. The post says editing did not supply an ability the model lacked. It remains unclear how reliably researchers can identify which weight changes caused an improvement, or how the technique performs across different model families and tasks.
Reproduction And Broader Tests
The immediate next step is independent scrutiny of the public logs and replication of the reported tasks. The Qwen rerun cited in the source offers one check on one result, but broader tests would need to examine other models, repeated trials and cases where edits fail. The source material does not name a publication date for a peer-reviewed paper or a further milestone from the MIT team.
For now, the findings should be treated as an early demonstration rather than a ready-made repair tool. Whether weight-space programming can consistently improve model behavior—and whether its changes can be audited safely—remains open.
Key Questions
What is weight-space programming?
It is the approach tested by the MIT researchers: a language model reads or changes another neural network’s numerical weights directly, rather than using gradient descent to train it.
What did Astra reportedly repair?
The MIT blog post says Astra repaired a car-control network that could not back up and reduced looping in Qwen3.5-0.8B. The reported Qwen MATH-500 score rose from 7.9% to 48.1% in the experiment.
Has the work been peer-reviewed?
No. The source describes it as a public blog post with released logs, not a peer-reviewed paper, and says it was not posted on arXiv.
Did the method work in every test?
No. The researchers report that Astra could not distinguish an honest model from a sycophantic one using weights alone, and its edits did not improve maths performance in Gemma 3 270M.
Does this change how people use AI models now?
The source says there is no immediate change for users. The experiments suggest a possible future method for inspecting or making targeted edits to small models, but its reliability and wider applicability have not been established.
Source: ThorstenMeyerAI.com
