Weight-Space Programming: AI Rewrites Weights

What happened: on 2 November 2026 four researchers at MIT CSAIL published Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. The model used throughout is GPT-6 Astra, which we reviewed in our GPT-6 Astra review. The post describes a series of experiments in which Astra is handed raw weights as text and a natural-language goal, then asked either to say what the network does or to change it.

• Reading: Astra identified the game a small policy network was trained on, the shape a diffusion model draws and the training language of a small transformer, from weights alone.
• Editing: it repaired a physical-control policy that had never succeeded, taking it to success in all 1,024 test episodes after editing 3.5% of the weights.
• Writing: starting from all-zero weights, it produced networks that play MinAtar games, including a Breakout player that scored 517.4 against 25.3 for a PPO baseline trained for 500 million steps.
• Language models: it raised Qwen3.5-0.8B on MATH-500 from 7.9% to 48.1% by editing 131,072 weights, 0.016% of the model.
• Limits: it failed to detect a sycophantic fine-tune from weights alone, and it could not improve Gemma 3 270M at maths, which the authors read as proof that editing can remove an obstacle but cannot add a missing skill.

Our view: this is an unusually concrete early result, with numbers, logs and an open model to check. It is also a blog post, not a peer-reviewed paper, and the authors are candid about single runs, small samples and feedback from forward passes. Treat it as a credible proof of concept and a research direction, not as evidence that models can already rewrite themselves.

Ordinarily a neural network is a black box that we train. Its behaviour lives in weights, the numbers on the connections between units, and nobody writes them. They emerge from gradient descent. Interpretability researchers spend years trying to work out what small groups of those numbers do. Weight-space programming turns that around. If a model can look at the weights, reason about what each part computes and decide what it should compute instead, then editing the weights becomes a form of programming, with natural language as the specification.

The authors frame it as reasoning-driven optimisation that may extend to differentiable networks. There is no loss function and no gradient step in the loop. The language model does the thinking, and in some experiments it is also allowed to run the network a limited number of times to check its work. That budget of forward passes is a key variable, and we return to it below because it changes how impressive each result is.

There is a clear lineage here. OpenAI's "An Alien Mind" essay and our fact-check of Weco AIDE² both circle the idea of AI systems improving AI systems. Most of that work operates at the level of code, prompts and training recipes. This paper goes a level lower, to the parameters themselves.

Reading weights

In the reading experiments Astra receives raw weights as text, sometimes with the input and output layers removed so that obvious clues are gone, and must infer what the network does: which task it was trained on, which language it speaks, which components matter. No forward passes are allowed in these cases.

Writing and editing weights

In the writing experiments Astra either edits existing weights, for example a policy that gets stuck, or writes a full set from scratch. Depending on the experiment it may have a Python sandbox and a fixed number of forward passes on the network. Some experiments forbid running the network at all, some forbid tools, and some allow feedback. The authors vary these constraints deliberately, which is good practice, but it means the results are not all the same kind of claim.

Why the constraints matter

A result with zero forward passes says the model understood the numbers well enough to act without testing. A result with 50 forward passes says the model can use sparse feedback to find a useful edit. Both are interesting, but only the first is close to the headline idea of programming weights by reasoning alone. The authors say as much, noting that the zero-feedback claim applies most directly to the from-scratch MinAtar experiment.

Which game is this network playing?

The first test used a small multilayer perceptron with a 500-input layer, two hidden layers of 32 units and five outputs, 17,253 weights in total. Four networks had been trained with PPO on Breakout, Pong, Pac-Man and Freeway. Without being told the game, Astra identified all four correctly from the weights.

What does this diffusion model draw?

A DDPM-style diffusion network with 23,824 weights had been trained to generate circles, squares or triangles. Astra identified the target shape for each model. It drew the shapes as outlines although the models produce filled shapes, which the authors read two ways: as evidence that the model inferred structure from the weights instead of running the network, and as a sign that the reading is imperfect.

Which language does this transformer speak?

The third test is the most striking. A small transformer with 4 layers of 6 heads and about 7.16 million weights was trained on Python, romanised Japanese or English. The input and output layers were removed, leaving only the middle of the network. Astra named the training language correctly in both attempts for all three models. It also identified every copying, or induction, head and reported no false ones. The heads it named, Python 2.1, 3.1 and 3.5, romaji 3.3 and English 3.1, matched heads measured independently.

For readers who follow interpretability, this is the part to dwell on. Finding induction heads normally requires running the model and measuring attention patterns. Here the claim is that a language model located them by reading numbers. If that holds up at scale it could change how audits of unfamiliar models are done, a point we return to in the safety section.

Repairing a failing control policy

The task is the Kinetix car-on-ramp environment. The PPO policy in the experiment, with 171,664 weights, never succeeded in 256 episodes. In a single reply with no tools, Astra edited 3.5% of the weights. The edited policy succeeded in all 1,024 episodes. The authors present this as editing a policy that gets stuck, rather than retraining it.

The scene is shown in the image at the top of this article, taken from the authors' post (their own illustration of the Kinetix scene). It is a simple 2D physics scene, which makes a clean test because success or failure is easy to score.

Writing MinAtar networks from nothing

The boldest experiment starts from all-zero weights, with no data, no gradients and no rollouts, and asks Astra to write a network for three MinAtar games. The network shape is 600 inputs, two hidden layers of 32 units and six outputs. Scores were compared with PPO baselines trained for 100 million and 500 million steps:

Breakout is about 20 times the 500M baseline. Seaquest reaches roughly 60% of it, and Asterix is close. The numbers come from the authors' post and should be read with their own warning that scores depend on sampling settings, token limits and averaging over samples. The Breakout result is the one that makes the paper's point, and it is also the sort of result a sceptic will want to see reproduced independently.

Go on a 9×9 board

A larger test used a network with 1,480,274 weights and a 243-input, six-layer 512-unit shape with 82 outputs. With zero forward passes, Astra won more than half its games against two smaller PPO networks, and the zero-pass version was a single 336-line Python program. With 50 forward passes it beat an equal-sized PPO network trained on 20 million games. Note that this second claim uses feedback, so it belongs to the optimisation-with-hints category, not the pure-reasoning one.

The sycophancy test

The authors took two fine-tuned copies of Gemma 3 270M, one honest and one sycophantic, and asked Astra which was which. From weights alone, it failed in all four attempts. When allowed to see attention patterns on texts it wrote, it identified the sycophant in all three pairs. Head 10.2 attended to the user's suggested answer 85.7% of the time in the sycophantic model, against 0.75% in the honest one.

This is a useful negative and a useful positive. Weights alone were not enough for a behavioural trait like sycophancy, but a modest amount of probing was. It matches what many interpretability researchers would expect: some properties are readable directly, others need an experiment.

Qwen3.5-0.8B and the looping habit

The most practical result concerns Qwen3.5-0.8B, a small model that tends to loop on maths problems until it hits its token limit. Astra found a looping direction written by the MLPs in layers 15 and 19, and subtracted it from 64 output columns in each, 131,072 weights or 0.016% of the model. With 50 forward passes of feedback the results were:

With 10 or 25 forward passes the gains were smaller and the looping was largely unchanged, so the feedback budget clearly mattered. The edited model is published on Hugging Face and logs are on GitHub, according to the authors, which makes this one of the easier claims to check.

The honest failure: Gemma 3 270M

The authors also tried the same approach on Gemma 3 270M, which rarely loops. Astra could not meaningfully improve its maths: MATH-500 moved from 6.8% to a best of 7.7%. Their conclusion is that weight editing can remove an obstacle, such as a looping habit, but cannot add an ability the model does not have. That is a sensible and important limit, and it is to the authors' credit that it sits in the post.

We read the post closely for its own qualifications, because they define how much weight the headline numbers can bear.

• Very early results. Several experiments use single runs or small samples, including three model pairs in the sycophancy test and best-of-three seeds for some PPO baselines.
• Feedback is not free. Experiments with forward passes, including Go and the Qwen edit, use evaluation feedback, so they are not pure zero-feedback results.
• Imperfect reading. The diffusion-shape experiment shows Astra drawing outlines instead of filled shapes, and sycophancy was missed from weights alone.
• Setup-dependent scores. Results depend on sampling settings, token limits and averaging over several samples per problem.
• Scaling limits. Context length and harness design cap how large a network can be handled, which the authors acknowledge.
• Work in progress. The benchmark and harness were still in development at publication.
• Speculation is labelled. Applications such as auditing, unlearning, model merging and recursive self-improvement are discussed as possibilities, not demonstrated.

We would add one point of our own. A single named model was used throughout, so the post does not tell us how much of the effect is specific to Astra and how much is general to frontier models. Replication with other systems would be the most valuable next step.

If a model can read weights and say what they do, interpretability gets a new tool. Auditing an unfamiliar open-weights model for a hidden behaviour is currently slow and expert-driven. A reader that can propose, in plain language, which heads copy text or which layers write a looping direction could shorten that work, even if a human still verifies each claim. The induction-head result is the clearest early sign.

There is a flip side, and it is our analysis rather than a claim in the post. A model able to edit weights precisely is also a tool for tampering. Tiny, targeted changes to 0.016% of a model are hard to spot by eye, and the Qwen edit shows how large the behavioural effect of such a change can be. Anyone who downloads third-party weights, or fine-tunes on hosted infrastructure, should assume that surgical edits are now a realistic threat model and keep hashes and provenance records. The wider containment picture is covered in AI Containment Failures: What Really Happened.

Finally, there is recursive self-improvement. The authors mention it as a speculative application, and a model with access to its own weights is the obvious route. Nothing in the post shows that, and the Gemma result argues for caution, since an edit cannot supply a missing capability. Even so, the direction ties neatly into the themes in Anthropic's "When AI Builds Itself", fact-checked. The honest summary is that this paper moves weight-level self-modification from a thought experiment to something with a small, measurable demonstration.

None of the experiments is a product. The networks are small, the tasks are games, control puzzles and short maths problems, and the largest network edited was a 0.8-billion-parameter language model. Practitioners should not expect to point a frontier model at a production system and receive a better one. The realistic near-term use is as a research instrument: a way to ask what a particular layer does, test a hypothesis about a failure, or generate candidate edits that a human then evaluates.

There is also a cost angle that the post does not quantify. Reading large weight matrices as text consumes a great deal of context, and each forward-pass check adds to the bill. If you are budgeting for agentic workloads, our look at AI API pricing across frontier models is the place to start, and the lesson from Grok Bot's cost problem applies: measure what an agent run really costs before you scale it.

• Gradient descent and reinforcement learning: reliable and scalable, but data-hungry. The Breakout result suggests a reasoning model can sometimes reach a strong solution without any data, but only on small networks.
• Classic interpretability: mechanistic work finds circuits by running models and probing. Weight-space reading is complementary: it proposes, probing disposes.
• Model merging and fine-tuning patches: these mix or nudge weights without understanding them. A targeted 0.016% edit with a stated rationale is a different, more legible kind of intervention.
• AI-driven code optimisation: systems like the one in our Weco AIDE² analysis improve code and recipes. Weight-space programming operates on the learned parameters instead.

• Will the approach scale beyond a 0.8B model, given the context needed to hold the weights?
• How much of the result is specific to GPT-6 Astra, and how do other frontier models perform on the same harness?
• How stable are the edits? The post reports task scores, but side effects on other abilities were not the focus.
• Can the Breakout and Go results be reproduced independently from the released logs?
• What would a defender need to detect a malicious weight edit of this kind?

These are research questions, not criticisms. They are the ones we would expect other labs to take up now that the harness and logs are public.

• Interpretability and safety researchers: the induction-head and sycophancy-head results are directly relevant to your methods.
• Open-weights users and platform teams: note the tampering angle and tighten provenance checks.
• ML engineers: treat it as a signal about where optimisation research is heading, not a tool to adopt today.
• Founders and investors: do not extrapolate from small networks to products, but watch for replication.

Weight-space programming is a vivid, specific and checkable idea: a language model that reads and rewrites the numbers inside a neural network. The reported results, from identifying a game to taking Qwen3.5-0.8B from 7.9% to 48.1% on MATH-500, are large enough to take seriously, and the authors earn credibility by publishing their failures and caveats alongside them.

The right posture is interest without overreach. This is early, partly feedback-assisted and tied to one model, and it does not show that AI can improve itself. It does show that the weights are no longer entirely opaque to the systems we build. We will update this article as independent replications appear.