Fine-tuning teaches a model new behavior by gradient descent. A group at MIT CSAIL is arguing that a frontier model can reach into another network’s parameters directly and rewrite them.
Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab published the results Monday under the title Sparks of Weight-Space Programming. Their subject was Qwen3.5-0.8B, an 800-million-parameter open model with a documented tendency to loop.
What changed
Asked to improve math performance, GPT-6 Astra edited 131,072 weights, about 0.016 percent of the model. Accuracy on MATH-500 rose from 7.9 to 48.1 percent. GSM8K climbed from 13.8 to 67.1, and IFEval from 21.0 to 62.3.
The mechanism was narrower than the headline. Runs that hit the token ceiling on MATH-500 dropped from 93 percent to 33 percent. The edit largely removed a failure mode.
The same run also showed the limit. A gain of about one point was all the edit produced on Gemma 3 270M, a model that seldom loops. Nor was it able to spot sycophancy in a model when reading weights alone.
What comes next
The authors call these very early results with no peer review, though their logs are public.
Simon Willison reran one of their experiments Tuesday on much smaller hardware: Qwen 3.8 27B, a 17GB build served locally through LM Studio. Fed the same prompts, which carry raw weight values as text, the laptop model could still say which game a policy network had been trained on.
That matters more than the math jump. Reading weights as a language suggests interpretability work that once needed a research team can run on a workstation.
