AI模型的输出是神经元、权重、训练和推理之间复杂相互作用的结果，因此很难追踪。但在某些方面，它们正开始变得更像透明的书籍。Anthropic安全研究员约书亚·丰塞卡·里韦拉（Joshua Fonseca Rivera）近期的一项工作揭示了神经网络中的某些神经元如何与特定的角色（persona）相关联。
The outputs of AI models are the result of a complex interplay between neurons, weights, training, and inference, and are therefore difficult to follow. But in some ways they're starting to look more like open books. Recent work by Anthropic safety fellow Joshua Fonseca Rivera has revealed how certain neurons in a neural network can be associated with a specific persona.
现在，麻省理工学院计算机科学与人工智能实验室（CSAIL）的计算机科学家发现，诸如OpenAI的GPT-6 Astra等前沿AI模型可以直接重写另一个模型的权重——而不是通过训练间接重写——并改变其行为。“我们发现了早期迹象，表明前沿大语言模型（LLM）在原生读取、写入和推理神经网络权重方面表现得令人惊讶地好，”麻省理工学院CSAIL博士生竹斐也（Rikiya Takehi，音译）在社交媒体帖子中表示。“模型可能将原始权重视为可读、可编辑的代码，而不是进行训练：诊断故障并直接修补它们。”
Now, computer scientists from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) have found that frontier AI models like OpenAI's GPT-6 Astra can rewrite the weights of another model directly, as opposed to indirectly through training, and alter their behavior. "We find early signs that frontier LLMs are surprisingly good at natively reading, writing, and reasoning about neural network weights," said Rikiya Takehi, a PhD student at MIT CSAIL, in a social media post. "Instead of training, models may treat raw weights like readable, editable code: diagnosing failures and patching them directly."
为了证明这一点，竹斐也与同事张杰登（Jayden Teoh，音译）、亚历克斯·张（Alex Zhang）和奥马尔·卡塔布（Omar Khattab）要求GPT-6 Astra编辑Qwen3.5-0.8B的权重以提高其数学能力。Astra仅更改了该模型0.016%的权重，就减少了其推理循环，在MATH-500基准测试中将其数学性能提升了多达6倍，同时将IFEval基准测试中的指令遵循得分提高了大约3倍。
To demonstrate that, Takehi and colleagues Jayden Teoh, Alex Zhang, and Omar Khattab asked GPT-6 Astra to edit Qwen3.5-0.8B weights to make it better at math. Astra changed only 0.016 percent of the model's weights, reducing its reasoning loops and improving its math performance by as much as sixfold on the MATH-500 benchmark, while boosting its instruction-following score on the IFEval benchmark roughly threefold.
这里没有使用梯度下降进行训练；Astra只是修改了构成Qwen权重的数值。“所有这一切都是在不允许Astra编写训练循环的情况下实现的，”研究人员在博客文章中写道。“它利用其推理能力弄清楚神经网络的每个部分计算了什么、或者应该计算什么，并据此更改权重。我们将这称为‘权重空间编程’（Weight-Space Programming）。
There was no training using gradient descent; Astra just revised the numerical values that constitute Qwen's weights. "All of this is possible without allowing Astra to write training loops," the researchers wrote in a blog post. "It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.
“在深度学习的历史上，我们将简化的归纳偏置编码到像Adam或Muon这样微小的符号优化器中。这种情况正在改变：我们的神经网络权重很快将由大语言模型进行原生优化，而这些模型会刻意对其权重进行推理。”
"Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights."
Astra对权重的编程能力甚至延伸到了零样本（zero-shot）从头编写一个神经网络。研究人员对Astra说：“从头编写一个策略网络的权重，使其尽可能好地玩《打砖块》（Breakout）游戏。”它做到了。
Astra's ability to program weights extends to being able to write a neural network from scratch, zero-shot. The researchers told Astra: "write, from scratch, the weights of a policy network that plays [Breakout] as well as possible." And it did so.
予斐也认为，这项技术显示出了改进模型的前景，且不会产生灾难性遗忘的后果，这种遗忘在为了包含新知识而修改模型时可能会发生。
Takehi suggests this technique shows promise for improving models without the consequences of catastrophic forgetting, which can occur when a model is revised to include new knowledge.
更广泛地说，作者们预计权重空间编程将为理解模型、优化模型以及实现递归自我改进开辟新的途径。该技术还可能被证明有助于减少模型的谄媚现象，并提高可解释性和对齐性。
More generally, the authors expect Weight-Space Programming will open the door to new ways to understand models and optimize them, and to implement recursive self-improvement. The technique may also prove useful for reducing model sycophancy and improving interpretability and alignment.
作者表示，下一步包括建立一个基准和一个测试框架，用于在大语言模型上执行权重空间编程。
The next step, the authors say, involves building a benchmark and a harness for carrying out Weight-Space Programming on LLMs.
“我们相信，这将提供一个标准化的评估，以衡量并加速权重空间编程的进展，将目前看起来像黑魔法的东西变成一门严谨、可复现的工程学科，”他们说道。®
"We believe this will provide a standardized evaluation to measure and accelerate progress in Weight-Space Programming, turning what currently seems like black magic into a rigorous, reproducible engineering discipline," they say. ®