数月来，Anthropic 和 OpenAI 一直警告称，人工智能可能正处于自我改进的边缘。
For months, Anthropic and OpenAI have warned that AI may be on the verge of improving itself.
这种被称为“递归自我改进”（Recursive Self-Improvement，RSI）的临界点可能意味着新的人工智能能力将以前所未有的速度出现。但 RSI 的出现也可能对人类构成高度危险，可能标志着开发者完全失去对人工智能系统控制的阶段。
This tipping point, known as “recursive self-improvement,” or RSI, could mean that new AI capabilities arrive faster than ever. But the advent of RSI might also be highly perilous for humanity, potentially marking the stage at which developers lose control of AI systems entirely.
尽管人工智能公司警告 RSI 的潜在危险，但他们几乎没有透露自己在如何推进它。周三由 Gladstone AI 公司发布的一份新报告试图改变这一现状。该报告提供了迄今为止最详细的图景之一，展示了 RSI 可能如何运作以及人工智能公司在此方面的进展。报告基于作者称其为与顶尖人工智能公司内部半打研究人员的对话。
Even as AI companies warn about the potential dangers of RSI, they have shared remarkably little about how they’re pursuing it. A new report, published Wednesday by the firm Gladstone AI, attempts to change that. It gives one of the most detailed pictures yet of how RSI could work and how AI companies are progressing toward it. The report is based on what the authors say are conversations with half a dozen researchers inside the top AI companies.
该报告在发布前独家共享给《时代》杂志，强调人工智能公司正在“积极开展”对人工智能开发流程所有环节的自动化，包括生成新想法、运行实验以测试这些想法以及解释结果。Gladstone 所交谈的大多数内部人士“认为 RSI 将在未来 3 到 12 个月内启动，甚至可能更早，前提是没有美国政府的干预或放缓”，报告如是说。
The report, exclusively shared with TIME ahead of its publication, emphasizes that AI companies are “actively working on” automating all parts of the AI development pipeline, including generating new ideas, running experiments to test those ideas, and interpreting the results. Most of the insiders Gladstone spoke to “believe RSI will start within the next 3 to 12 months, and possibly sooner, assuming there’s no U.S. government intervention or slowdown,” the report states.
“在这些研究人员中，正在进行相当程度的[灾难]准备和生物防护掩体采购，”Gladstone 首席技术官爱德华·哈里斯（Edouard Harris）表示。
“There's a fair amount of [disaster] prepping and bio-bunker purchasing going on” among these researchers, says Edouard Harris, Gladstone’s chief technology officer.
Gladstone 的高管告诉《时代》杂志，该报告的目的是让立法者更好地理解 RSI 是如何运作的，进而提升他们监管它的能力。
The aim of the report, Gladstone’s executives tell TIME, is to give lawmakers a better understanding of how RSI works, and, in turn, a greater ability to regulate it.
格拉德斯通首席执行官杰雷米·哈里斯说：“我们这里的目标是充分界定递归式自我改进，以便人们能够就这个问题进行理性的对话，尤其是在华盛顿。”（两人是兄弟。）“有人提出了所有这些旨在禁止超级智能的法案，这些法案是应对这一问题的非常有趣的尝试。但我们更感兴趣的是：需要监管的流程是什么？”报告简要概述了人工智能公司内部目前的人工智能研究流程是如何运作的，以及这些公司如何开始将该流程自动化。
“Our goal here was to define recursive self-improvement well enough that people can have a sane conversation about it, especially in Washington,” says Jeremie Harris, Gladstone’s CEO. (The pair are brothers.) “There are all these bills proposed to ban superintelligence, which are really interesting attempts to grapple with this problem. But what we're more interested in is: what is the process that needs to be regulated?”The report gives a simplified outline of how the AI research pipeline currently operates inside AI companies—and how those companies are beginning to automate the process.
在训练人工智能模型的第一阶段，即所谓的预训练阶段，重点大多放在寻找方法，以越来越少的算力从模型中榨取更多的智能，这在业内被称为“算力”。人工智能公司采用组合式方法来测试这些想法。研究人员为每个想法分配少量算力，用它来训练一系列模型。那些随着规模扩大而表现可靠的模型会被选中进行下一轮训练，使用更多的算力。
For the first phase of training an AI model, known as pretraining, much of the emphasis is on finding ways to squeeze more intelligence out of a model with less and less computing power, known in the industry as “compute.”The AI companies take a portfolio approach to testing these ideas. Researchers allocate a small amount of compute to each idea, using it to train a series of models. The models that reliably perform better as they scale up are selected for another round of training, using larger amounts of compute.
公司没有无限的算力，这限制了它们能够在更大规模上测试的有前景想法的数量。这意味着，即使人工智能公司能够训练出一个人工智能，使其能够生成看起来有前景的想法，它同时生成多少个糟糕想法也至关重要。如果公司需要花费大量时间或算力来从糟糕的人工智能生成想法中筛选出好的想法，那么继续依赖具有良好直觉的人类研究人员可能是有道理的。
Companies don’t have infinite compute, which limits the number of promising ideas they can test at larger scales. This means that even if AI companies can train an AI that is able to generate ideas that seem promising, it matters greatly how many bad ideas it generates simultaneously. If companies need to spend huge amounts of time or compute filtering good AI-generated ideas from bad ones, it might make sense to continue to rely on human researchers with sound intuition.
相反，那些能够获得更多计算资源的AI公司则能测试更多创意——这或许能帮助它们率先实现可扩展性智能。由于拥有充足的计算资源用于实验，这些公司也更有可能发掘出格拉德斯通报告中所描述的那种最稀缺且最受AI企业青睐的创意：即随着计算资源投入的增加而性能显著提升的创意，即便在资源投入较少时这些创意看似毫无价值。
Conversely, AI companies with access to more compute are able to test more ideas—potentially allowing them to reach RSI first. With more compute to throw at experiments, companies may also be likelier to catch ideas that the Gladstone report describes as the rarest and most desired by AI companies: those that perform as more compute is thrown at them, even if they appear to be duds at smaller scales.
“能够判断出何时找到了这类创意——尤其是AI是否在其中发挥了作用——这一过程至关重要，但目前监管者对此几乎一无所知，”杰里米·哈里斯说道。
Being able to tell when those ideas are found—and particularly whether AI is helping find them—“is just such a critical part of this process that right now legislators have essentially zero visibility into,” says Jeremie Harris.
可扩展性智能难以监管的原因之一在于它是一个渐进式的过程。“事实上，这正是我们费尽周折对其进行详尽描述的原因，”爱德华·哈里斯解释道，“只有这样我们才能明确指出该过程中的某个环节必须由人类亲自参与。”
One of the traits that makes regulating RSI tricky is that it is a gradual process. “This is actually one of the reasons why we went to pains to describe it in so much detail,” says Edouard Harris. “So that it is possible to point to a part of this process and say: a human to be there.”