

自从人工智能看起来更像科幻小说而非科学的时代起,一个想法就一直吸引着该领域的顶尖研究人员。
Since the days when artificial intelligence seemed more like fiction than science, one idea has captivated the field’s top researchers.
如今,领导人工智能实验室的人们相信,这一想法可能会改变他们的行业、全球经济,乃至人类的命运。OpenAI的首席科学家称这是保持领先地位的唯一途径。Anthropic的一位联合创始人说,这“可能是世界上最重要的技术趋势”。除非你在人工智能领域工作,否则你可能从未听说过它。
These days, the people leading AI labs believe it could transform their industry, the global economy and perhaps the fate of humanity. OpenAI’s chief scientist has called it the only way to remain on the frontier. A co-founder of Anthropic says it “might be the most important technical trend in the world.”And unless you work in AI, you’ve probably never heard of it.
这种现象被称为“递归自我改进”,即RSI,大致可以理解为人工智能构建更智能的人工智能。Anthropic将其称为“闭环”,即一个循环:每一次改进都让机器在自我改进方面变得更好、更快。
It’s a phenomenon known as “recursive self-improvement,” or RSI, which roughly translates to AI building smarter AI. Anthropic refers to it as “closing the loop,” a cycle in which each improvement makes the machines better and faster at improving themselves.
人工智能公司表示,他们现在已经看到递归自我改进的早期迹象,这正在加速他们的研究——并让所有人感到不安。随着这项技术加速自身演进,风险在于我们可能难以跟上步伐,并将其控制在掌控之中。
AI companies say they are now seeing the early signs of recursive self-improvement, which is speeding up their research—and freaking everyone out. As this technology accelerates its own evolution, the risk is that we struggle to keep pace and keep it under control.
关于这一深刻转变的最佳数据来自Anthropic,该公司最近发布了一张图表,展示其人工智能研究与开发中有多少是由人工智能完成的。
Our best data about this profound shift comes from Anthropic, which recently published a chart showing how much of its AI research and development is being done by AI.
要产生这张图表中的数据,需要一整个克劳德“村庄”。一个克劳德代理人挖掘了公司的Slack消息和内部文件,识别出15000项人工智能研发任务。一个克劳德侦探调查了这些工作的确切完成方式。然后,一个克劳德法官权衡证据,并将工作按六级标准进行评级,从无人工智能到完全自主的人工智能。
It took a whole village of Claudes to produce the data in this chart. A Claude agent dug through the company’s Slack messages and internal documents to identify 15,000 tasks of AI R&D work. A Claude detective investigated exactly how that work was done. Then a Claude judge weighed the evidence and rated the work on a six-level scale, from no AI to fully autonomous AI.
现在再看一眼那张图表,你会注意到Anthropic内部正在发生一些非凡的事情。
Now take another look at that chart and you’ll notice that something extraordinary is happening inside Anthropic.
最引人注目的柱状图是橙色的“AI主导”条,这意味着人工智能完成了大部分研发工作,而人类进行监督——这是六级中的第五级。
The most telling bars are the ones in orange for “AI leads,” meaning it performs most of the R&D while a human supervises—the fifth of the six levels.
一年前,AI 并未主导这项工作中的任何部分。直到今年 3 月,它仅主导了 1% 的工作。到了 8 月,Claude 已主导了未来版本 Claude 26% 的研发工作。而这一比例如今几乎可以肯定更高了。
A year ago, AI led none of this work. As recently as March, it only led 1%. By August, Claude was leading 26% of the work on future Claudes. And that number is almost certainly higher today.
尽管公司的研究人员正在将工作委托给 AI,但值得铭记的是,这项工作中 0% 是完全由 AI 完成的。它仍未达到第六级自动化。但以这种进步速度,AI 将人类排除在循环之外,可能只是时间、算力以及实验室内人类决策的问题。
Even as the company’s researchers delegate to AI, it’s worth remembering that 0% of this work is being done entirely by AI. It still hasn’t reached the sixth level of automation. But at this rate of progress, AI taking us out of the loop might just be a matter of time, computing power and the decisions of human beings inside the labs.
“我们尚未达到那一步,递归自我改进也非不可避免,”Anthropic 近期写道。“但它到来的时间可能比大多数机构做好准备的时间要早。”AI 公司表示,让模型训练、测试并微调下一代模型,对其成功至关重要。但它们也对这种成功可能意味着什么感到恐惧。悲观派已离开实验室,警告称自我改进的 AI 可能会毁灭我们所有人。该领域最著名的研究人员呼吁加强监管,CEO 们则呼吁在该快速发展的技术达到完全 RSI(递归自我改进)之前放缓脚步。
“We are not there yet, and recursive self-improvement is not inevitable,” Anthropic recently wrote. “But it could come sooner than most institutions are prepared for.”AI companies have said that models training, testing and fine-tuning the next generation of models is essential to their success. They’re also terrified of what that success could mean. Doomers have quit the labs and warned that self-improving AI might kill us all. The field’s most prominent researchers have asked for greater oversight and CEOs have called for a slowdown before this rapidly advancing technology reaches full RSI.
“如果不加控制,它可能会超越我们理解和控制这些系统的能力,因此必须非常谨慎地推进,甚至根本不应推进,”Anthropic CEO 达里奥·阿莫代伊曾写道。但自 AI 发展史上,人类一直被自我改进 AI 的概念所吸引。
“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully,” Anthropic CEO Dario Amodei has written, “if at all.”But humans have been tantalized by the concept of self-improving AI for the entire history of AI.
1956 年,开创性研究人员在达特茅斯学院聚会开展一个夏季项目时,他们就已经对机器如何自我改进感到好奇。巧合的是,提出这一开创性研讨会的科学家之一,正是一位名叫 Claude 的人——才华横溢的数学家克劳德·香农。
When pioneering researchers met at Dartmouth for a summer project in 1956, they were already curious about how machines could improve themselves. As it happens, one of the scientists who proposed that seminal workshop was a Claude—the brilliant mathematician Claude Shannon.
十年内,科学家们便开始梦想制造出在所有方面都比人类更聪明的机器,包括机器的设计。
Within a decade, scientists were dreaming about machines that would be smarter than us at everything, including the design of machines.
“因此,”另一位杰出的数学家I.J.古德写道,“前提是机器足够温顺,愿意告诉我们如何让它保持在控制之下,那么第一台超级智能机器将是人类永远不需要再做的最后一项发明。”几十年后,让这台机器诞生的男男女女,从未像现在这样担心自己控制它的能力。他们中的许多人自从听起来像科幻小说时起,就一直在思考AI系统自主构建AI系统的问题。他们现在不确定该作何感想。
“Thus,” wrote I.J. Good, another brilliant mathematician, “the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.”Decades later, the men and women bringing that machine to life have never been so worried about their ability to control it. Many of them have been thinking about AI systems autonomously building AI systems since it sounded like sci-fi. They aren’t sure what to think now.
“我不知道如何理清思绪,”Anthropic联合创始人杰克·克拉克最近写道。
“I don’t know how to wrap my head around it,” Anthropic co-founder Jack Clark recently wrote.
克拉克最初是一名科技记者,后来他对所报道的故事着迷到了想要亲身参与其中的地步。他加入了一家名为OpenAI的初创研究实验室,最终带着一小群同事离开,创立了Anthropic。
Clark started his career as a tech journalist before he became so obsessed with the story he was covering that he wanted to be part of it. He took a job at a fledgling research lab called OpenAI and eventually left with a small group of colleagues to start Anthropic.
“从那时起,我一直试图说服世界:总有一天,AI系统可能会变得如此强大,以至于能够改变国家安全、改变地球的经济轨迹,并最终可能获得自我构建的能力,”他在10月向国际货币基金组织发表演讲时说。“好吧,那个时刻已经到来。”就在几个月前,这个时刻似乎还在几年之外。它到来得甚至比克拉克预期的还要早。去年11月休完育儿假,他于2月回归时,面对的是一家截然不同的公司。在他换尿布的日子里,Anthropic已经变了样。“奇怪的事情开始发生了,”他在通讯中写道。
“Ever since, I’ve been trying to convince the world that one day, AI systems might become so powerful that they could alter national security, change the economic trajectory of the planet and perhaps eventually gain the ability to build themselves,” he said in an October speech to the International Monetary Fund. “Well, the moment has arrived.”Only a few months ago, this moment seemed a few years away. It arrived even sooner than Clark expected. After taking parental leave last November, he returned in February to a very different company. While he was changing diapers, Anthropic had changed. “Weird stuff had started to take place,” he wrote in his newsletter.
公司的模型突然变得足够好,以至于工程师们开始将编码工作外包给它们,有些人甚至完全停止了写代码。
The company’s models were suddenly good enough that engineers had started outsourcing their coding to them and some had stopped writing code altogether.
到了5月,事情变得如此离奇、如此之快,以至于他得出结论:递归自我改进(RSI)很可能在2028年底前实现。“这是一个不情愿的观点,”克拉克写道,“因为其影响太大,让我感到自身渺小。”随后,他再次缩短了他的时间表。
By May, stuff had gotten so weird, so fast, that he concluded that RSI was likely by the end of 2028. “It’s a reluctant view,” Clark wrote, “because the implications are so large that I feel dwarfed by them.”Then he shortened his timeline again.
他不再认为2028年是RSI最可能出现的年份。他认为是2027年。
He no longer believes 2028 is the most likely year for RSI. He thinks it’s 2027.
而且你不需要任何图表就能看出这意味着什么:虚构正濒临变为科学。
And you don’t need any charts to see what that means: Fiction is on the verge of becoming science.