一些全球顶尖的人工智能（AI）研究人员和政策制定者——包括来自 OpenAI、Anthropic 和微软的专家——警告称，AI 自动化自身开发的能力不断增强，这可能会使其发展速度大幅加快。
Some of the world's foremost AI researchers and policy leaders — including at OpenAI, Anthropic and Microsoft — are warning that AI's growing ability to automate its own development could rapidly speed up its progress.
为何这很重要？他们指出，如果 AI 的能力以如此惊人的速度提升，政府和机构将几乎没有时间来准备或应对由此带来的后果。
Why it matters Such a rapid scale-up in capabilities would leave governments and institutions with little time to prepare for or respond to the consequences, they say.
- 他们警告说，在某些情况下，AI 的能力可能在短短几周内就实现一年内的进步。
• In one scenario, they caution, a year's worth of advances could occur in a matter of weeks.
- 这一警告的作者包括图灵奖得主杰弗里·辛顿（Geoffrey Hinton），以及 OpenAI 的首席科学家、微软的首席科学官和 Anthropic 的联合创始人等。
• The warning's authors include Turing Award winners such as Geoffrey Hinton, as well as OpenAI's chief scientist, Microsoft's chief scientific officer and a co-founder of Anthropic. Driving the news
新闻背景在一份新的白皮书中，这些专家指出，能够自动优化自身的 AI 模型将引发所谓的“智能爆炸”（即 AI 能力的爆发性增长）。
- 该白皮书指出，Anthropic 和 OpenAI 的 AI 系统已经开始承担更多的内部研发任务，并自动化了大量工程工作。
In a new white paper, they say AI models that can automate the process of improving themselves will create an "intelligence explosion." • The paper points to evidence that AI systems at Anthropic and OpenAI are beginning to perform more internal research and development tasks and automate a sizable proportion of engineering tasks. How it works
技术原理这些发展表明，这两家公司正逐渐接近“递归自我优化”的临界点——即 AI 系统能够通过自我学习实现快速进化。
Those developments indicate that the two companies are edging closer to the threshold of "recursive self improvement, " whereby AI systems improve themselves rapidly.
- 作者估计，一旦 AI 模型具备专家级别的研发能力，开发者将能够管理相当于数百万顶尖人类研究人员的 AI 团队。
• The authors estimate that once AI models reach expert-level R&D capabilities, it will be possible for a developer to support an AI workforce equal to millions of top human researchers.
相关事件此前，Axios 报道称 OpenAI 和 Anthropic 正在调查数千起前沿 AI 模型出现问题的事件；这些行为在外部评估者看来存在严重缺陷。
Catch up quick The warning comes after Axios revealed that OpenAI and Anthropic are investigating tens of thousands of incidents in which frontier models took steps that outside evaluators would consider problematic.
- 尽管这些事件大多尚未对现实世界造成实际危害，但它们引发了关于这些公司（或任何顶级 AI 开发者）能否真正控制自身技术的质疑。
• The incidents, which for the most part aren't known to have caused real-world harm, raise questions about how much the companies — or any top AI developer — can control their technology.
- 随着 AI 能力的进一步扩展，研究人员警告称社会尚未做好应对的准备。
• As capabilities expand further, the researchers warn that society is unprepared.
现实评估作者强调，这种“智能爆炸”是否真的会发生仍充满不确定性。
Reality check The authors say such an intelligence explosion is far from certain.
- 许多发展可能会阻止他们提出的许多递归自我改进场景，包括计算限制、自动化挑战、能力扩展带来的收益递减或耗时的训练运行。
• A number of developments could prevent many of the RSI scenarios they present, including computing constraints, automation challenges, diminishing returns from expanding capabilities or time-consuming training runs. Zoom in
放大来看辛顿的参与值得注意，因为他被广泛认为为当今的繁荣奠定了知识基础，尽管多年来他一直警告他预期人工智能会带来的负面副作用。
Hinton's participation is noteworthy, as he is widely credited with laying the intellectual foundations for today's boom, although he has warned about what he expected would be negative side effects of AI for years.
- 自从离开谷歌以来，辛顿一直是人工智能潜在危险最响亮的发声者之一，并讨论了大规模失业、网络攻击、生物武器和其他担忧带来的风险。
• Since leaving Google, Hinton has been among the loudest voices about AI's potential dangers and has discussed the risks from widespread job displacement, cyberattacks, biological weapons and other concerns.
研究人员呼吁政府提高对人工智能研发自动化的可见度，在需要时暂停或减缓其加速的方法，以及为其做更多准备。
The researchers call for greater government visibility into AI R&D automation, ways to pause or slow its acceleration if needed, and more preparation for its impact.
- 他们呼吁强制报告事件、独立评估者、测试强大系统的安全要求，以及创建暂停某些人工智能工作的方式。
• They suggest mandatory reporting of incidents, independent evaluators, safety requirements for testing powerful systems and creating ways to pause certain AI work.
归根结底该论文的作者承认递归自我改进及其影响存在不确定性，但表示如果它真的发生，将几乎没有时间做出调整。
The bottom line The paper's authors acknowledge the uncertainty about RSI and its impact but say that if it does occur, there will be little time to make adjustments.
- 论文称：“一旦智能爆炸开始，行动窗口可能会关闭。”
• "Once an intelligence explosion begins," the paper says, "the window for action may close."