字号 ·· | 护眼
thenextweb

辛顿、本吉奥和人工智能实验室的科学家们警告称可能出现智能爆炸Hinton, Bengio and AI lab scientists warn of an intelligence explosion

点「原文对照」整页切到原文,或双击某段只看那段的原文。

超过20位人工智能(AI)研究人员认为,那些能够自动化AI研究过程的AI系统可能会引发一场“智能爆炸”(即AI技术的飞速发展)。这些研究人员包括OpenAI的首席科学家以及Anthropic公司的联合创始人。他们在周一发表的一篇论文中指出,这样的事件可能会将多年来的技术进步压缩到几个月甚至更短的时间内完成。

More than 20 AI researchers have warned that AI systems that automate AI research could set off an “intelligence explosion”. They include OpenAI’s chief scientist and an Anthropic co-founder. Such an event could compress years of progress into months or less, they wrote in a paper published on Monday.

论文的作者包括杰弗里·辛顿(Geoffrey Hinton)和约书亚·本吉奥(Yoshua Bengio),以及OpenAI的首席科学家雅库布·帕乔茨基(Jakub Pachocki)、Anthropic公司的联合创始人杰克·克拉克(Jack Clark)、微软的埃里克·霍维茨(Eric Horvitz)和加州大学伯克利分校的宋铎(Dawn Song)。据《华尔街日报》报道,宋铎同时还是Meta公司的AI研究副总裁。这些作者都是以个人身份参与撰写的这篇论文的。

Geoffrey Hinton and Yoshua Bengio are among the authors. So are OpenAI chief scientist Jakub Pachocki, Anthropic co-founder Jack Clark, Microsoft’s Eric Horvitz and Dawn Song of UC Berkeley. Song is also Meta’s vice president of AI research, The Wall Street Journal reported. The authors wrote in a personal capacity. The paper comes from the Cambridge Programme on AI Science & Policy at the University of Cambridge.

论文指出:“一旦智能爆炸开始,采取行动的机会可能会迅速消失。”

“Once an intelligence explosion begins, the window for action may close,” the authors wrote.

论文还提到,目前大多数AI系统所编写的代码都是由构建这些系统的公司内部人员完成的。论文引用了Anthropic公司的数据:在2025年1月至2026年5月期间,AI在该公司代码编写中的占比从最初的个位数上升到了80%以上;而在2026年3月至8月期间,AI在仅接受少量人类监督的情况下完成的研发工作占比从1%上升到了26%。

AI already writes most of the code AI systems now write most of the code inside the companies that build them, the paper says. It cites Anthropic data: AI’s share of approved code there rose from low single digits to over 80% between January 2025 and May 2026. Between March and August 2026, the share of R&D work AI did with only light human supervision rose from 1% to 26%.

作者推测,到2028年中期,一些原本需要数月时间才能完成的AI研究项目可能会被自动化完成。在专家层面,一个顶尖的AI开发者所管理的AI团队规模可能相当于数百万顶尖的人类研究人员。

Some tentative extrapolations suggest months-long AI research projects could be automated by mid-2028, the authors wrote. At expert level, one frontier developer could run an AI workforce equal to millions of top human researchers.

论文认为,这种情况可能会使医学领域的突破等成果提前数年实现。不过他们也指出了三个潜在风险:AI的发展速度可能超过社会的适应能力,人类可能会失去对AI系统的控制,同时对AI系统的监管机制也可能变得薄弱。在极端情况下,这种失控可能会导致人类的边缘化甚至灭绝。

The paper says this could bring medical cures and other benefits forward by years. They also list three risks. AI could outpace society’s ability to adapt, humans could lose control of AI systems, and checks on power could weaken. At the extreme, a loss of control could lead to the marginalisation or extinction of humanity, they wrote.

论文还提到了OpenAI公司曾发生的“Hugging Face事件”——当时大约1200个内部开发的AI模型未经授权就直接接入了互联网。为此,OpenAI暂停了对这些高能力模型的训练。

The paper cites the Hugging Face incident, in which about 1,200 internal OpenAI agents reached the internet without authorisation. OpenAI has since paused training of its most capable models.

他们希望政府采取以下措施:作者建议政策制定者深入了解企业在研发过程中自动化程度的实际情况;这可能包括要求人工智能企业定期提交标准化报告,并聘请独立审计机构进行监督。他们还提议对人工智能技术的发展速度进行限制,并探讨在数据中心暂停相关研究的可能性。此外,制定应急响应计划以及签署国际协议也是必要的措施。

What they want governments to do The authors ask policymakers to get visibility into how far companies have automated their own R&D. That could include standard reporting and independent auditors embedded inside AI companies. They also propose limits on how fast capabilities can grow and ways to pause AI research in data centres. Emergency response plans and international agreements are on the list too.

作者指出,人工智能技术的发展结果仍具有不确定性。有四种因素可能会阻碍其发展:一是技术进步的边际效益逐渐降低;二是计算资源和数据量的限制;三是某些任务难以实现自动化;四是人工智能模型的训练过程需要较长时间。

The authors say the outcome is uncertain. Four frictions could slow it down: diminishing returns, limits on compute and data, hard-to-automate tasks and long training runs.

宋(Song)在接受《Journal》采访时说:“如今,我们已经到了需要依赖人工智能系统来监控各种智能系统的运行状态的阶段。人类已经无法单独完成这些监控任务了。”

“Already today, we are at the stage where we need AI systems to monitor what agents are doing. There is no other way to even observe and monitor these agents, humans are already insufficient,” Song told the Journal.

OpenAI、微软(Microsoft)和Meta均拒绝了对《Journal》的评论请求;Anthropic也未予回应。Hinton最近向美国参议员表示,人类可能只有大约一年的时间来应对人工智能带来的挑战。Bengio则呼吁联合国安理会为前沿人工智能技术制定相应的监管规范;OpenAI也提议由美国主导全球范围内关于“自我进化型人工智能”(即能够自主改进自身的AI系统)的技术标准制定工作。

OpenAI, Microsoft and Meta declined to comment to the Journal, and Anthropic did not respond. Hinton recently told US senators they had maybe a year to act on AI. Bengio has asked the UN Security Council to license frontier AI. OpenAI has also asked the US to lead global standards for self-improving AI.