作者：Seth Fiegerman Anthropic PBC、OpenAI、Meta Platforms Inc.和Microsoft Corp.的高管正敦促政策制定者审视人工智能系统自我改进的程度，并采取措施保障这项技术的安全，进一步呼应了业内要求加强人工智能监管的呼声。
By Seth Fiegerman Top executives from Anthropic PBC, OpenAI, Meta Platforms Inc. and Microsoft Corp. are urging policymakers to scrutinize the extent that artificial intelligence systems are improving themselves and take steps to safeguard the technology, adding to calls within the industry for greater oversight of AI.
在一篇新论文中，20多名人工智能行业领袖和研究员表达担忧称，利用人工智能帮助自动开发未来模型这一新兴做法，终有一天可能推动技术发展迅速提速，作者将这种现象称为“智能爆炸”。
In a new paper, more than 20 AI leaders and researchers expressed concerns that the emerging practice of using artificial intelligence to help automate the process of developing future models could one day lead to a rapid acceleration in the technology’s progress, a phenomenon the authors call an “intelligence explosion.”
论文称，这种情形将加剧人工智能能力发展速度超过社会驾驭和适应该技术能力所带来的风险。该论文由Anthropic联合创始人Jack Clark、OpenAI首席科学家Jakub Pachocki以及人工智能先驱Geoffrey Hinton和Yoshua Bengio等人撰写。研究人员还警告称，“自动化人工智能研发可能削弱人类监督。”论文呼吁政策制定者更深入地了解顶级人工智能实验室如何将研究工作自动化，同时制定机制“引导和约束智能爆炸”，包括对参与自动化人工智能研发的数据中心实施更严格的安全措施和监管。该团体还呼吁政策制定者考虑针对各种人工智能起飞情形制定紧急响应计划。
Such a scenario would heighten the risk that AI’s capabilities outpace society’s ability to steer the technology and adapt to it, according to the paper, which was authored by Anthropic co-founder Jack Clark and OpenAI Chief Scientist Jakub Pachocki, as well as AI pioneers Geoffrey Hinton and Yoshua Bengio, among others. The researchers also warned of the potential that “automating AI R&D could weaken human oversight.” The paper calls for policymakers to gain more visibility into how AI is automating research within the top AI labs as well as to develop mechanisms to “steer and constrain an intelligence explosion,” including by implementing stricter safety measures and oversight of data centers involved in automated AI research and development. The group also calls for policymakers to consider emergency response plans for various AI takeoff scenarios.
领先的人工智能开发商近期进一步就未来先进人工智能系统可能带来的灾难性风险发出警告。近期发生的一系列涉及人工智能智能体的安全事件，以及一些公司认为该技术正越来越善于自我改进，均加剧了这种担忧。Anthropic本月表示，如今其人工智能研发工作中已有超过四分之一由Claude聊天机器人主导，而年初这一比例几乎为零。
Leading AI developers have recently escalated their warnings about the potential for catastrophic risks from future, advanced AI systems, fueled by a series of recent breaches involving AI agents and a sense within some firms that the technology is getting better at improving itself. Anthropic said this month that more than a quarter of its AI R&D work is now led by its Claude chatbot, up from essentially nothing at the start of the year.
“初步证据表明，由软件驱动的智能爆发有可能发生，”周一发布的一篇论文这样写道。“如果这种情形真的发生，它或许将成为历史上影响最深远的技术发展。”Anthropic首席执行官达里奥·阿莫代本月早些时候发布了一篇长篇博客文章，呼吁整个行业支持一项更广泛的工作，以减缓最先进模型的开发速度。OpenAI的萨姆·奥尔特曼和埃隆·马斯克迅速表示支持阿莫代的计划。阿莫代和奥尔特曼上周还在联合国安理会发言时，敦促世界各国领导人共同努力应对人工智能问题。
“Preliminary evidence suggests that a software-driven intelligence explosion is possible,” according to the paper released Monday. “If one does happen, it could be the most consequential technological development in history.” Anthropic Chief Executive Officer Dario Amodei issued a lengthy blog post earlier this month calling on the industry to support a broader effort to slow the pace of developing the most advanced models. OpenAI’s Sam Altman and Elon Musk quickly lent their support to Amodei’s plan. Amodei and Altman also urged world leaders to work together on AI during remarks last week before the United Nations Security Council.
研究人员在这篇论文中表示：“相对于其潜在后果，我们的准备还远远不够。”随后，他们列出了政策制定者应当考虑的一些措施。“一旦智能爆发开始，采取行动的窗口期可能就会关闭。”《华尔街日报》此前报道了这篇论文。
“Relative to the stakes, we are not sufficiently prepared,” the researchers said in the paper, before outlining steps policymakers should consider. “Once an intelligence explosion begins, the window for action may close.” The Wall Street Journal reported on the paper earlier.