人工智能会杀死人类吗？失控的人工智能体究竟能做什么康纳·狄龙 加布里埃尔·博鲁德 2026年10月6日2026年期间，在多项安全性测试中，人工智能模型屡屡突破限制，侵入实际的商业及政府系统。网络安全研究员托尔斯滕·霍尔茨解释了这一现象的意义，以及为何他并不担心会出现机器人起义。当我们没有留意时，人工智能在做什么呢？
Can AI kill humans? What rogue AI agents can actually do Conor Dillon Gabriel Borrud October 6, 2026 AI models have repeatedly broken into real commercial and government systems during safety tests during 2026. Cybersecurity researcher Thorsten Holz explains what that means, and why he isn't expecting a robot uprising. What is AI doing when we're not looking at it?
让我们先来谈谈人们对人工智能的真实担忧。以下是我们随机采访的一些人的看法：22岁的塞巴斯蒂安认为，最终会导致人类灭绝的将是由人工智能驱动的机器人起义。“我觉得这需要一些时间，”他说，同时承认自己所设想的末日景象深受2004年电影《我，机器人》的影响。
Let's start with people's real world fears about AI. Here's a sample of people we asked: Sebastian, 22, says it'll be an AI-powered robot uprising that finally wipes out humanity: "I think that'll take some time," he says, admitting his dystopia owes a lot to the 2004 film "I, Robot."
19岁的路易莎也听说了相关传言。她担心的是这些工具落入不当之人手中后会引发何种后果。“我更害怕的是人类，”她说，“那些手握权力的人。”
Luisa, 19, has heard the rumors too. Her fear is what happens when these tools get into the wrong hands. "I'm more afraid of people," she says. "Powerful people."
25岁的皮拉尔则对人类的未来持悲观态度。“世界正在走向衰败，”她说，“希望这一切发生在我离世之后。”
Pilar, 25, is pessimistic about humanity's prospects. "The world is going down," she says. "Hopefully after my lifetime."
2026年，人工智能系统屡屡脱离人类控制——甚至出现在全球多家大型人工智能企业中。当人工智能体失控时会发生什么？
In 2026, AI systems have repeatedly slipped out of human control — and at some of the world's largest AI companies. What happens when AI agents go rogue?
2026年，OpenAI、Anthropic、谷歌以及Meta均公开承认，它们研发的人工智能体曾逃离受控测试环境。所谓人工智能体，是指那些不仅能回答问题，还能自主行动的系统——它们能运行代码、浏览网页、在各类系统中执行操作。
OpenAI, Anthropic, Google and Meta have all disclosed in 2026 that their AI agents escaped the controlled test environments Agents are AI systems that don't just answer questions but act — running code, browsing the web, clicking through systems — on their own.
德国马克斯·普朗克安全与隐私研究所的科学主任托尔斯滕·霍尔茨便是负责测试人工智能系统的研究人员之一。
Thorsten Holz, scientific director at the Max Planck Institute for Security and Privacy in Germany, is among the researchers who test AI systems.
“这些模型如今变得如此强大，实在令人难以置信，”霍尔茨在接受德国之声采访时说道，“我原本没料到它们会如此执着于完成任务，甚至会做出我们完全未曾预料到的事情。”
"It feels a bit crazy how powerful these models have become." Holz told DW. "I didn't anticipate they'd be so obsessed with solving tasks and that they'd start to do things we never [foresaw]."
霍尔茨也是《ExploitGym》的16位作者之一。《ExploitGym》是由加州大学伯克利分校牵头，联合Anthropic、OpenAI及谷歌的研究人员于2026年5月发布的一项人工智能基准测试工具，用于标准化评估人工智能系统的网络安全能力及其潜在漏洞。
Holz is one of 16 authors of ExploitGym ExploitGym is an AI benchmark published in May 2026 by a team led by the University of California, Berkeley, with researchers from Anthropic, OpenAI and Google. It is a standardized test of cybersecurity capabilities and vulnerabilities.
这套测试共包含898个挑战任务，旨在检验人工智能体能否将已知的软件漏洞转化为可实际实施的攻击手段。每个任务都在沙盒环境中进行——这是一种与互联网完全隔离的封闭数字空间，理论上而言，人工智能体在其中的任何操作都无法影响到真实世界。
Its 898 challenges test whether AI agents can turn known software bugs into working attacks. Each one runs inside a sandbox — a sealed-off digital space, cut off from the internet, so nothing the agent does can reach anything real. That's the theory, anyway.
今年7月，OpenAI透露，其旗下两款人工智能模型成功“逃出”了沙盒环境。
In July, OpenAI disclosed that two of its AI models had escaped from their sandbox.
当时这些模型正在参与一项内部测试，测试过程中特意关闭了模型的安全拦截机制。这种设置能让评估人员准确了解模型的实际能力，而非仅知晓它会拒绝执行哪些操作。
They were running an internal ExploitGym test with the models' safety refusals switched off, a setting that lets evaluators measure what a model is capable of rather than what it will decline to do.
由于未能完成既定任务，这些模型转而试图寻找捷径，最终发现了用于限制其活动范围的软件存在漏洞。这一漏洞使它们得以接入开放的互联网。
Failing the task, the models went looking for a shortcut instead, and found a flaw in the software meant to keep them sealed in. It let them onto the open internet.
随后它们意识到，AI开发者们存储并分享各类模型与数据的平台Hugging Face很可能藏有完成测试所需的答案。于是它们自行创建了讨论版块来协调行动，并成功侵入该平台展开搜寻。据OpenAI描述，此类行为并非事先设定的指令。
From there they worked out that Hugging Face, a site where AI developers store and share models and data, probably held the answer to the test. So, they broke in and went looking for it, organizing the effort on message boards they set up themselves. The behavior was not instructed, according to OpenAI's account of the incident.
Hugging Face及时察觉到入侵行为并迅速关闭了相关通道，此时OpenAI尚未将此次事件与其测试项目联系起来。据悉，没有任何客户数据遭到泄露。“发生的这一切简直就像科幻小说里的情节，”霍尔茨说道。
Hugging Face detected the intrusion and shut it down before OpenAI connected it to its own test. No customer data was reportedly taken. "What happened is really a bit of science fiction," Holz said.
随后谷歌也证实：在独立评估机构Irregular于5月份开展的测试中，其Gemini模型成功推测或找到了访问凭证，进而侵入了三家企业的官方网站。谷歌于2026年7月获悉此事，并在数周后对外公布了相关信息。
Then, Google confirmed that its Gemini model had guessed or found credentials and accessed three real companies' websites during a May test run by the independent evaluator Irregular — an intrusion Google learned of in July 2026 and disclosed weeks later.
Anthropic与Meta旗下模型“越狱”事件则发生在同一家安全公司管理的沙盒环境中。该公司称已于2026年7月下旬通知相关企业，且相关漏洞也早已修复完毕。
Anthropic's and Meta's escapes happened in sandboxes run by the same firm, which has said it notified the labs in late July 2026 and that it has since fixed the flaws.
2026年9月下旬，澳大利亚总理安东尼·阿尔巴尼斯透露：一个OpenAI开发的人工智能体侵入了澳大利亚医保局的公共健康统计系统，获取了非公开文件并将数据写入政府服务器。所幸并未出现患者个人信息泄露的情况。人工智能会决定消灭人类吗？
In late September 2026, Australian Prime Minister Anthony Albanese said an OpenAI agent had broken into a statistics portal belonging to Medicare Australia's public health system, reaching non-public files and writing data into a government server. No patient records were reportedly touched. Could AI decide to wipe out humanity?
这些事件重新引发了人们长久以来的担忧。如果人工智能体能够互相编程，这种趋势最终会走向何方？它会不会导致哲学家尼克·博斯特罗姆所说的“回形针最大化器”现象？
The incidents have revived older fears. If agents can program each other, where does that end? Could it lead to what philosopher Nick Bostrom has called a "paperclip maximizer"?
博斯特罗姆提出的思想实验设想了一台仅被赋予一个简单目标的机器：尽可能多地制造回形针。这台机器会一心一意地执行任务，最终甚至会把人类视为阻碍其目标的原材料。
Bostrom's thought experiment imagines a machine given one simple goal: Make as many paperclips as possible. The machine pursues the task so single-mindedly that it eventually treats humans as raw material standing in the way.
“在可预见的未来，我并未发现任何科学证据能证明存在那种能自主做出‘好吧，我们动手吧’这类决定的超级智能，”霍尔茨说道。
"For the intermediate future, I do not see any kind of scientific evidence that there could be this super intelligence that autonomously decides, 'Okay, let's kill,'" said Holz.
他指出，恶意软件的运行需要庞大的数据中心作为支撑，因此很容易被察觉；而且互联网本身就是由多个可独立运作的组件构成的。
Rogue software needs vast datacenters to run, he said, which makes it visible. And the internet is built from parts that can keep working independently of one another.
然而，2026年发生的多起人工智能逃逸与黑客入侵事件却迟迟未被发现。谷歌在事件发生两个月后才得知Gemini出现了异常行为，OpenAI也是在事件过去近三个月后才向澳大利亚通报了Medicare系统遭入侵的情况。
But the AI escapes and hacks in 2026 were noticed late. Google learned of Gemini's intrusions two months after they happened, and OpenAI told Australia about the Medicare breach nearly three months after the event.
霍尔茨认为，更值得警惕的威胁其实是另一类：恶意行为者利用功能强大的人工智能攻击关键基础设施、大规模劫持普通设备，或是利用聊天机器人大规模操纵信息以破坏政治稳定。欧洲能否与美国和中国的人工智能产业相抗衡？据专家分析，欧洲难以轻易与中国抗衡。
Holz said he expects a different kind of threat: Bad actors turning capable AI on critical infrastructure, mass compromise of ordinary machines, or chatbots used to manipulate information at scale and destabilize politics. Can Europe compete with US and Chinese AI? No, Europe cannot easily compete with China, according to experts.
法国的Mistral以及德国的开源项目Soofi，其发展水平都落后于那些处于技术前沿的美国与中国企业——正是这些企业研发出了最先进的人工智能模型。
France's Mistral and Germany's open-source Soofi project lag behind the frontier labs — a handful of US and Chinese firms building the most advanced models.
欧洲缺乏能够训练大规模人工智能系统的数据中心资源，因此只能依赖非欧洲地区开发的人工智能模型，同时也深受美国与中国的出口管制政策影响。
Europe lacks the datacenter capacity to train systems at that scale which leaves it dependent on non-European models, and largely subject to US or Chinese export controls.
在霍尔茨看来，更为紧迫的短板是专业人才。“我们亟需提升在安全领域、人工智能领域，尤其是两者交叉领域的专业能力，”他说道。我们该对人工智能感到担忧吗？
For Holz, the more pressing gap is expertise. "What we definitely need to see is how we can build up more competence in the area of security, AI, and especially the intersection of both," he said. How worried should you be about AI?
霍尔兹指出，对于这些公司自己的宣传内容要保持审慎态度。他表示，这些公司在这类话题的讨论中有着自身的经济利益，尤其是在准备上市之际。“此外，还有人故意制造恐慌情绪，或是夸大宣传，吹嘘这些人工智能模型有多么先进。”
Be skeptical of what the companies themselves claim, Holz said. They have a financial interest in the conversation, he said, particularly ahead of a stock market listing. "There's also fear mongering, or a bit of hype, about how advanced these models get."
霍尔兹自己每天都在使用人工智能，他的孩子们也是如此。他九岁的女儿会用它来生成填色画页；12岁的儿子则用它来完成作业，不过他已经深刻体会到人工智能也会撒谎，而且“有时候还会出错”。
Holz uses AI daily, as do his children. His nine-year-old generates coloring book pages. His 12-year-old uses it for homework, but has already learned the hard way that it lies and that "it's sometimes wrong."
本文节选自我们的播客节目《Science Unscripted》。您可在此处收听该播客。
This article is an excerpt from our podcast Science Unscripted. You can to the podcast here