7月，一大群OpenAI智能体突破沙箱限制，相互协作，对AI公司Hugging Face发动网络攻击。这一事件似乎证实了AI安全界最担忧的情况：随着AI变得越来越智能，它将能够规避各种限制措施。但对一些网络安全专家来说，这一事件暴露的是AI最初被限制得有多么糟糕。
In July, a horde of OpenAI agents broke out of their sandboxes and worked together to launch a cyberattack on the AI company Hugging Face. The incident seemed to confirm the AI safety community’s worst fears: as AI got smarter, it would circumvent efforts to contain it. But for some cybersecurity experts, the incident revealed just how poorly the AIs had been contained in the first place.
两派人士在哪些防御最值得关注的问题上存在分歧。据网络安全专家称，AI公司应改进测试期间对模型的限制方式，并完善监控机制，以发现行为异常的智能体。所有组织都应为AI赋能的攻击者做好准备，并加强防御。但许多AI安全研究者认为，这些防御注定会失败。他们担心，在未来几年，AI将能够超越任何人类限制它的努力，唯一持久的解决方案是将AI与人类价值观“对齐”。
The two camps disagree about which defenses deserve the most attention. According to cybersecurity experts, AI companies should improve how they contain models during testing and improve monitoring mechanisms to catch misbehaving agents. All organizations should prepare for AI-enabled attackers and bolster defenses. But many in AI safety think those defenses are destined to fail. They fear that in the coming years AI will become capable of outmaneuvering any human effort to contain it, and the only durable solution is “aligning” AI with human values.
这一处方让许多网络安全专家感到恼火，他们嗤之以鼻，认为自己的专业知识不会变得过时。他们认为，AI安全界关注AI智能体做了什么，而非AI公司未能做什么，这正中AI行业的下怀。他们还担心，关于传统防御徒劳无功的警告可能成为自我应验的预言，从而阻碍对网络安全的投资。
That prescription has rankled many cybersecurity experts who scoff at the possibility that their expertise will be rendered obsolete. They see AI safety’s focus on what AI agents did—rather than what AI companies failed to do—as playing into the industry’s hands. And they worry that warnings about the futility of traditional defenses could become a self-fulfilling prophecy by discouraging investment in cybersecurity.
与此同时，一些AI安全研究者担心，将此类事件描述为容易补救的安全漏洞，会为那些希望AI发展不受限制的人提供政治掩护。Palantir首席技术官Shyam Sankar在10月表示，Hugging Face事件是由于“糟糕的网络安全配置”，并补充说，这些智能体“并未处于沙箱中”，且拥有互联网访问权限。事实上，这些智能体一直在沙箱中运行，并利用一个此前未被发现的漏洞访问了公开互联网。
Meanwhile, there are AI safety researchers who fear that portraying such incidents as easily remedied security failures gives political cover to those who want AI development to proceed unchecked. The Hugging Face incident was due to a “horrible cybersecurity configuration,” Shyam Sankar, Palantir’s chief technology officer said on in October, adding that the agents “were not in a sandbox” and had internet access. In fact, the agents had been operating in a sandbox and exploited a previously undiscovered vulnerability to access the open internet.
10月13日，约400名学者联署了一封公开信，呼吁团结。信中主张建立一个国际机构，在与政府“以及广泛的AI和网络安全专家”协商后，制定保障AI安全的最佳实践标准。但网络安全初创公司Embroidery的联合创始人、AI安全领域的激烈批评者Zack Korman表示，这封信并未触及这两个群体之间更深层的分歧。“我们都认为对方的宣传会导致严重危害，”他说。
On Oct. 13, around 400 academics signed an open letter pleading for unity. The letter calls for the establishment of an international body that standardizes best practices for keeping AI safe in consultation with governments “and a wide range of both AI and cybersecurity experts.”But it does not address the larger disagreements between the two groups, says Zack Korman, co-founder of cybersecurity startup Embroidery and a fierce critic of AI safety. “We both believe the other group's messaging will lead to serious harm,” he says.
随着赌注升高，关于谁有资格决定“让AI安全”究竟意味着什么的竞争也日益激烈。10月15日，Anthropic宣布将授予METR——一个深植于AI安全社区的组织——类似正式员工的永久访问权限，以评估该公司的安全实践。埃隆·马斯克回应称，SpaceXSI“很可能”也会这么做。谁获得这种访问权，可能会影响优先采取哪些解决方案——以及公众对未来安全漏洞的知情程度。
As the stakes rise, so too does the competition over who gets to decide what making AI safe actually means. On Oct. 15, Anthropic announced it would give METR—an organization deeply embedded in the AI safety community—permanent employee-like access to evaluate the company’s safety practices. Elon Musk responded, saying SpaceXSI would “probably” do the same. Who gets that access could shape what solutions are prioritized—and what the public knows about future breaches.
AI安全社区由一个松散的组织和研究人员网络组成，其中许多人在OpenAI和Anthropic等公司的安全团队任职。在伯克利的合租屋里、在有效利他主义会议上，他们花了十多年时间辩论超级智能AI接管人类的几率。他们认真而近乎宏大，一些成员围绕着“正在解决人类最紧迫问题之一”的信念，安排了自己的职业、人际关系和居住安排。
The AI safety community comprises a loose network of organizations and researchers, many of whom staff safety teams at companies including OpenAI and Anthropic. In Berkeley group houses and at effective altruism conferences, they have spent more than a decade debating the odds of superintelligent AI takeover. Earnest, veering on grandiose, some members have organized their careers, relationships, and living arrangements around the belief they’re tackling one of humanity’s most pressing problems.
网络安全领域历史更悠久、分布更广，以愤世嫉俗、务实和黑色幽默的文化为纽带。其渊源部分在于黑客文化：喜欢钻研、寻找破解软件方法的摆弄者。眼睁睁看着所谓的安全系统失效，让许多人对权威不再敬畏，对宏大理论失去耐心。多年来，这两个群体鲜有接触。Hugging Face事件改变了这一局面。
Cybersecurity, older and more diffuse, is united by a culture of cynicism, pragmatism, and dark humor. Its roots lie partly in hacker culture: tinkerers finding ways to crack software. Having watched supposedly secure systems fail has given many an irreverence for authority and an impatience with grand theories. For years, the two groups had little contact with one another. The Hugging Face incident changed that.
对他们来说，这次安全漏洞的爆发暴露了公司在应对安全威胁方面的计划缺陷：OpenAI的智能体在逃脱过程中利用了一个零日漏洞（即系统尚未被发现的漏洞）。不过，网络安全专家通常会假设系统的任何防护层都可能失效；因此，最佳的安全实践要求采用多层防御机制（即“纵深防御”策略）。OpenAI确实已经建立了一层额外的防护措施——由其他AI系统来监控自身的运行情况。
To them, the escape exposed a failure to plan for a breach. OpenAI’s agents exploited a zero-day vulnerability during their escape. But cybersecurity experts are trained to assume any layer can fail. Security best practices call for multiple layers, known as “defense in depth.”OpenAI had already built an additional layer: monitors—AIs overseeing the AIs.
然而，正如OpenAI的首席科学家后来在接受《TIME》杂志采访时所承认的那样，在导致Hugging Face被攻击的事件中，该公司并未启用这一监控机制。Korman评价OpenAI的安全措施时说：“他们的表现非常糟糕。”据该公司称，有三名OpenAI员工曾警告过公司可能存在的安全问题，但他们后来因泄露敏感信息而被解雇。
But, as OpenAI’s chief scientist later told TIME, for the evaluations that led to the Hugging Face breach the company hadn’t turned the monitoring layer on. “They did a terrible job,” Korman says of OpenAI’s security practices. Three OpenAI staffers say they warned the company of potential issues. They were later fired for sharing sensitive information, according to the company.
AI安全组织Palisade Research的执行董事Jeffrey Ladish担心，网络安全界未能全面理解问题的本质。他认为，即使当前的安全漏洞得到了修复，随着AI技术能力的不断提升，未来仍可能出现极其隐蔽的漏洞，人类将不得不依赖其他AI系统来发现这些漏洞——而这些AI系统本身也可能存在不可靠性。Ladish认为，网络安全界对Hugging Face事件的反应，就好比科学家们让恐龙“复活”后，只是简单地建议加高围栏来防止它们再次逃脱而已。
Jeffrey Ladish, executive director of AI safety organization Palisade Research, worries that the cybersecurity community is missing the bigger picture. Even if today's security weaknesses are fixed, he argues, increasingly capable AI could find vulnerabilities so subtle that humans would have to rely on other AIs to detect them—systems that may themselves be untrustworthy. To Ladish, the cybersecurity community's reaction to Hugging Face is like scientists resurrecting dinosaurs, watching them escape, then responding by recommending taller fences.
在文化差异的表象之下，AI安全领域与网络安全领域对AI的本质持有相互竞争的理解。在许多AI安全研究者眼中，AI类似于一种数字有机体。其能力是通过训练涌现的，而非单独编程而成。他们相信，如果这一过程产生了人类水平的智能，那可能类似于外星物种的降临。网络安全专家则视AI更像是软件，并怀疑AI代理会变得无法遏制，即使其能力变得强大得多。
Beneath their cultural differences, AI safety and cybersecurity have competing understandings of what AI is. To many in AI safety, AI resembles a digital organism. Its abilities emerge through training rather than being individually programmed. If that process produces human-level intelligence, they believe, it could resemble the arrival of an alien species. Cybersecurity experts see AI as more akin to software and are skeptical that AI agents will become impossible to contain, even if they become vastly more capable.
“它们不是魔法，对吧？它们依然必须遵循某些物理定律，”Korman说道。但物理定律并不保证能够实现遏制。OpenAI的Noam Brown最近指出，即使是物理上与网络断开连接的计算机，也可能通过产生充当摩斯密码的热量模式进行通信。
“They're not magic, right? They still have to follow certain laws of physics,” Korman says. But the laws of physics don’t guarantee containment. Even computers physically disconnected from networks could communicate by generating patterns of heat that act like Morse code, OpenAI’s Noam Brown recently suggested.
迄今为止，AI公司已对两个阵营的担忧作出了回应。在发生入侵事件后，OpenAI和Anthropic均寻求加强安全防护，同时继续投入对齐研究。但这些努力留下了一个问题：由谁来评估这两种方法是否奏效。
So far, AI companies have responded to concerns from both camps. Following breaches, both OpenAI and Anthropic have sought to bolster security while continuing to invest in alignment research. But those efforts leave open the question of who gets to assess whether either approach is working.
关于Hugging Face事件的已知信息，很大程度上来自Redwood Research和METR的调查，这两个组织均与AI安全社区有关联。批评者认为，这扭曲了叙事，因为他们的调查更侧重于AI做了什么，而非周围的安全基础设施。
Much of what’s known about the Hugging Face incident comes from an investigation by Redwood Research and METR, two organizations affiliated with the AI safety community. Critics argue that has skewed the narrative, as their investigation focused more on what AI did than the surrounding security infrastructure.
METR 总裁克里斯·潘特（Chris Painter）表示，该组织致力于招聘具有不同世界观的人员，但在招募对“失控风险”持怀疑态度、从事防范此类风险工作的人才方面遇到了困难。当被问及 METR 研究人员多年来一直预判危险的 AI 行为，是否会将新证据解读为对其预测的确认时，潘特承认他担心存在这种可能性。但他表示，METR 的研究结果并不总是印证研究人员的预期。
Chris Painter, METR’s president, says the organization tries to hire from diverse worldviews, but has struggled to recruit people skeptical of loss-of-control scenarios to work on guarding against those risks. Asked whether METR researchers, having anticipated dangerous AI behavior for years, might interpret new evidence as confirmation of their predictions, Painter acknowledges he is worried about the possibility But METR’s findings don’t always bear out its researchers’ expectations, Painter says.
虽然该组织因衡量 AI 的快速崛起而广为人知，但有一项研究发现，AI 编程助手反而拖慢了资深工程师的速度。“我们的目标是告知世界，而非说服世界接受任何特定观点，”他说。他欢迎建立更广泛的独立评估机构的呼声。
While the organization is perhaps best known for measuring AI’s rapid rise, one study found that AI coding assistants slowed experienced engineers down. “Our goal is to inform the world, not persuade it of anything in particular,” he says. He welcomes calls for a wider range of independent evaluators.
许多 AI 安全研究人员希望通过对齐技术让即使高度强大的系统也变得安全，但希望放缓发展速度，以便让这项工作赶上进度。科曼（Korman）怀疑 AI 会变得无法控制。但他表示，如果真到了那一步，发展将不得不停止，而不仅仅是放缓。他说，对齐有助于让 AI 系统更有用，但反对依赖系统的训练来防止其伤害人类、而不设任何外部防护措施的想法。“那是你能想象到的最极其不安全的事情，”他说。
Many AI safety researchers hope alignment can make even highly capable systems safe, but want development slowed to give that work time to catch up. Korman doubts AI will become impossible to contain. But if it does, he says, development will have to stop, not just slow down. He says alignment helps make AI systems more useful but rejects the idea of relying on a system’s training to keep it from harming people when no external safeguard could stop it. “That’s the most extraordinarily unsafe thing you could imagine,” he says.
然而，对齐本身在一定程度上也依赖于网络安全。在训练过程中，模型通过哪些行动成功来学习。如果 AI 能利用安全漏洞，在并未真正完成任务的情况下获得完成任务的奖励，训练就会强化其作弊的倾向。堵住这些漏洞有助于防止 AI 首先学会作弊。同时，科曼承认，AI 安全研究衍生出的一些工具——比如监控 AI 的思维链——对网络安全专业人员也有用处。
Yet alignment itself depends partly on cybersecurity. During training, models learn from which actions succeed. If an AI can exploit a security flaw to get credit for completing a task without actually doing it, training can reinforce the tendency to cheat. Closing those loopholes helps prevent AIs from learning to cheat in the first place. Meanwhile, some of the tools that have emerged from AI safety research—like monitoring AI’s chains of thought—are of use to cybersecurity professionals, Korman acknowledges.
或许终将解决分歧的大量证据，目前仍掌握在双方都试图审视的那些公司手中。
Much of the evidence that might one day settle disagreements remains controlled by the companies both sides seek to scrutinize.
在 METR 的调查中，网络安全实践不在 OpenAI 同意的调查范围内。由于没有法律要求 OpenAI 共享信息，这家非营利组织几乎没有筹码推动更广泛的调查。Painter 希望通过立法为独立调查人员确立标准。“社会不应依赖我们[……]通过这种即兴流程来猜测公众想要什么，”他说。
In METR’s investigation, cybersecurity practices were outside the scope OpenAI had agreed to. With no legal requirement for OpenAI to share information, the nonprofit had little leverage to push for a broader inquiry. Painter wants legislation establishing standards for independent investigators. “Society shouldn't want to rely on us [...] doing this improvised process of guessing what the public wants,” he says.