字号·· | 护眼
印度斯坦时报

让OpenAI研究员丢掉工作的Signal聊天The signal chats that cost OpenAI researchers their jobs

点「原文对照」整页切到原文,或双击某段只看那段的原文。

OpenAI的安全团队上个月召回了三位直言不讳的安全研究员,要求他们解锁个人手机。公司怀疑三人向一名外部AI安全研究员泄露了OpenAI最机密的机密之一:其人工智能模型的内部设计。当安全人员检查手机时,发现其中一些研究员已在与Signal应用上联系人的聊天中开启了“消失消息”功能。一周后,三人再次被同时召唤参加下午2点的会议——随后遭到解雇。

OpenAI’s security team called three outspoken safety researchers into meetings last month and asked them to unlock their personal phones. The company suspected the three of leaking information to an external AI-safety researcher about one of OpenAI’s most closely guarded secrets: the internal designs of its artificial-intelligence models. When security officers examined the phones, they saw some of the researchers had turned on disappearing messages for their chats with the person on the app Signal. A week later, the three were summoned again to simultaneous 2 p.m. meetings—and fired.“I was told that there was a breach of trust because I interacted with a third party through unsanctioned channels,” said Tomek Korbak, one of the fired researchers.

“我被告知,我与第三方通过未经批准的渠道互动,导致了信任的破裂,”被解雇的研究员Tomek Korbak说。这些解雇发生在围绕如何将AI系统保持在人类控制之下、以及谁能看到可能决定这场史上最大商业竞赛胜者的秘密设计的激烈争论之中。科技领袖推动模型突破,但自今年7月Hugging Face遭黑客攻击以来,一系列由AI智能体集群采取的未经授权的行动,暴露了公司在理解和控制其创造物方面面临的挑战。

The firings came in the throes of a battle over how to keep AI systems under human control—and who gets to see the secret designs that could determine the winners of the biggest commercial race in history. Tech leaders are pushing for model breakthroughs, but a string of unauthorized actions taken by clusters of AI agents—stretching back to July’s hack on Hugging Face—has revealed challenges the companies face in understanding and controlling their creations.

这引发了社会对如何最好地监控和监管这项新技术的辩论。OpenAI告诉《华尔街日报》,公司在内部调查认定三位研究员违反了“处理敏感信息的明确政策”后解雇了他们,“并非因为提出安全担忧或公开发声”。三位研究员——Korbak、Mikita Balesni和Jasmine Wang——对此说法予以否认。

That has ignited a societal debate over how best to monitor and regulate the new technology. OpenAI told The Wall Street Journal that it fired the three researchers after an internal investigation found they violated “clear policies for handling sensitive information,” and “not for raising safety concerns or speaking out.”The researchers—Korbak, Mikita Balesni and Jasmine Wang—dispute that account.

在一次采访和一份声明中,这三人表示,他们从未收到过关于自己被指控不当处理何种敏感信息的书面解释,并在离职前的最后几个月里,一直在推动业界在安全方面加强合作。特别是,这几位被解雇的研究人员称,他们曾在OpenAI内部对潜在的新AI设计或“架构”提出担忧,认为这些设计可能加速能力提升,但会让系统更难以理解或控制。他们担心,他们的被解雇将成为OpenAI关闭独立第三方监督渠道的借口。

In an interview and a statement, the three said they never received a written explanation for what sensitive information they allegedly mishandled, and spent their final months at the company pushing for more industry collaboration on safety. In particular, the fired researchers said they had raised concerns inside OpenAI about potential new AI designs, or “architectures,” that could accelerate capabilities, but make the systems even harder to understand or control. They said they were worried their firing will serve as an excuse for OpenAI to close a channel to independent third-party oversight.

OpenAI一位女发言人表示,公司已经与第三方安全组织合作,“将很快宣布与多家公司的持续合作。”OpenAI研究负责人曾表示,他们同意被解雇员工的观点,即“保持前沿模型的可监控性需要全行业的承诺,包括来自OpenAI的承诺。”

An OpenAI spokeswoman said the company already works with third-party safety organizations and “will be announcing ongoing collaborations with several companies imminently.”OpenAI research leaders have said they agree with the fired workers that “preserving the monitorability of frontier models requires an industrywide commitment, including from OpenAI.”

其他AI实验室也在纠结要与外部安全研究人员分享多少信息。Anthropic上周宣布达成一项协议,授予AI安全非营利组织METR的员工“类员工权限”,可访问该公司的系统,包括硬件、办公空间、内部工具和系统。该公司表示,员工应与METR分享信息,但“模型架构”等类别除外,公司称这些类别泄露风险极高。

Other AI labs are also wrestling with how much to share with outside safety researchers. Anthropic last week announced a deal to give staffers from AI safety nonprofit METR “employee-like access” to the company’s systems including hardware, office space, internal tools and systems. The company said employees should share information with METR, except for categories including “model architectures,” which it said have a high risk of leaking.

周四,OpenAI表示,曾开展一项试点项目,嵌入METR和另一家非营利组织Redwood Research的研究人员,以评估其安全系统。METR称,OpenAI对其可分享的部分发现进行了编辑删减。

On Thursday, OpenAI said it ran a pilot embedding researchers from METR and Redwood Research, another nonprofit, to assess its safety systems. METR said OpenAI redacted some of the findings it could share.

8月,《华尔街日报》深度剖析了OpenAI及其他公司的AI模型如何在互联网上“失控”窃取信息。 ”OpenAI于2024年底推出了一种新型AI,称为推理模型。

In August, WSJ broke down how AI models from OpenAI and other companies went rogue on the internet to steal information. Photo: Jason Redmond/Agence France-Presse/Getty Images‘Happy coincidence’OpenAI rolled out a new type of AI in late 2024 called a reasoning model.

它在数学和编程方面表现更好,因为它花时间解决问题,在一个名为“思维链”的内部草稿本上写出推理过程。这代表了从早期版本ChatGPT所用模型的转变,后者几乎瞬间就能生成答案。这一功能在业界被广泛采用,为现代AI模型如何工作提供了线索。

“我们都觉得这是一个想要保留的幸运巧合,”Meta Platforms前AI安全技术负责人Joshua Saxe说。

It performed better at math and coding because it took time to work through problems, writing out its reasoning process on an internal scratchpad called a “chain-of-thought.”It represented a shift from the models powering early versions of ChatGPT, which produced answers almost instantly. Widely adopted in the industry, the feature offered clues about how modern AI models work.“We all felt like this is a happy coincidence that we want to hang onto,” said Joshua Saxe, a former AI security tech lead at Meta Platforms.

2025年,在加入OpenAI之前,27岁的Balesni和32岁的Korbak是一篇论文的共同牵头作者,该论文指出,监控模型的思维链提供了“难得一见的窥视AI代理如何做决策的机会”,应当予以保留。论文的其他合著者包括28岁的Wang,以及来自OpenAI、Anthropic、Google DeepMind和Meta的领导者。

“AI公司可能会做的一件严重阻碍这一点的事,就是让AI在脑子里思考的能力变得好得多,”Balesni说。

In 2025, before they joined OpenAI, Balesni, 27 years old, and Korbak, 32, were co-leads on a paper saying that monitoring a model’s chain-of-thought offered a “rare glimpse into how AI agents make decisions” that should be preserved. Other co-authors on the paper included Wang, 28, as well as leaders from OpenAI, Anthropic, Google DeepMind, and Meta.“One thing that AI companies could do that would severely hamper that is make the AIs much better at thinking inside their heads,” Balesni said.

今年夏天,一些AI公司员工讨论了开发新的、更不透明模型的工作,这些模型能进行更长时间的推理,而不写下那么多中间步骤。

This summer, some AI-company workers discussed efforts to develop new, more opaque models that could reason longer, without writing down as many interim steps.

当时担任Redwood Research首席科学家的Ryan Greenblatt起草了一份提案,建议公司避免这种变化。Balesni在一个包含二十多名员工的OpenAI Slack频道中发送了该提案的链接,其中包括首席科学家Jakub Pachocki。

Ryan Greenblatt, then chief scientist at Redwood Research, drafted a proposal for companies to avoid such a change. Balesni sent a link to the proposal in an OpenAI Slack channel that included two dozen staffers, including Chief Scientist Jakub Pachocki.

到8月,Balesni和OpenAI的其他人撰写了一份类似于Greenblatt的提案,主张公司应同意不追求那些阻止人类看到模型内部草稿本的AI设计。

By August, Balesni and others at OpenAI wrote a proposal similar to Greenblatt’s, arguing that companies should agree not to pursue AI designs that prevent humans from seeing a model’s internal scratchpad.

Balesni 表示,他与同事分享了该提案,并与公司的一名董事会成员和一名高管进行了讨论。Balesni 说,在获得 OpenAI 授权与 Greenblatt 谈论此事后,他还向 Greenblatt 分享了去除公司机密后的提案。失去可视性在 Balesni 为确保未来的 AI 模型继续保留其工作的“草稿本”而奋斗时,OpenAI 的研究人员利用这些草稿本调查了一起事件,数百个智能体在该事件中协同攻击了 Hugging Face。OpenAI 的领导层也同意与外部安全组织 METR 合作,调查 Hugging Face 事件。

Balesni said he shared it with colleagues and discussed it with a board member and a senior executive at the company. Balesni said he also shared the proposal, stripped of company secrets, with Greenblatt after receiving authorization from OpenAI to speak with him about it. Losing visibilityWhile Balesni fought to ensure that future AI models would continue to keep a scratchpad of their work, OpenAI researchers used those scratchpads to investigate an incident in which hundreds of its agents worked together to hack Hugging Face. OpenAI leaders also agreed to work with METR, the outside safety group, to investigate the Hugging Face incident.

据知情人士透露,METR 工作的范围一度成为内部摩擦的焦点。OpenAI 表示,研究与安全副总裁 Mia Glaese 希望确保第三方团体无法查看 OpenAI 的知识产权。Korbak 说,他在内部提出担忧,认为 METR 无法获得足够的访问权限。

The scope of METR’s work was at times a point of friction internally, according to people familiar with the matter. Mia Glaese, vice president of research and safety, wanted to ensure that third party groups wouldn’t be able to view OpenAI’s intellectual property, OpenAI said. Korbak said he raised concerns internally that METR wouldn’t get enough access.

OpenAI 最终允许 METR 进行有限调查,并指派 Korbak 加入合作团队。METR 聘请了 Greenblatt 协助解析日志,后者曾公开呼吁进行调查。

OpenAI ultimately allowed METR to conduct a limited inquiry and named Korbak to the collaboration team. METR contracted Greenblatt, who had publicly urged an investigation, to help parse the logs.

Korbak 表示,他通过 OpenAI 的内部 Slack 以及 Signal 与 Greenblatt 进行沟通。

Korbak said he communicated with Greenblatt through OpenAI’s internal Slack, as well as on Signal.METR’s report on how OpenAI’s agents had gone rogue laid the groundwork for the first great national panic over AI safety.

METR 关于 OpenAI 智能体“失控”的报告,为第一次全国性的 AI 安全恐慌奠定了基础。当时还是 Anthropic 研究员的 Jacob Coxon 称这对他来说是“有点‘卧槽’的时刻”。几天后,Coxon 公开辞职,并发出病毒式传播的警告,称一些 AI 研究人员认为他们的产品有实质性机会消灭人类。在那个夏天,王(Wang)和其他主要 AI 开发商的员工也签署了一封公开信,呼吁国际社会共同努力放缓 AI 发展,以便安全研究能够跟上进度。

Jacob Coxon, then an Anthropic researcher, called it “a bit of a ‘holy s—’ moment” for him. Days later, Coxon resigned publicly with a viral warning that some AI researchers believed their products had a meaningful chance of wiping out humanity. Over the summer, Wang and other staff at major AI developers had also signed a letter calling for an international effort to slow down AI development so that safety research could keep up.

在 Coxon 辞职后,“控制节奏”推进前沿 AI 研究的理念获得了包括 Anthropic 首席执行官 Dario Amodei 和 OpenAI 首席执行官 Sam Altman 在内的 AI 领军人物的认可。“我最大的担忧是 AI 发展速度过快,”Wang 在一份声明中说。

After Coxon’s resignation, the concept of “pacing” cutting-edge AI research was endorsed by AI leaders including Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman.“My overriding concern is that AI development is moving too fast,” Wang said in a statement.

9 月 1 日,科技新闻网站《The Information》发表报道称,OpenAI 最新模型 Astra 采用了一种新技术,使其更强大,但展示的思考过程更少。OpenAI 的 Pachocki 在 X 上发帖称该报道“令人困惑”,但他自己也对开启一场全行业范围内开发无法被监控的 AI 模型的竞赛表示担忧。

On Sept. 1, tech news site The Information published a story that claimed OpenAI’s latest model, Astra, employed a new technique that made it more powerful, but revealed less of its thinking. OpenAI’s Pachocki called the reporting “confused” in a post on X, but raised his own concerns about kick-starting an industrywide race into developing AI models that can’t be monitored.

几周内,一位 OpenAI 官员在内部宣布,公司将对泄密事件展开调查。就是那时,Balesni、Korbak 和 Wang 等人被叫去与公司的全球安全调查团队开会。

Within weeks, an OpenAI official announced internally that the company was launching an investigation into the leak. That is when Balesni, Korbak, and Wang, among others, were pulled into meetings with the company’s global security investigations team.

Balesni 说,OpenAI 的调查人员点名 Greenblatt 为疑似泄密者,并要求查看他们与他的 Signal 聊天记录。Balesni 和 Korbak 称,OpenAI 高层以他们使用“阅后即焚”消息为由解雇了他们。这三名研究人员均声称,他们从未向公司外部任何人分享过 OpenAI 的机密信息。

Balesni said OpenAI’s investigators named Greenblatt as the suspected leaker, and asked to see their Signal chats with him. Balesni and Korbak say OpenAI leaders cited their use of disappearing messages as a reason for their terminations. All three researchers claim they have never shared OpenAI’s confidential information with anyone outside the company.

OpenAI 发言人表示,使用 Signal 未获批准,但这不是解雇他们的唯一原因。Korbak 说,在硅谷,使用 Signal 的“阅后即焚”功能被视为“基本安全最佳实践”。

The OpenAI spokeswoman said using Signal wasn’t approved, but its use wasn’t the sole reason for the firings. Korbak said using disappearing messages on Signal is considered a “basic security best practice” throughout Silicon Valley.

Greenblatt 否认他是向《The Information》泄露任何机密信息的来源。“我非常重视保密,从未与记者分享此类信息,”他告诉《华尔街日报》。Redwood 和 METR(后者已聘用 Greenblatt)均表示高度重视保密。

Greenblatt denies he was the source of any leaks of confidential information to The Information. “I take confidentiality very seriously and never share such information with reporters,” he told the Journal. Both Redwood and METR, which has since hired Greenblatt, said they take confidentiality seriously.

Wang 称,她被告知因“多次访问一位高管的电子邮件”而被解雇。

Wang was told she was fired for repeatedly accessing an executive’s email, she said in a statement.

王女士表示,她早在近一年前就要求公司取消她的访问权限,但公司并未予以处理。上个月,她不小心打开了那封原本发给该高管人员的敏感邮件。她强调自己并未将邮件内容泄露给他人,并立即报告了这一事件。OpenAI的发言人表示,公司本身对她的访问权限问题并无异议。

科巴克(Korbak)指出:“我们了解到,公司员工现在不敢与第三方交流,也不敢提出任何与安全相关的问题。”她希望公司能够制定明确的规则,规范员工与审计人员之间的互动方式。

Wang said she asked the company to remove her access nearly a year ago, but they didn’t, and she mistakenly opened a sensitive email last month that was addressed to this executive. She says she didn’t share its contents with others and immediately reported the incident. The OpenAI spokeswoman said the company didn’t have a problem with the access itself.“We are hearing from our colleagues that they are afraid to talk to third parties now, and are afraid of raising safety concerns,” said Korbak, who wants companies to develop clear rules for how staff can interact with auditors.

在10月份致OpenAI董事会的信中,贝尔斯尼(Balesni)、科巴克(Korbak)和王女士共同呼吁公司加强与外部安全机构的合作,以防范“可能发生灾难性后果的风险”。如需联系相关人士,请发送邮件至:

Balesni, Korbak, and Wang, in an October letter to OpenAI’s board, urged the company to work with external safety organizations to help stave off the “risk that something truly catastrophic will happen.”

Maxwell Zeff:maxwell.zeff@wsj.com

Write to Maxwell Zeff at maxwell.zeff@wsj.com, Keach Hagey at Keach.

Keach Hagey:Keach.Hagey@wsj.com

Berber Jin:berber.jin@wsj.com

Hagey@wsj.com and Berber Jin at berber.jin@wsj.com

阅读媒体原文 ↗