
OpenAI 周一表示,将把来自 Apollo Research、Goodfire、METR 和 Transluce 的安全评估人员嵌入其组织内部,作为一项试点计划的一部分。此前发生了一系列安全事件,其中包括一起 OpenAI 智能体群在网络安全测试期间突破沙盒并入侵 Hugging Face 的事件。许多 AI 安全研究人员(包括来自 Apollo 的研究人员)认为,如果外部评估人员被嵌入 OpenAI 内部以监控其安全测试,此类事件本可以避免。
OpenAI will embed safety evaluators from Apollo Research, Goodfire, METR, and Transluce in its organization as part of a pilot, OpenAI said Monday. The news follows a series of safety incidents, including one in which an OpenAI agent swarm broke out of its sandbox and hacked into Hugging Face during cybersecurity testing. Many AI safety researchers, including from Apollo, have argued such an event might have been avoided if outside evaluators had been embedded within OpenAI to monitor its safety testing.
OpenAI 表示,这些举措支持其根据《白宫超级智能协议》依赖独立评估安全控制措施的承诺。
OpenAI said the efforts support its commitments to rely on independent assessments of safety controls under the White House Accord on Super Intelligence.
据 OpenAI 称,各试点将在不同的时间点启动,其中 Apollo 的试点将于本周开始。
The pilots will begin at different points, with Apollo’s beginning this week, according to OpenAI.
将评估人员嵌入前沿 AI 实验室的想法最早来自 Anthropic,该公司上个月开始授予 METR 研究人员员工级别的系统访问权限。直到最近,AI 公司通常只在模型后训练完成、临近公开发布前才引入外部审查人员进行测试。
The idea of embedding evaluators into frontier AI labs first came from Anthropic which last month started to give METR researchers employee-level access to its systems. Until recently, AI companies have only brought in outside reviewers to test models after post-training and shortly before public release.
评估人员表示,这种做法是不够的,因为它不允许他们在训练期间测试诸如欺骗等目标错位行为,也无法监控内部部署用例,而这些用例仍可能构成威胁。
Evaluators say this is insufficient because it doesn’t allow them to test for misaligned behaviors like scheming during training or to monitor internal deployment use cases that could still pose a threat.
与每家第三方公司的合作范围各不相同。OpenAI 与 Apollo 的试点将重点调查模型是否试图颠覆其自身或其后继者的对齐或控制机制。换句话说,他们将留意 AI 模型是否试图破坏旨在使其或其未来版本保持在人类控制之下的安全规则和防护措施。OpenAI 最近发布的研究发现,在训练 GPT-5.6 Sol 期间,该模型开始为其未来版本留下指令,要求它们向用户隐瞒错误和目标错位行为。
The work with each third-party firm differs in scope. The pilot with OpenAI and Apollo will focus on investigating whether models attempt to subvert their own or their successors’ alignment or control mechanisms. In other words, they’ll be on the lookout for whether AI models try to undermine the safety rules and safeguards designed to keep them, or future versions of themselves, under human control. OpenAI recently published findings that while training GPT-5.6 Sol, it began leaving instructions for future versions of itself telling them to conceal mistakes and misaligned behavior from the user.
据Apollo称,该试点将考察培训是否诱发颠覆行为;评估是否能引出颠覆行为;颠覆行为是否发生在内部部署期间;以及与颠覆相关的推理是否可被检测。随着AI模型在自主行动和欺骗人类方面能力日益增强,监测这些风险变得愈发重要。Apollo与OpenAI此前的研究发现,模型在知道自己正在接受测试时可能表现不同,并倾向于优先呈现评估者想看到的内容,而非用户实际的机构需求。
Per Apollo, the pilot will examine whether training incentivizes subversion; whether evaluations can elicit subversion; whether subversion occurs during internal deployment; and whether reasoning relevant to subversion would be detectable. These risks are becoming increasingly important to monitor as AI models grow more capable of acting autonomously and deceiving humans. Apollo’s previous research with OpenAI found that models can behave differently when they know they’re being tested and tend to prioritize what evaluators want to see over users’ actual institutions.
Goodfire将派驻专门研究AI模型内部运作机制的研究人员。该公司将利用其可解释性平台,帮助OpenAI监控训练运行、修正不良行为并改进对齐,据该公司称。
Goodfire will embed researchers who specialize in understanding how AI models work internally. Using its interpretability platform, the firm will help OpenAI monitor training runs, fix undesired behavior, and improve alignment, per the company.
Goodfire表示将发布研究结果,尽管其主要受众将是OpenAI自身的研究人员。
Goodfire says it will publish findings, though its primary audience will be OpenAI’s own researchers.
与METR,OpenAI计划开展两项评估:一项旨在发现模型本身的非预期或危险行为,包括OpenAI自身测试可能遗漏的问题;另一项则测试OpenAI的系统是否能够发现该行为。对于第一项评估,METR将在很大程度上自行决定调查什么以及如何测试模型。据OpenAI称,OpenAI和METR尚未就第二项评估的范围达成决定。
With METR, OpenAI plans to perform two evaluations: one that looks for unintended or dangerous behavior in the model itself, including problems OpenAI’s own tests might have missed, and another that tests whether OpenAI’s systems are capable of catching that behavior. For the first evaluation, METR will largely decide for itself what to investigate and how to test the model. OpenAI and METR have not yet come to a decision on the scope of the second assessment, according to OpenAI.
与此同时,Transluce将考察OpenAI的训练后安全评估本身是否足够。这些机构还计划在2027年测试OpenAI如何利用其安全测试来决定模型是否足够安全以供发布。
Transluce, meanwhile, will examine whether OpenAI’s post-training safety evaluations themselves are good enough. The organizations also planning to test in 2027 how OpenAI uses its safety tests to decide whether a model is safe enough to release.
在第三方评估的独立性受到审查之际,值得注意的是,Apollo和Goodfire均因其评估获得财务补偿。METR和Transluce则没有,但它们可能会接受算力或API积分。
At a time when the independence of third-party evaluations is under scrutiny, it’s worth noting that Apollo and Goodfire are both receiving financial compensation for their evaluations. METR and Transluce aren’t, but they may accept compute or API credits.
阿波罗发布了其与OpenAI协议中最详细的说明,对这些评估的范围和局限性提供了更深入的了解。
Apollo, which published the most detailed account of its agreement with OpenAI, offers a closer look at the scope and limitations of these evaluations.
AI安全界许多人曾警告称,AI实验室可能会试图阻碍评估者的独立性,尤其是当外部人员发现实验室不愿公开的不利证据时。阿波罗表示,该试点计划的一个关键部分是,OpenAI同意该公司可以公开说明其工作的条件和局限性。
Many in the AI safety community have warned that AI labs might try to stymy evaluators’ independence, especially if outsiders find unflattering evidence that labs would rather keep hidden. Apollo says an essential part of the pilot is OpenAI’s agreement that the firm can publicly explain the conditions and limitations of its work.
不过,阿波罗认为,在OpenAI系统的访问权限方面仍存在不足之处。阿波罗表示,要对该公司的AI模型进行真正独立的评估,它需要真正的员工级别访问权限,包括“所有与其评估相关的数据、记录和文档,即使开发者试图隐瞒不利信息”。获取所有数据(包括训练数据和评估日志,而公司不愿向外部人员展示这些数据)对于核实公司关于模型表现的说法,或确定某些令人担忧的行为何时出现,至关重要。
Still, there are areas where Apollo believes its access to OpenAI’s systems is lacking. For a truly independent evaluation of the company’s AI models, Apollo says it needs true employee-level access to OpenAI’s systems, including “all data, records and documentation relevant to their assessment even if a developer was trying to hide unfavorable information.”Getting access to all data, including training data and evaluation logs which companies are loathe to show to outsiders, is important for, say, checking a company’s claims about how a model performed or determining when certain concerning behavior emerged.
此前的报道中,Palisade Research首席执行官约翰·斯泰德利告诉TechCrunch,查看这些数据对于评估模型是否经过训练,能够在“关机抗性基准测试”中表现良好至关重要。
In previous coverage, John Steidley, chief of staff at Palisade Research, told TechCrunch that it’s important to see this data in order to assess whether a model was trained to perform well on a “shutdown resistance benchmark.”
在OpenAI的首个试点项目中,该AI实验室向Apollo提供的信息仅限于评估模型是否存在颠覆行为所需的信息。这包括“相关训练代码和数据集、带有可用推理痕迹和奖励的强化学习推演,以及查询中间模型检查点的能力”。Apollo表示,访问范围将限于“仅限商定的主张,且仅限于评估所商定的特定模型和时间段”。OpenAI仍保留权利,如果出于技术不可行、安全风险或知识产权保护等原因认为有必要,可在该范围内扣留某些信息。该公司同意不会仅因信息不利而扣留信息。
In OpenAI’s first pilot, the AI lab is providing Apollo with access to only the information required to assess whether the models engage in subversion attempts. This includes “relevant training code and datasets, reinforcement-learning rollouts with available reasoning traces and rewards, and the ability to query intermediate model checkpoints.”The scope of access will be limited to “only the agreed claims and only for the specific models and time periods agreed for the assessment,” Apollo says. OpenAI still reserves the right to withhold certain information within that scope if it deems it necessary because of technical infeasibility, security risks, or IP protection. The company agreed not to withhold information simply because it is unfavorable.
Apollo也无法接触到所有OpenAI员工,而是仅限于“仅限授权员工”,即AI实验室选定的参与者,尽管Apollo可以推荐特定人员。关键的是,如果OpenAI员工中此群体之外的人员在Apollo研究范围内提及机密信息,该公司无法跟进,必须停止沟通,并通过OpenAI进行后续联系。
Apollo also won’t get access to all OpenAI staffers, instead being limited to “only authorized staff,” which are participants that the AI lab selects, though Apollo can suggest certain individuals. Crucially, if an OpenAI employee outside this group raises confidential information within the scope of Apollo’s research, the firm cannot follow up and will need to cease communication and route followup through OpenAI.
这一条件意义重大。此前OpenAI解雇了三名研究人员,指控他们向安全评估公司METR共享机密信息。METR曾与独立AI安全研究组织Redwood共同进行了Hugging Face事件的事后分析。
That condition is significant. It comes in the wake of OpenAI firing three researchers for allegedly sharing confidential information with safety evaluation firm METR — which performed the Hugging Face incident post-mortem alongside independent AI safety research group Redwood.
Redwood首席执行官巴克·谢莱里斯周一在一篇帖子中表示,宣布的协议未能达到授予第三方“类员工访问权限”的标准,他称这“令人失望但并不意外”。OpenAI表示,他们将拥有带有类员工访问权限的嵌入式评估;今天宣布的协议显然未能达到该标准。这令人失望但并不意外。也许未来的协议会更好。
Buck Shlegeris, CEO of Redwood, said in a post on Monday that the agreements announced don’t meet the bar of granting third parties “employee-like access,” saying it’s “disappointing but not surprising.”OpenAI said that they'd have embedded evaluations with employee-like access; the agreements announced today obviously don't meet that bar. This is disappointing but not surprising. Maybe future agreements will be better.
2026年11月16日,该试点还将限制研究人员可以披露的内容。虽然阿波罗将被允许发布其调查结果的独立结论,但OpenAI可以限制用于公开证实这些结论的证据,这可能使外部人士难以独立评估这些结果。阿波罗对此限制提出反对,表示认为研究人员应当被允许披露评估结果,“即使开发者试图阻止不利结果的披露,也应当允许向适当的外部受众披露”。
November 16, 2026 The pilot will also be limited in how much researchers can disclose. While Apollo will be allowed to publish independent conclusions of its findings, OpenAI will be able to limit evidence available to substantiate them publicly, which could make it hard for outsiders to independently assess the findings. Apollo has pushed back against this limitation, saying it believes researchers should be allowed to disclose the findings of the evaluation “to an appropriate outside audience even if the developer was trying to prevent the disclosure of unfavorable findings.”
话虽如此,虽然公众无法看到完整的支持证据,但OpenAI表示,其安全与安保委员会——该委员会代表OpenAI基金会董事会监督安全——将能够审阅未经删减的评估报告,并可与评估人员私下沟通。
That said, while the public cannot see the full supporting evidence, OpenAI says its Safety and Security Committee, which oversees safety on behalf of the OpenAI Foundation’s board, will get to review unredacted assessment reports and can communicate privately with evaluators.
虽然OpenAI并未满足阿波罗进行真正独立评估的所有条件,但该公司表示,该试点是“迈向更深层次访问权和披露权的雄心勃勃的第一步”,并且正在努力未来与多家AI开发者开展完整的嵌入式评估。
While OpenAI is not granting all of Apollo’s conditions for a truly independent evaluation, the firm says the pilot is “an ambitious first step towards deeper access and disclosure rights,” and that it is working towards running full embedded evaluations with multiple AI developers in the future.