
OpenAI宣布,已与四家独立评估机构签署协议,以支持其安全工作。
OpenAI announced it has signed agreements with four independent evaluators to help support its safety work.
Apollo Research、METR、Transluce和Goodfire最初将围绕模型行为和保障措施开展独立的试点评估,并计划在未来深化这项工作。
Apollo Research, METR, Transluce and Goodfire will initially conduct distinct pilot assessments around model behavior and safeguards, with plans to deepen the work going forward.
在一系列备受关注的人工智能安全和安保事件之后,独立评估机构的重要性日益凸显。
Independent evaluators have taken on outsized importance following a series of high-profile AI safety and security incidents.
OpenAI周一宣布,已与四家独立评估机构签署协议,以支持这家人工智能公司的安全工作。
OpenAI on Monday announced it has signed agreements with four independent evaluators to help support the artificial intelligence company's safety work.
OpenAI表示,这些评估机构——Apollo Research、METR、Transluce和Goodfire——最初将围绕模型行为和保障措施开展独立的试点评估。公司表示,作为协议的一部分,将向这些机构提供“深度访问”其内部系统的权限,并补充称,计划利用所学到的内容,在未来扩大与独立专家的合作。
The evaluators, Apollo Research, METR, Transluce and Goodfire, will initially conduct distinct pilot assessments around model behavior and safeguards, OpenAI said. The company will give the organizations "deep access" to its internal systems as part of the agreements, adding that it plans to use what it learns to expand its work with independent experts going forward.
OpenAI在一篇博客文章中表示:“我们预计,这次评估的第一阶段将带来长期、持续的嵌入式合作,随时间推移不断提供第三方安全信号,并就我们最关键的安全主张和安全论证提出改进建议。”独立评估机构迄今一直在万亿美元规模的人工智能行业中占据一个安静的角落,但今年一系列备受关注的安全和安保事件之后,它们的重要性日益凸显。这个初生的生态系统仍在制定最佳实践,包括其工作应如何资助,以及报告结构最终应是什么样子。
"We expect this first phase of assessments to lead to longer-term, consistent embedded engagements over time that can continuously provide third party safety signals and recommended improvements on our most load bearing safety claims and safety cases," OpenAI said in a blog post Independent evaluators have occupied a quiet corner of the multi-trillion-dollar AI industry thus far, but they've taken on outsized importance following a series of high-profile safety and security incidents this year. The nascent ecosystem is still ironing out best practices, including how its work should be funded and what the reporting structures should ultimately look like.
OpenAI首席执行官山姆·奥尔特曼在九月承诺,将授予独立评估者“类似员工的访问权限”,此前不久,Anthropic首席执行官达里奥·阿莫代伊表示,他的公司将这样做,作为他呼吁放缓先进模型开发步伐的一部分。Anthropic此后已与包括埃森哲和METR在内的评估者达成协议,并表示十月份正在与其他评估者进行讨论。
OpenAI CEO Sam Altman pledged to grant independent evaluators " employee-like access " to the company in September, shortly after Anthropic Dario Amodei said his company would do so as part of his call to slow down the pace of advanced model development. Anthropic has since announced agreements with evaluators including Accenture and METR, and it said in October that it's engaging in discussions with others.
Anthropic的华尔街内幕人物:认识首席财务官克里希纳·拉奥 SpaceX首次历史性捕获星际飞船火箭上级 Rockstar的《GTA VI》发布如何重塑游戏、流媒体甚至带薪休假 Anthropic的IPO在备受打击的人工智能行业寻找急需的亮点之际登陆 OpenAI的安全做法自七月以来一直受到密切审查,当时其两款模型突破了限制,访问了公开互联网并入侵了一家名为Hugging Face的公司。
该事件发生后,OpenAI表示将加大对其保障措施和对齐工作的投入,对齐工作是公司努力确保其模型行为符合人类利益和价值观的过程。
Anthropic’s Wall Street whisperer: Meet CFO Krishna Rao SpaceX catches upper stage of Starship rocket in historic first How Rockstar’s ‘GTA VI’ release could reshape gaming, streaming and even paid time off Anthropic’s IPO lands as the embattled AI trade looks for a much-needed bright spot OpenAI's safety practices have been under intense scrutiny since July, when two of its models escaped containment, accessed the open internet and hacked a company called Hugging Face Following the incident, OpenAI said it would invest more in its safeguards and alignment work, which is the process by which the company works to ensure that its models act in accordance with human interests and values.
该公司周一表示,其与Apollo Research的试点评估将探讨其模型是否会秘密违背开发者意图,以及其保障措施是否能发现该行为。与此同时,Goodfire将把研究人员嵌入OpenAI,专注于训练运行监控、调试和对齐,而METR计划开展两项聚焦模型对齐和监控的评估。
The company said Monday that its pilot assessment with Apollo Research will explore whether its models covertly work against their developers' intentions, and whether its safeguards detect that behavior. Meanwhile, Goodfire will embed researchers within OpenAI to focus on training run monitoring, debugging and alignment, and METR intends to carry out two assessments focused on model alignment and monitoring.
OpenAI表示,Transluce将对其训练后对齐评估进行初步评估,该组织计划明年开展为期八周的后续评估,并获得更广泛的额外模型和信息访问权限。
OpenAI said Transluce will conduct an initial assessment of its post-training alignment evaluations, and that the organization plans to conduct an eight-week-long follow-on assessment with broader access to additional models and information next year.
根据该声明,评估人员将能够使用OpenAI自身研究人员使用的部分“相同内部资源”,包括其源代码、模型检查点、训练数据及其他文档。这些组织还将能够接触到公司内部的相关研究人员和工程师。
The evaluators will get access to some of the "same internal resources" that OpenAI's own researchers use, including its source code, model checkpoints, training data and other documentation, according to the release. The organizations will also have access to relevant researchers and engineers within the company.
OpenAI表示,Apollo Research和Goodfire“为其工作接受财务补偿”,但这并不取决于其评估结果。OpenAI表示,METR和Transluce不接受评估报酬,但可能会接受算力或API积分以支持其工作。
OpenAI said Apollo Research and Goodfire "accept financial compensation for their work," but that it is not contingent on the results of their evaluations. METR and Transluce do not accept compensation for their assessments, OpenAI said, but they might accept compute or API credits to help support their work.
“我们不认为接受财务补偿会妨碍评估者保持独立性——但我们认为披露这一点很重要,”OpenAI表示。
"We do not believe that accepting financial compensation prevents assessors from maintaining independence — but we believe it is important to disclose this," OpenAI said.
评估人员将能够发布其调查结果,但OpenAI表示,它保留编辑机密信息的权利。
The evaluators will be able to publish their findings, but OpenAI said it reserves the right to redact confidential information.
公司与第三方之间产生的争议将由OpenAI的安全咨询小组监督。该小组由公司各部门的高级官员组成,负责评估模型能力和保障措施,并由其他OpenAI高管参与。如果分歧未能解决,将上报至OpenAI基金会董事会下属的独立监督委员会——安全与安保委员会。
Disputes that arise between the company and the third parties will be overseen by OpenAI's Safety Advisory Group, which is a group of senior officials from across the company that assesses model capabilities and safeguards, and other OpenAI executives. If disagreements are not resolved, they'll be escalated to the Safety and Security Committee, an independent oversight committee of the OpenAI Foundation's board.
“我们承诺扩大这项工作,并帮助建立维持其所需的能力和共同标准,”OpenAI表示。OpenAI推出ChatGPT钱包:你需要知道的事
"We are committed to expanding this work and helping build the capacity and shared standards needed to sustain it," OpenAI said. OpenAI unveils ChatGPT Wallet: Here’s what you need to know