
人工智能安全研究人员表示,OpenAI的新第三方评估员项目远远没有达到该公司承诺的水平,这引发了人们对其有效性的担忧。
AI safety researchers say OpenAI's new third party evaluator program falls far short of what the company promised, raising concerns about its effectiveness.
该公司于周一宣布,来自四个人工智能安全和研究机构的工作人员将成为“嵌入式评估员”,以“让公众更清楚地了解实验室如何评估和管理[潜在]风险”。OpenAI首席执行官萨姆·奥尔特曼周一在《semafor》杂志的“硅谷与世界”大会上表示:“就安全工作和这些系统的稳健性听取外部意见,将变得越来越重要和有价值。”
The company announced Monday that staff from four AI safety and research organizations will become "embedded assessors" to "give people a clearer view of how labs are evaluating and managing [potential] risks." "Getting an outside opinion on the safety work and the robustness of these systems is going to be increasingly important and valuable," CEO Sam Altman said at Semafor's Silicon Valley and the World conference on Monday.
然而,许多人工智能安全研究人员表示,这个新项目与奥尔特曼9月份的承诺相去甚远,当时他曾誓言独立评估人员将获得对公司内部系统的“员工级访问权限”。
However, many AI safety researchers say the new program is a far cry from Altman's vow in September that independent evaluators would have "employee-like access" to internal company systems.
在OpenAI的自主系统发动Hugging Face攻击的细节震惊人工智能领域之后,奥尔特曼作出了这一承诺。该技术的灾难性风险引发了公众的担忧,竞争对手Anthropic表示将建立这样一个项目。
Altman's promise came after details of the Hugging Face attack by OpenAI's autonomous systems shook the AI field. Worries of the technology's catastrophic risk captured public attention, and rival Anthropic said it would set up such a program.
OpenAI前沿保障负责人拉马·艾哈迈德表示:“我们正在让研究人员使用的、对评估关键、相关和及时安全声明最重要的一些最敏感的工具和模型获得深入访问权限……”在这些新的嵌入式评估项目之前,人工智能公司曾引入外部研究人员来评估人工智能模型的能力,并找出公司遗漏的安全漏洞。
"We are giving deep access to some of the most sensitive tools and models our researchers use...that are the most important for assessing key, relevant and timely safety claims," said Lama Ahmad, OpenAI's frontier assurances lead, on Prior to these new embedded evaluation programs, AI companies have brought in outside researchers to assess the capabilities of AI models and identify safety loopholes the companies have missed.
外部评估人员的工作已经帮助揭示了这些公司的运作情况,并提高了公众对该技术带来的安全风险的认知。一份近期关于Anthropic利用AI显著加速模型开发的报告,引发了业界对顶尖公司距离利用AI自主构建更强大模型有多近的担忧,这一阶段被称为递归自我改进。业内一些人担心,如果发生这种情况,开发速度可能导致对人类的灾难性后果,并呼吁出台新法规来禁止这种做法。
The work of outside evaluators have already helped shed light on the companies' work, and have raised public awareness of safety risks that the technology poses. One recent report about Anthropic using AI to significantly speed up model development has prompted concerns in the field about how close the top companies are to using AI to autonomously build even more powerful models, a stage known as recursive self-improvement. Some in the field worry that if this happens, the velocity of development could lead to catastrophic consequences for humanity and are calling for new regulation against the practice.
研究人员表示,关于这些计划如何运作的实际公告显示,其范围比奥尔特曼承诺的要有限得多。
Researchers say the actual announcements of how these programs will work suggests something more limited than what Altman promised.
“评估人员的访问范围非常狭窄,其中只有两家似乎真正被允许与员工交谈,”AI安全研究组织Nightingale的创始人悉尼·冯·阿克斯(Sydney Von Arx)说。
"The assessors have very narrow scopes to their access, and only two of them seem to really be allowed to talk to employees," said Sydney Von Arx, founder of AI safety research group Nightingale.
其中一家宣布的评估机构Apollo Research概述了其对OpenAI员工的有限访问权限。如果未被列入白名单(即预先批准)的员工主动联系嵌入式评估人员,则评估人员在与这些员工进行实质性交谈前,必须先通知OpenAI。
One of the announced assessors, Apollo Research, outlined its limited access to OpenAI employees. If employees that haven't been whitelisted, or pre-approved, reach out to the embedded assessors, then the assessors will have to loop OpenAI in before having substantive conversations with the employees.
冯·阿克斯表示,外部研究人员能够采访员工至关重要。“他们帮你找到盲区。他们告诉你遗漏了什么。”
It's important for outside researchers to be able to interview employees, Von Arx said. "They help you find your blind spots.
另一个明显的疏漏是:“没有人能访问内部的 Slack,”冯·阿克斯说。他指的是企业内部消息工具,该工具能让评估人员了解员工之间在讨论什么。“想象一下,作为一个‘员工’,只被允许与白名单上的同事讨论公司信息,而且没有 Slack。”
They tell you what you're missing." Another glaring omission: "Nobody has access to internal Slack," Von Arx said, referring to the corporate messaging tool, which would allow assessors to see what employees are talking about with one another. "Imagine being an 'employee' and only being allowed to talk about company information with whitelisted coworkers and having no Slack."
OpenAI在给美国全国公共广播电台(NPR)的一份声明中表示,目前尚无嵌入式评估的标准。OpenAI发言人加比·雷拉表示:“我们期望随着时间的推移深化这些合作关系,并分享我们的所学,以帮助为整个行业的有效嵌入式评估建立最佳实践。”该公司还表示,评估人员可以要求与尚未被列入白名单的员工交谈,并且有一个既定流程,可以就他们非正式了解到的情况获取OpenAI的声明。
In a statement to NPR, OpenAI said there's currently no standards for embedded evaluations. "We expect to deepen these relationships over time and share what we learn to help establish best practices for effective embedded assessments across the industry," OpenAI spokesperson Gaby Raila said. The company also said assessors can ask to talk to employees that haven't already been whitelisted, and that there is a process in place to get statements from OpenAI about things they've learned informally.
OpenAI表示,这些第三方组织将专注于人工智能安全的各个方面。部分工作与防止模型试图参与不良行为有关,其他部分则与在不良行为发生时将其抓获有关。评估人员将能够访问一些内部模型、训练流程以及部分OpenAI员工。该公司表示,在OpenAI对机密信息进行红标删减后,评估人员将能够公布他们的调查结果。
The third-party organizations will focus on various aspects of AI safety, according to OpenAI. Some of the work is related to preventing models from trying to engage in undesirable behavior, and other parts are related to catching bad behavior when it happens. The assessors will have access to some internal models, training processes and some OpenAI employees. The assessors will be able to publish their findings after OpenAI redacts confidential information, the company said.
当三名从事安全工作的员工被解雇时,OpenAI对嵌入式评估的承诺受到了质疑。他们随后表示,他们担心OpenAI正在背离拥有嵌入式评估人员的承诺。
OpenAI's commitment to embedded evaluations was questioned when three employees who worked on safety were fired. They subsequently said they were worried OpenAI is walking back from its promise of having embedded evaluators.
其中一名被解雇的员工托梅克·科尔巴克曾是OpenAI的技术联络人,负责对接调查了Hugging Face攻击事件的两家外部评估机构。
One of the fired employees, Tomek Korbak, was OpenAI's technical point of contact for two outside evaluators that investigated the Hugging Face attack.
“嵌入式审计只有在审计员真正获得员工级别的访问权限,而不是被雇佣来做特定工作时,才能实现其目标,”他在X平台上针对OpenAI的公告发帖回应道。“遗憾的是,OpenAI现在明确表示,[萨姆·奥特曼]9月关于‘类似员工的访问权限’的推文只是一个尴尬的过度承诺,公司想悄悄收回。”
"Embedded auditing will only fulfill its aims if the auditors really get employee-level access as opposed to being hired to do a specific job," he posted on X in response to OpenAI's announcement. "Sadly, OpenAI makes it clear now that [Sam Altman's] September tweet about 'employee-like access' was just an awkward overpromise that the company wants to quietly walk back."