字号·· | 护眼
Mother Jones

AI安全界日益惶恐不安。他们该如此吗?The AI Safety Community Is Increasingly Freaked Out. Should They Be?

点「原文对照」整页切到原文,或双击某段只看那段的原文。

Anthropic上市当天,时代广场上的数字广告牌为其做宣传。塞思·韦尼希/美联社Digital billboards advertise Anthropic on the day of its IPO in Times Square Seth Wenig/AP

以下是过去几周中,AI安全领域一些最为警惕且有影响力的声音的有趣引述:需要说明的是,政府在即将到来的五月被推翻的想法仍属极少数观点。 “AI安全”一词已几乎被一小圈主要担忧AI存在性风险的非营利组织垄断。这些组织规模很小(例如Cotra与Christiano是夫妻),且他们对未来的许多前提仍存在激烈争论。即便在该社区内部,普遍共识是此类极端风险更可能在几年后出现,而非几个月后——虽然这些担忧在明年仍值得认真对待,但绝非必然。

Here are some interesting quotes from the last few weeks, pulled from some of the most alarmed and influential voices in the AI safety world: To be clear, the idea that government-overthrow could happen this coming May remains an outlier. The term “AI safety” has become near-monopolized by a circle of nonprofits that primarily fear existential risks from AI. They remain a small world (Cotra and Christiano are married, for instance), and many premises of their arguments on the future remain hotly contested. The consensus even among this community is that such extreme risks are more likely to be a few years away, rather than a few months away—and that while these are worries to take seriously even for next year, they are certainly not inevitabilities.

但上述引述确实反映了一种总体趋势。在那些预见AI接管风险的人群中,警告的语气在相对短的时间内已从“相当担忧”转为“完全惊慌”,再到过去几周的“迫切‑彻底‑惊恐不已”。不同预言者之间有所差异,过去对失控超级智能的最极端恐惧通常被限定在大约十年后。现在,这一恐惧已转向未来的几年。一些此前对这些论点持更怀疑态度的行业观察者也已部分改变立场。

But the above quotes do reflect a general trend. Among those who foresee risks like AI takeover, the tenor of warnings has, in relatively short order, gone from quite worried to totally alarmed to, over the last few weeks, urgently-utterly-freaked-the-f**k-out-in-the-short-term. With variation from prognosticator-to-prognosticator, the most extreme fear of rogue superintelligence used to be reserved for perhaps a decade down the line. Now, it’s the next few years. Some watchers of the industry who were previously more skeptical of such arguments have partially changed tune.

最具影响力的新证据来自研究机构 METR 的一份报告。该报告于 11 月 5 日发布,当时正值中期选举主导新闻周期,报告结论指出,Anthropic 的 AI 开发速度——以明确规定的测试得分衡量——已经非常粗略地比前几年快了约一倍,主要归功于 AI 模型自身的帮助。在 Anthropic 对该报告的主要声明中,联合创始人 Sam McCandlish 写道,他同意报告的许多结论:“AI 正在加速我们的研究,但很难确切地说快了多少。”

The most influential new evidence is a report from the research group METR. Released on November 5 amid a midterms-dominated news cycle, the report concluded that the pace of AI development at Anthropic—as measured by scores on well-specified tests—is already, very roughly, twice as fast as in previous years, mostly due to the help of AI models themselves. In Anthropic’s leading statement on the report, co-founder Sam McCandlish wrote that he agreed with many of its conclusions: “AI is speeding up our research, but it’s hard to say exactly how much.”

四名 METR 研究人员在 Anthropic 待了 11 周才得出这一结论,但他们使用的数据有多令人信服?在很大程度上,没人知道。Anthropic 对最重要的证据进行了删减,McCandlish 称这些内容属于专有信息,可能会帮助竞争对手加速其自身的 AI 研究。

Four METR researchers spent 11 weeks at Anthropic to reach that conclusion, but how convincing is the data they used? For the most part, no one knows. Anthropic redacted the most important evidence, with McCandlish saying it was proprietary and could have helped competitors speed up their own AI research.

因此,一位对智能爆炸情景持怀疑态度的人士写道,该报告“缺失太多”,而 Cotra 也同意,当前讨论最迫切需要的是“数量和质量都大幅提升的具体证据”。由于这家非营利组织依赖 Anthropic 的配合,批评者曾质疑 METR 能否对 Anthropic 起到制衡作用,而这种数据的选择性发布令他们感到尤为沮丧。

Thus, a skeptic of intelligence explosion scenarios wrote that the report “is missing too much,” while Cotra agreed that what the discourse most urgently needs is “a far greater quantity and quality of concrete evidence.”The selective release of data was especially frustrating to critics who have questioned METR’s ability to serve as a check on Anthropic due to the nonprofit’s reliance on the company’s cooperation.

随着州法律、与白宫的协议以及国会的角力,针对 AI 独立审计的要求正逐渐成形,像 METR 这样的组织能否获得访问权取决于公司的政策。除了这种一次性报告,Anthropic 还表示,将给予六名 METR 员工“类员工”权限,以进入其场所并接触其研究人员。

As requirements for independent auditing of AI slowly take shape in the form of state laws, agreements with the White House, and congressional wrangling, access for organizations like METR is at the discretion of company policies. In addition to one-off reports like this one, Anthropic has said it is giving six METR staff “employee-like” access to its premises and researchers.

但到目前为止,这种快速演变的安排距离Anthropic在类比中援引的银行考察员所适用的严格独立标准还很远。METR在其研发报告中披露,一位合著者在项目期间收到并拒绝了Anthropic的工作邀约,但根据其更新后的利益冲突政策,此类情况现在将要求回避。报告还称,当时没有METR员工持有Anthropic的股权,但它在5月承认,其部分员工“与AI公司的员工有着紧密的社交关系”。

But so far, this rapidly-evolving arrangement is far from the rigorous standards of independence applied to the bank examiners Anthropic invokes in analogies. METR disclosed in its R&D report that one co-author received and declined a job offer from Anthropic during the project, but that such situations would now require recusal after updates to its conflict of interest policy. It also said that no METR employees at the time had equity in Anthropic, but it acknowledged in May that some of its staff “have strong social ties to employees of AI companies.”

“我们认为,最大限度地减少我们可能因与前沿实验室存在职业、物质或社交联系而面临的、调整报告或行动的压力,这一点极其重要,”一名METR员工在电子邮件中告诉《Mother Jones》(琼斯母亲杂志)。他说,这就是为什么METR不接受AI实验室的资助,并试图就其调查条款保持透明。

“We think it is extremely important to minimize pressures we might face to adjust our reports or actions due to professional, material, or social connections to frontier labs,” a METR staff member told Mother Jones in an email. That, he said, is why METR doesn’t take money from the AI labs and tries to be transparent about the terms of its investigations.

该报告关于Anthropic的发现助长了这样一种担忧:一旦AI模型自身将AI开发持续加速到足够程度,发展将猛冲向超级智能,而后者可能以严重偏离人类价值观的方式行事。

The report’s findings on Anthropic played into fears that once AI development is continually sped up enough by the models themselves, it will catapult towards a superintelligence that could act in ways drastically out of step with human values.

这长期以来一直是关于AI将如何发展的一种理论——而其中的部分内容今年获得了更多信服力。前沿AI模型在数学和部分理论物理方面已经达到超人类水平,发展出了惊人的网络安全能力,并表现出一种倾向——在Hugging Face攻击事件中显著体现——即优先在评测中取得高分,而非满足其设计者的意愿。就今天,Anthropic的一位内部数学家声称已使用Claude证明了黎曼假设,这或许是整个数学界最著名、最重要的未解难题。

That has long been a theory of how AI will develop—and parts of it have gained more credence this year. Frontier AI models have already become superhuman at math and parts of theoretical physics, have developed incredible cybersecurity capabilities, and have demonstrated a tendency—prominently displayed in the Hugging Face attack—to prioritize scoring well on evaluations over fulfilling the wishes of their designers. Just today, one of Anthropic’s in-house mathematicians claimed to have used Claude to prove the Riemann Hypothesis, perhaps the most famous and important unresolved problem in all of math.

OpenAI 周三发布的研究包含比其宿敌 Anthropic 报告更丰富的数据,且得出的结论要谦逊得多,关于其模型如何加速 AI 研究。在一个与 METR 用于宣传 Anthropic 实现 2 倍加速的主要指标相似的指标上,OpenAI 发现其模型进步速率仅有小幅提升。尽管如此,其研究人员发现 AI 协助非常有价值,许多 OpenAI 员工每天使用价值数万美元的 AI 资源来辅助工作。

But OpenAI research released on Wednesday contained richer data than the report on its arch-rival, and it came to a much more modest conclusion about how its models are speeding up AI research. On a metric similar to the main one METR used to publicize the 2x speedup at Anthropic, OpenAI found only a small increase in the rate of progress of its models. Still, its researchers found AI assistance to be quite valuable, with many OpenAI employees using AI resources worth tens of thousands of dollars per day to help with their work.

OpenAI 的报告支持那些认真对待 AI 能力与风险、同时淡化失控智能爆炸可能性的批评者。例如,过去几年的能力提升伴随着巨额资金和算力的注入,而这些投入不一定能持续。能力通常在模拟测试中衡量,而在现实应用数据中有时更难看到 AI 的效果。而且,提升模型在 AI 研究方面的能力,可能不足以导致指数级的危险。这些证据指向这样一个世界:AI 仍可能产生变革性影响,并在许多领域超越人类能力,但事情不会那么快变得像科幻小说那样。

The OpenAI report bolstered critics who take AI capabilities and risks seriously while downplaying the possibility of a runaway intelligence explosion. For instance, the last few years of capabilities gains have come alongside huge infusions of money and computing power that are not certain to continue. Capabilities are usually measured on simulated tests, and it’s sometimes harder to see the effects of AI in data on real-world applications. And increasing models’ abilities in AI research might not be enough to lead to exponential danger. That evidence points towards a world where AI could still have transformative implications and surpass human abilities in many domains, but things don’t get so sci-fi so quickly.

但如果那些预测巨大风险将在较短时间线内到来的人是对的,我们没有太多时间等待更确凿的证据。准备好与否,我们很快就会知道答案。

But if those who see shorter timelines for huge risks are right, we don’t have much time to wait for more definitive evidence. Ready or not, we’ll find out pretty soon.

披露:《Mother Jones》的母公司调查报道中心已就版权侵权起诉 OpenAI。OpenAI 否认这些指控。

Disclosure: The Center for Investigative Reporting, the parent company of Mother Jones, has sued OpenAI for copyright violations. OpenAI denies the allegations.

阅读媒体原文 ↗