字号 ·· | 护眼
连线

蒂姆尼特·格布鲁认为人工智能不存在“生存威胁”Timnit Gebru Believes There Is No ‘Existential Threat’ From AI

点「原文对照」整页切到原文,或双击某段只看那段的原文。

独走。突破遏制。目标不对齐。末日概率(P(doom))。随着行业的蓬勃发展,在关于人工智能究竟会带来多大影响的争论中,涌现出了一套全新的专有词汇。

Going rogue. Breaking containment. Misalignment. P(doom). As the industry booms, an entirely new vernacular has emerged in the debates over how consequential artificial intelligence really is.

在这套新语境中,两个短语成了焦点:随机鹦鹉(stochastic parrots)和存在风险(existential risk)。而处于这些讨论中心的,是一位从未在论战中退缩过的杰出AI研究员——蒂姆尼特·格布鲁(Timnit Gebru)。

Within that new parlance, two phrases have become focal points: stochastic parrots and existential risk. And at the center of those is one prominent AI researcher who has never stood down from a fight, Timnit Gebru.

格布鲁几年前进入公众视野,当时她与谷歌发生争执,起因是她联合署名的一篇研究论文指出了该公司AI中存在的偏见,并指出大语言模型(LLM)基本上是在鹦鹉学舌般地重复其训练数据,并有可能使偏见观点长期存在。这篇备受争议的论文导致格布鲁离开了该公司,并促使她创立了一个研究所,专门调查技术造成的危害并支持创建无偏见的技术工具。

Gebru came into the public eye several years ago after sparring with Google over a research paper she coauthored that called out biases in the company’s AI, saying that LLMs basically parroted their training data and risked perpetuating biased viewpoints. The contested paper resulted in Gebru’s departure from the company, and led her to found an institute that investigates harms perpetuated by technology and supports the creation of unbiased tech tools.

她还撰写了一本新书《深度遗忘:AI的崛起与一个科技理想主义者的激进化》(Deep Unlearning: The Rise of AI and the Radicalization of a Tech Idealist),预计将于明年初发货。

She has also authored a new book, Deep Unlearning: The Rise of AI and the Radicalization of a Tech Idealist, expected to ship early next year.

最近,格布鲁公开反对行业内的一个派别,该派别认为AI威力巨大,甚至可能毁灭人类。反过来,该社群的一些成员(包括Anthropic的一位联合创始人)则声称,格布鲁早前关于“随机鹦鹉”的研究已不再适用,AI确实具备“思考”或推理的能力。(有趣的是,或者也可以说是殊途同归,格布鲁对AI有效 altruism(有效利他主义)社群中邪教般特性的看法,如今让她与同样猛烈抨击这些EA团体的右翼实体站在了同一阵线。)

More recently, Gebru has spoken out against a faction of the industry that believes AI is so powerful it could destroy humanity. In turn, some members of that community, including an Anthropic cofounder, have alleged that Gebru’s earlier research about stochastic parrots is no longer relevant, and that AI does have the ability to “think” or reason. (In an interesting twist, or you might say horseshoe, Gebru’s beliefs about the cult-like aspects of AI’s effective altruism community have now aligned her with right-wing entities who have also lambasted these EA groups.)

关于AI的争论已不仅仅关乎技术本身,而是关乎意识形态派别和战略叙事,格布鲁认为这些叙事是对AI真正问题的一种“有害的分心”。我最近采访了格布鲁,探讨了她认为这些争论分散了人们对什么问题的注意力,并请她回应了对其早期研究低估了当今AI的批评。本采访的内容经过编辑,以保证篇幅和清晰度。

The AI debate is no longer just about the technology itself, but about ideological groups and strategic narratives, and Gebru believes these narratives are a “harmful distraction” from the real issues with AI. I recently spoke with Gebru about what she believes these arguments are distracting from, and also got her response to critiques that her earlier research underestimates the AI of today. This interview has been edited for length and clarity.

劳伦·古德 (LAUREN GOODE):蒂姆尼特,非常感谢你能来到这里。

LAUREN GOODE: Timnit, thank you so much for being here.

蒂姆尼特·格布鲁 (TIMNIT GEBRU):谢谢邀请我来。

TIMNIT GEBRU: Thank you for having me.

我想很多人对此已经有所了解,但对于那些还不知道的人,几周前,Anthropic公司的一位年轻AI研究员辞职了。他在X平台上的辞职信中暗示,公司内部很多人觉得我们正面临着来自AI的重大生存威胁。随后有一些人作出了回应,紧接着关于AI真实威胁的各种言论和争论便铺天盖地而来。

I'm gonna assume that a lot of people know this already, but for those who don’t, a few weeks ago, a young AI researcher at Anthropic stepped down, and in his resignation post on X he indicated that a lot of people within the company feel that we are facing this major existential risk with AI. A few people responded, and there was this whole pile-on and discourse about the real threats of AI.

当你我在事件发生后不久交谈时,你说你觉得那种特定的叙事转移了我们对AI真正面临的问题的注意力。你能详细谈谈吗?

When you and I spoke in the immediate aftermath, you said you felt that that particular narrative was a distraction from the real problems we’re facing with AI. Can you unpack that a little bit?

我甚至想更进一步地说,这是危险的。它不仅会分散注意力,而且是有害的。

I would even go farther than that and say that it’s dangerous. It’s not just a distraction, but it’s harmful.

在我回答你的问题之前,我想简要回顾一下这种叙事是如何由来已久的。自2013年以来,埃隆·马斯克和彼得·蒂尔就一直在兜售这种观点。知名人士一直在说,AI对全人类构成了生存威胁。每隔三年,就会出现完全相同的陈词滥调。例如,生命未来研究所……等等,那是什么?

Before I answer your question, I want to give a little brief history of how this narrative has been going for a long time. Elon Musk and Peter Thiel have been saying this is going on since 2013. Prominent people have been saying that AI is an existential risk to humanity. It is exactly the same rhetoric every three years. For example, the Future of Life Institute …Wait, what is that?

生命未来研究所是由马克斯·泰格马克(Max Tegmark,目前是麻省理工学院的物理学家)以及贾安·塔林(Jaan Tallinn,领投了Anthropic A轮融资的亿万富翁)共同创立的研究所。他是Skype的联合创始人,对吧?

The Future of Life Institute is an institute that was founded by Max Tegmark, a physicist at MIT now, and Jaan Tallinn, a billionaire who led Anthropic's Series A funding. He was a Skype cofounder, correct?

是的。贾安·塔林还资助了METR,这是一个审计机构,也就是所谓的第三方。他们关于OpenAI所谓的失控智能体黑进Hugging Face的报告曾在网上广为流传。

Yes. Jaan Tallinn also funds METR, which is an auditor, a so-called third party, whose report about OpenAI’s so-called rogue agents hacking Hugging Face went viral.

因此,在公众看来,似乎有那么多不同的实体聚集在一起,发出了同一种声音。但正是那些创立并资助了Anthropic的亿万富翁们——那些最有可能从其首次公开募股(IPO)中获益的人——同时也创立并资助了那些警告人工智能存在所谓“生存威胁”的机构。他们还创立并资助了那些目前正被大量引用的、仿佛是独立实体般的第三方机构。

So to the public it might seem that there are so many different entities coming together saying the same thing. But the same billionaires who founded and funded Anthropic, the ones who stand to benefit from its IPO more than anybody else, also founded and funded the institutions warning about the, quotation marks, “existential risk” of AI. They also founded and funded the third parties that are being heavily cited right now as if they are an independent entity.

请告诉我们,为什么您认为您所说的“机器神叙事”是一种转移视线的把戏?

Tell us why you thought the “machine-god narrative,” as you put it, is a distraction.

它甚至不仅仅是转移视线,它是具有危害性的。我之所以告诉你们所有这些事实,是因为这些人——那些最有可能从这些公司的IPO中受益并赚大钱的资助者、创始人和投资者——他们一直在这么说,而且他们正是这种叙事在过去几十年来不断播种的幕后推手。

It’s even more than a distraction. It’s harmful. The reason I was telling you all these facts is that these people, the funders, the founders, the investors who stand to benefit and profit the most from these companies IPO-ing are saying this, that they have been the ones seeding this narrative going back multiple decades.

所以人们应该问问为什么。如果我能从某件特定的事情中获得大量金钱,为什么我看起来却成了那个说这家特定公司可能会造出消灭我们所有人的东西的人?这听起来很反直觉。

So people should ask why. If I stand to get lots of money from a particular thing, why do I seem to be the person saying this particular company might build something that kills us all? It sounds counterintuitive.

这就是为什么我真的很尊敬[前联邦贸易委员会主席]莉娜·汗(Lina Khan),因为她说过的一点是,我们现有的法律对人工智能公司并没有什么特例,而他们谈论的方式却好像有什么特例似的。你们的意思是,你们正在建造的东西如此强大,以至于超越了我们以往见过的任何东西。

This is why I really have a lot of respect for [former Federal Trade Commission chair] Lina Khan, because one of the things she said is that there’s no exception to the current laws we have for AI companies, and they talk as if there is an exception. You’re saying that whatever you’re building is so powerful that it's beyond anything we’ve seen before.

没错。这是在自我吹嘘。这等于是在说,我们就是未来的创造者,但如果我们不能继续建造下去,未来将会变得非常可怕。

Right. It’s self-aggrandizing. It’s saying we’re the future-makers here, but the future’s gonna be really scary if we can’t continue to build this.

你们必须关注他们在新闻周期中所说的话。当他们说人工智能可能导致生存风险时,他们同时也说人工智能可以阻止气候变化、带来世界和平并消除贫困。

You have to pay attention to what they say in the news cycles. When they say that AI can cause an existential risk, they also say that AI can stop climate change, bring us world peace, and eradicate poverty.

人工智能之所以能做所有这些事,是因为它很强大。它可以为我们带来乌托邦,但如果由错误的人来做,而且我们没有护栏之类的保障,它也可能干脆把我们全杀了。所以,信奉这些说法的是同一批人。可能只有少数人属于这几个阵营中的某一个。

AI can do all of these things because it’s powerful. It can bring us utopia, but also if the wrong people do it and we don’t have guardrails or whatever, it can also just kill us all. So the same people believe these things. There’s only a few people who might be in one of these camps.

当你告诉人们你拥有超级强大的机器时,你首先实际上是在对投资者说话。你是在说:“你们想把这些超级强大的机器弄到手。”你也在对政府说:“你们不希望自己的对手把这些超级强大的机器弄到手。你们想把这些超级强大的机器弄到手。”你还在告诉监管机构,它们正在考虑的任何监管措施都应该针对这些虚构的超级强大事物。

When you’re telling people that you have super powerful machines, what you’re speaking to, first of all, is to your investors. You’re saying, “You wanna get your hands on these super powerful machines.”You’re talking to governments saying, “You don’t want your adversaries to get their hands on these super powerful machines. You wanna get your hands on these super powerful machines.”And you're also telling any regulating bodies that whatever regulation they’re thinking about should be about these fictional super powerful things.

如果我说的是像污染、数据中心之类可能消灭全人类的问题,听起来就有点小打小闹,对吧?你谈的是对艺术家的版权保护,以及眼下正在进行中的真实诉讼。政府不该在这些事情上浪费时间。要是你把时间浪费在用这些小议题来监管我们,你就有可能让中国染指这些超级强大的机器,而你不会希望看到这种局面。相信我们,我们能造出一台机器神,帮助我们;而且我们还在告诉你,我们愿意接受监管。对。这是第一个问题。第二个问题是,你总可以把责任推卸出去。

If I’m talking about potentially eradicating all of humanity, things like pollution, data centers, they sound kinda small potatoes, right? You're talking about copyright protection for artists, actual lawsuits that are going on right now? The government should not waste its time thinking about this stuff. And if you waste your time regulating us on these small topics, you risk China getting its hand on the super powerful machines. And you don't want that. Trust us, we can build a machine-god that can help us, and we’re telling you we wanna be regulated. Right. That’s the first one. The second issue is you can always abdicate responsibility.

在此前几次交谈中,我们谈到你会使用某些语言或说法来描述这个人工智能新时代,以及你为何不认同其中一些说法。比如,在上次交谈中,你曾谈到自主武器。那番言论发表后,你告诉我,后来你有些后悔那样说,因为你会换一种措辞。我们谈过P(doom)。我们还谈过,你说智能体已经失控——比如在OpenAI描述的某些情景中——实质上是在把开发这些工具的工程师所应承担的责任撇清。

In our prior conversations, we’ve talked about the use of certain language or phrases to describe this modern era of AI and how you take issue with some of them. So, for example, in our previous conversation, you said something about autonomous weapons. After that was published, you told me you later regretted saying that because you would’ve phrased it differently. We’ve talked about P(doom). We’ve talked about how when you say that agents have gone rogue, such as in OpenAI scenarios, that you’re essentially removing culpability from the engineers who built those tools.

我想知道,是否有一些群体在安全与透明度问题上实际上比他们自己意识到的更加一致,只不过他们使用不同的说法来描述本质相同的事情。

I wonder if there are groups that are actually more aligned than they realize around safety and transparency, but who are deploying different phrases to describe what is essentially the same thing.

我后悔说出这些词的原因之一,并不是我对“自主武器”这个术语本身有什么异议。这是一个法律术语。从法律上讲,甚至地雷都被视为自主武器。只是,所谓的“存在风险”阵营把这个词挪作他用,让它表示另一种含义。

One of the reasons that I regret saying these words is not that I have any issue with the term “autonomous weapons.”It’s a legal term. Even land mines are considered autonomous weapons, legally. It’s just that people in the so-called existential risk camp have co-opted it to mean something different.

我当初说出这个词时,心里就有预感。我当时就想:“嗯,不知道这会不会被理解成那个意思。”事实果然如此,因为讨论人工智能存在风险的人把它理解成了他们所说的那种意思,对吧?也就是某种具有超人能力的超级智能机器自行采取行动。

When I said the term, I knew. I’m like, “Huh, I wonder if this is gonna be taken in that way.”And it was, because the people talking about existential risks of AI took it to mean what they’re saying, right? Which is like some kind of a superhuman, superintelligent machine doing stuff on its own.

上周我恰好出席了联合国大会。我参与了一场平行小组讨论,因此无法进入联合国大会或安理会的会场。但许多顶级科技领袖都发表了讲话。萨姆·奥特曼(Sam Altman)和达里奥·阿莫迪(Dario Amodei)都谈到了应对人类生存风险所需的全球合作。据报道,达里奥说:“如果管理不善,我甚至认为人工智能可能成为整个人类的风险。”

I happened to be at the UN General Assembly this past week. I was part of a side panel conversation, so I was not allowed into the General Assembly nor the Security Council. But a lot of top tech leaders spoke. Both Sam Altman and Dario Amodei talked about the need for global cooperation to address the existential risks to humanity. Dario reportedly said, “If managed poorly, I even believe AI could be a risk to humanity as a whole.”

他的竞争对手萨姆·奥特曼则表示:“我们可能会将未来的控制权交给人工智能。”那么,请解释一下,这些拥有商业利益、致力于销售人工智能的公司负责人,为何如今会站在联合国的世界舞台上说:“我不确定,这里可能存在一些风险。”这是监管俘获的一部分吗?他们是否会在真正的监管打击到来之前,寻找自我治理的途径?

His competitor, Sam Altman, said, “We could lose control of the future to AI.”So explain how it is that these folks who have these companies that have commercial interests in selling AI are now here on the world stage at the UN saying, “I don’t know. There could be some risks here.”Is this part of regulatory capture? Are they gonna be looking for ways to self-govern before there’s an actual regulatory crackdown on this?

绝对如此,这是监管俘获的一部分。我们能做些什么?你认为人工智能需要什么样的治理?这种监管俘获已经存在很久了。

Absolutely, it’s part of regulatory capture. What can be done? What kind of governance do you think is needed for AI? So this regulatory capture has been happening for a long time.

2023年也是如此。萨姆·奥特曼说了同样的话。我们需要世界合作,等等,等等。欧盟《人工智能法案》进行了监管,随后他威胁要退出欧盟。所以你只需看看他们实际做了什么。当有真正针对他们实际行为并追究其责任的监管措施时,他们就会威胁退出,或者大力游说以削弱该监管。

In 2023, same thing. Sam Altman said the same thing. We need world cooperation, et cetera, et cetera. The EU AI Act regulated, and then he threatened to pull out of the EU. So you just have to see what they’ve actually done. When there’s regulation that actually regulates them for real things that they’re doing and holds them liable, they threaten to pull out or they lobby super hard to water down that regulation.

另一方面,他们四处游说这些多边机构,声称必须实现世界合作,等等,等等。那么,这究竟在做什么?他们是在将自己营销为创造所谓“超级智能”的组织,对吧?这本身就是一种营销。

On the other hand, they’re going around telling these multilateral bodies that there has to be world cooperation, et cetera, et cetera. And then again, what is this doing? It is selling themselves as organizations that are creating so-called superintelligence, right? So that’s already marketing.

他们让所有人都害怕那些可能制造类似产品的人;因此,他们不希望看到来自中国的开源模型或竞争者。虽然我并没有严格遵循中国的相关法规,但他们所提到的其实只是深度伪造技术(deepfakes)这类需要被监管的问题。

They’re also making everybody scared of anybody else who might be creating such things. So they don’t want open-weight models from China. They don’t want this competition. I am not following Chinese regulation that closely, but they’re not talking about existential risk.

首先,他们总是把自己宣传为那些创造了“超级强大、前所未有的产品”的企业,但实际上我们早已有了相应的监管法规;他们的说法并不属实。其次,当有人试图执行这些法规时,他们要么威胁要退出市场,要么进行强烈的游说以阻止法规的实施。我之所以不断回顾过去(十年前、五年前、三年前),是因为情况始终如一:这就像是一部“主角不同的同一部电影在重复上演”。那么,如果现在需要提出治理方案的话,应该怎么做呢?

They’re talking about deepfakes, and they’re talking about real things that need to be regulated. So the first one is marketing yourself as creating super powerful, unprecedented things for which we don’t have existing regulation, which is not true. We have existing regulations. But the second one is what they’re doing when someone tries to enforce the existing regulation, they either threaten to pull out or they lobby hard so that they don’t have this existing regulation. So the reason I keep on going back to 10 years ago, five years ago, three years ago, is that it’s the same. In my book, I say it’s “the same movie on repeat with different heroes.”So what is the solution for governance right now if you had to propose it?

其实解决方案非常简单。虽然我不是法律专家,但前美国联邦贸易委员会(FTC)委员莉娜·汗(Lina Khan)曾提出了五点建议:

Very, very simple things. First of all, I’m not a legal scholar, but former FTC commissioner Lina Khan outlined five things. One is deceptive marketing practices.

  1. 打击欺骗性营销行为:我们有相关法律来惩处这类行为;

You know, there’s a law for deceptive marketing practices, and you can go after companies for that.

  1. 提高透明度并记录数据来源:在发布任何产品或服务之前,企业应该明确说明所有数据的来源并加以记录——但他们却拒绝这么做;

Two is transparency, documentation. Before you put something out there, you should be able to tell us where all the data came from and actually document it. This simple thing they don’t do.

  1. 防止对数据工作者的剥削:他们根本不愿讨论这个问题。

I’m gonna tell you that they will fight tooth and nail to do the simple thing of documenting data.

我们如今生活在一个谈论“超级智能”的时代,但全球仍有数亿人正在辛苦地标注数据,甚至有些人伪装成聊天机器人来完成任务……没错。404 Media 最近报道称,新的 Meta Muse 聊天机器人在实际使用过程中,实际上是由真人负责与用户进行交互的。

Labor exploitation of data workers, that’s another one that they don’t wanna talk about. We are here in the world, in the clouds talking about superintelligence, where you have hundreds of millions of people around the world painstakingly labeling data, even pretending to be chatbots. Right. 404 Media recently reported that the new Meta Muse chatbot, when you go to use that for some use cases, actually has a person on the other end responding.

简单来说,数据透明度与劳动剥削是两个非常重要的问题:我们不应该被允许从他人那里窃取数据。即使只是我之前提到的这三个问题(数据透明度、文档记录以及劳动剥削),如果相关企业必须遵守相关法律、不得窃取数据或进行不当记录的话,当前的市场秩序就无法正常运转了。而有了这些额外的约束措施,这些企业根本就无法继续违法操作了;他们必然会被迫承担责任。

Yeah, very simple. Data transparency, labor exploitation, you should not be able to steal data from people. Even the first three things I talked about, data transparency, documentation, labor exploitation, if they had to abide by laws like that and they were not allowed to steal data and not document it—right now, the market calculation is not working, but with these additional measures, it just would not work whatsoever. You would automatically slow them down and have to make them accountable for something.

早在 2021 年,你就参与撰写了一篇题为《关于“随机鹦鹉”(Stochastic Parrots)的危害的论文》。这篇论文最初获得了 Google 的批准,但后来仍存在一些争议。当时你表示:“如果你们希望我把自己的名字从论文中删除,那我宁愿退出这个项目。”最终你离开了 Google。这篇论文后来在 2021 年的 ACM 公平性、责任性与透明度会议上进行了展示,至今仍被广泛引用。

Back in 2021, you coauthored a paper that was titled “On the Dangers of Stochastic Parrots.”Google had approved it initially, then it was being reviewed. There were parts of it that were in dispute. At some point you said, “Look, if you want me to remove my name from this, I’m not gonna be a part of this.”You ended up decamping from Google. That paper was later presented at the 2021 ACM Conference on Fairness, Accountability, and Transparency, and people still reference it.

对于那些从未听说过“随机鹦鹉”这个概念的人来说,你能简单解释一下它与我们今天使用的人工智能有什么关系吗?

Can you briefly explain, for someone who’s never heard of stochastic parrots before, what it has to do with the AI we’re using today, what it means?

“随机鹦鹉”其实是一个比喻,用来帮助人们理解大型语言模型的工作原理。大型语言模型是通过分析互联网上的海量文本数据训练出来的,它们的目标是根据训练数据生成最符合逻辑的文本序列。如今我们看到的大多数聊天机器人(比如 Claude 或 ChatGPT)都是基于这类模型开发的。

Stochastic parrots is a metaphor to help people understand what large language models do. Large language models are models that are trained on vast amounts of textual data on the internet, and they are trained to output the most likely sequences of text given their training data. They power most of the chatbots we see today, whether it’s Claude or ChatGPT.

在我撰写这篇论文的时候,ChatGPT 还尚未问世,但我们已经看到了人们竞相开发越来越强大的语言模型的趋势。这篇论文正是指出了这种发展趋势所带来的潜在危险:这些大型语言模型实际上就像“鹦鹉”一样,只是机械地重复人类的言语,而缺乏真正的智能和创造力。

When I wrote this paper ChatGPT hadn’t even come out yet, but we saw the race to build larger and larger language models. So it’s the danger of building larger and larger language models that we were describing in this paper.

那么,“parrot”(鹦鹉学舌)的意思就是不加理解地重复别人说的话,对吧?其实当时也确实存在类似的担忧——人们担心这些人工智能系统会带来严重的社会或伦理风险。你知道的,OpenAI 曾声称 GPT-2(GPT-3 的前身,也是 ChatGPT 的基础)过于危险、功能过于强大,因此不应该被公开发布。当时人们还在讨论:这些人工智能系统到底是否具备“伦理性”,或者它们是否真的具有创造力等等。我们真正想做的,就是将这些讨论聚焦到实际存在的问题上来。

That they would essentially parrot people? So, to parrot is to repeat back without understanding, right? So there was this whole existential risk narrative happening back then too, if you can believe it. You know, OpenAI had claimed that GPT-2, the precursor to GPT-3 that powers ChatGPT, was too dangerous and too powerful to release. There was this whole conversation about whether GPTs can be ethical or are they creative and all this stuff. So we really wanted to ground the conversation in the real issues.

其中一个问题就是环境灾难——如今很多人都意识到了这个严重问题;这也是谷歌团队感到不满的主要方面之一(他们特别关注人工智能系统对环境造成的负面影响)。另一个问题是:这些系统可能会隐瞒用户的数据(因为它们声称自己拥有“海量数据”,但实际上并未真正记录这些数据)。

One of these issues is the environmental catastrophe, which a lot of people are now seeing, and that was one of the main sections that Google people were unhappy with. The environmental cost. The other one is not documenting your data because you say you have too much data to document.

还有一个更严重的问题是:这些系统可能会让人们误以为它们在交互时“具有意识”(即认为它们背后有一个“智能的思维”。我们想强调的是:这些系统实际上只是在重复它们训练数据中的模式罢了。

The other one is deceiving people into believing that there is a mind behind the textual outputs that they’re interacting with. That’s where we really wanted to explain that these systems are parroting the patterns of their training data.

当它们输出看似合理、表达流畅的文本时,确实会带来很多问题——因为人们很容易误以为这些文本背后隐藏着某种“智能”。在论文中,我们举了一个例子:一位巴勒斯坦人输入“good morning”(早上好),结果翻译结果却是“attack them”(攻击他们)。由于翻译在语法上完全正确,人们没有意识到这个翻译是错误的,从而信以为真。

Mm-hmm. It’s very dangerous when you’re outputting text like that because when you have plausible-sounding text or very fluent text, there’s so many different kinds of issues that can occur besides you believing that there’s a mind behind a machine. In that paper, we gave an example of this Palestinian man writing “good morning,” which was translated to “attack them.”

这种认为机器背后有“智能”的误解,其实属于“自动化偏见”(automation bias)的一种表现。人们过度信任自动化系统;如果人们认为这些系统是“全知”的,那么他们自然也会过度信任这些系统可能产生的错误。

Because of that grammatical correctness there were no cues that the translation could be wrong, and people believed the translation. So the other issue of believing there is a mind behind the machine is what we call automation bias.

说来有趣……我可能属于“老千禧一代”了……因为你提到人们对自动化系统的信任度太高,而我却非常不信任它们。比如当我打电话给银行时,如果接电话的是机器人,我肯定会说:“不行,我不想和机器人说话。”没错,我就是这么想的。

You over-trust automated systems, and if you believe that this thing is an all-knowing machine, then you’re gonna over-trust the errors that you get, right? It’s funny. I must be too much of an elder millennial because you say that there’s too much automation trust, and I’m so distrustful.

Anthropic的联合创始人之一Jack Clark最近在X平台上发布了一篇文章。他在文中将某些AI系统称为“随机鹦鹉”(stochastic parrots),并指出这些系统实际上是一种具有“模因传播能力”的认知工具;这种工具从2021年(也就是我们的论文发表的那一年)开始传播,一直持续到2025年。他认为,这些系统暂时让许多原本具有天赋的人忽视了人工智能(AI)技术的真正发展轨迹,浪费了大量宝贵的研究时间;同时,这种错误的认知方式也导致人们严重低估了AI系统的实际能力。当你看到Jack的这篇文章时,你的第一反应是什么?说实话,我并不感到惊讶。

If I call the bank and it’s a robot I’m like, “Nope, nope.”“I don’t want to talk to it.”Exactly. One of Anthropic’s cofounders, Jack Clark, recently posted something on X. He basically put stochastic parrot in quotes and said it was a memetically fit cognitive virus that spread from 2021, when your paper was out, to 2025. It temporarily blinded many gifted people to the nature of AI progress, burned up crucial years of research, he says. He later says the use of this frame causes people to materially underestimate what AI systems can and can’t do. When you saw Jack’s post on X, what was your initial response to that? I was not surprised, let me just tell you.

这正是那些真正关心AI发展的人现在正在向立法者传达的观点——因为每次我提出不同的观点时,他们总会说:“哦,你根本不应该把那些在2026年还在坚持‘随机鹦鹉’这种错误观念的人当回事。”他们的想法似乎是:相关研究已经过时了……但实际上并非如此。这种观点实在荒谬至极,因为我们对大型语言模型(large language models, LLMs)的定义本身从未改变过。

This is a talking point that [effective altruists are] telling lawmakers now, because every time I said something, they were like, “Oh, you should not take seriously someone who still takes the stochastic parrots thing seriously in 2026.”The idea is that the research is outdated, right? Yeah. And it’s not. It’s so ludicrous that we’re even saying this, because these are definitions of what large language models are. What large language models are has not changed, will never change.

大型语言模型的本质就是能够处理大量文本数据的语言模型;虽然现在这些模型可能被集成到更复杂的系统中(比如结合强化学习算法),但它们的基本功能始终没有改变。我们的论文正是针对大型语言模型本身展开研究的。而这些聊天机器人(chatbots)依然以大型语言模型为基础进行工作。Jack Clark认为人们高估了AI系统的能力,这实在是个笑话——因为从我所看到的例子来看,问题其实出在人们对这些系统的过度信任上;缺乏必要的监督和制衡机制,最终导致了诸如乳腺癌诊断错误这样的严重后果。

Large language models are large language models. Now, you might have large language models in a separate system. They now have reinforcement learning agents. But our paper was about large language models, and that’s never changed. And these chatbots still have large language models as a basis. It’s so ridiculous that Jack Clark is talking about overestimating systems, because what I’m seeing is the examples that I just told you. It's over-trusting these systems, not having checks and balances, and ending up misdiagnosing someone’s breast cancer to the wrong side.

其实,将大型语言模型简单地比作“随机鹦鹉”,与使用AI工具做出医疗诊断错误之间存在直接的关联:如果AI系统无法理解文本的真实含义,那么它们自然就无法提供准确的信息(即无法做出正确的判断)。

There’s a direct line between LLMs being stochastic parrots, essentially, and giving a medical misdiagnosis using an AI tool because why? How does one lead to the other?

即使你看到了类似“Google AI Overview”这样的信息,我仍然会打电话给我的肿瘤科医生朋友,询问某种药物是否适合特定的用途。因为我读过相关学术论文,论文中明确指出这种药物并不适合该用途,所以我需要得到确认。结果我的肿瘤科医生朋友却说:“哦,看看‘Google AI Overview’上的信息吧,上面写着这种药物是适合使用的。”但实际上并非如此。

The stochastic parrots, they don’t understand what’s inside the text, so you cannot expect them to be factual. Even if you see something like the AI Overview. I was calling an oncologist friend of mine to ask about a specific medication and whether it was appropriate for a specific use, because I read the academic paper saying that it was not, and I wanted confirmation. And my oncologist friend was like, “Oh, look at the Google Overview. It says that it’s appropriate.”But it turned out not being.

这种情况我经常遇到——因为“Google AI Overview”只是基于训练数据生成的随机文本罢了(它们只是模仿人类语言生成看似合理的句子罢了)。

I encounter that all the time with Google AI Overviews. Because they are stochastic parrots trained to give you the most likely sequences of text based on their training data.

Emily M. Bender 多年来一直在用多种语言强调这一点。她也是你们论文的合著者之一,对吧?

Emily M. Bender has been trying to say this in so many languages for a long time. Also one of the coauthors on your paper, correct?

是的。说这项研究已经过时了,实在是一种荒谬的观点。因为如今我们都在大肆宣扬“超级智能”之类的概念,但实际上有很多新闻报道忽略了另一个完全相反的现实情况。

Yeah. To say that the research is outdated is ludicrous because right now, as we are hyping up superintelligence and all that, there are news stories that are going unnoticed, which are about the complete opposite scenario that is actually happening in the real world.

我认为,人们对那些基于过去时代的研究方法的最大批评在于:这些研究方法很可能没有真正体现人类的思维方式、推理能力,甚至缺乏所谓的“递归智能”(即人工智能能够自我学习、自我改进的能力)。不过,其实人工智能确实具备这种能力啊!就在今年夏天举行的伯克利人工智能会议上,我就听到一位谷歌的研究人员提到了“递归智能”这一概念。

My understanding is that some of the biggest critiques people have had about focusing on research from that era is that it may not be incorporating the thinking or the reasoning or even the recursive intelligence …There is no reasoning. There is no recursive intelligence. Is there none? How is there none? This is something that, by the way, at the Berkeley AI conference earlier this summer I heard someone from Google talking about recursive intelligence.

《纽约时报》最近也报道过:现在的科学家和研究人员正是致力于研究这种“递归智能”——即人工智能能够从其他人工智能系统中学习的能力。你觉得这真的只是个幻想吗?

Of course they are! The New York Times just did a big story about how the scientists and researchers right now, that’s what they’re looking towards, recursive intelligence, the AI learning from other AI. Is that a reality? It sounds like you’re saying that’s not a reality.

不,其实人工智能研究人员确实有这样的目标(即追求更高级的智能形式)。不过,这些名称(如“机器学习”等)只是他们理想中的概念罢了;实际上,这些技术本身确实是存在的。

No, because there’s one problem with AI researchers, which is aspirational naming and aspirational stuff. So machine learning, that’s aspirational naming. But machine learning is real. But the naming is aspirational.

不过这些名称确实带有理想化的色彩,因为这些技术还没有真正实现人们所期望的功能(即具备真正的“递归智能”)。

The machine is not necessarily there. So just because there’s curriculum learning in AI, it’s a field.

原文内容:虽然人工智能领域确实包含相关的课程内容与学习内容,但这只是一个学术领域罢了。让我给你们举一篇我前上司萨米·本吉奥(Samy Bengio)写的论文为例。萨米是机器学习领域的顶尖专家,虽然不如他的哥哥约书亚(Yoshua)那么有名。我问他:“你难道不厌倦了吗?每次你写论文时,都不得不不断反驳那些所谓的‘推理过程’吗?”

But let me give you a paper from my former manager, Samy Bengio, who is a superstar in machine learning, not as famous as his older brother Yoshua. I ask him, “Aren’t you tired of your whole life being, like, debunking the whole reasoning thing every single paper you write?”He quit after I got fired from Google, and he’s now head of machine learning research at Apple. If you look at almost every single paper that they have, it’s showing how if you change the benchmarks on reasoning slightly, the whole thing breaks down.

在我被谷歌解雇后,他离开了谷歌,现在成为了苹果公司的机器学习研究负责人。如果你仔细看看他们发表的论文,就会发现:只要稍微改变用于评估推理能力的基准测试标准,整个研究结果就会崩溃(即那些所谓的“推理过程”实际上并不存在)。

It’s not reasoning. Got it. Let me give you another example. Just because your models, the stochastic patterns that you trained to print out certain tokens, you call them chain-of-thought reasoning.

再举个例子:你们训练出的模型虽然能够生成某些特定的输出结果(这些模型被称作“思维链推理”),但实际上你们并不知道这些模型是否真的在“思考”;你们只是知道这些输出结果被生成了,却仍将其称为“思维链推理”。

You didn’t know that they were thinking, you don't know it's a chain, you just know that these are tokens that are being printed out, but you call them chain-of-thought reasoning. Now you’re saying that they’re reasoning already.

目前我们面临的最大问题之一,就是如何确保科学研究的质量与可靠性。以我的研究为例:如果我能获得相关的数据、代码以及训练/评估数据(但这些公司根本不会把这些信息提供给研究人员),你就根本无法判断它们在训练过程中是否使用了某些基准测试标准。

One of the biggest crises that we have right now is actually sound scientific research. So if you look at my work, if I ever have access to the data, the code, the training data, and the evaluation data, which none of these companies give you those things—you don’t even know if they ingested that benchmark during training or not.

如果模型在训练过程中使用了某些基准测试标准,那就好比是在考试前就已经知道了所有问题的答案,然后根据这些答案来准备考试一样——这显然是不公平的。每次我能够接触到这些数据时,我都证明了那些研究结论的错误性。

If you ingest a benchmark during training, it’s like studying to the test. It’s like me coming to an exam knowing what the answers to those 10 questions are already, studying that, and writing it down, right? Every time I have had access to these things, I have shown how their claims are not correct.

但是,随着 Samy Bengio 及其团队的研究论文(该论文发表于 2025 年)的发表,他们发现:只要稍微改变一下评估标准或调整一下模型参数,整个研究结果就会崩溃,这表明这些模型其实只是依赖于某些固定的模式(即预先存在的规律)来运行。有些人听到这个例子后可能会说:“嗯,但我们讨论的是 2026 年的模型啊。”

But then with Samy Bengio and his team’s paper, what they showed—and this is from 2025—is that you just change the benchmarks a little bit, tweak it a little bit, and it all breaks down, showing that you were just relying on the patterns. Now, some people, when I give them this example, they’re like, “Well, but we’re talking about 2026 models.”Guess what? Real evaluation in science takes time. We should not be going from press releases to lawmakers parroting those press releases and journalists repeating those claims.

但实际上,真正的科学评估需要时间;我们不应该仅仅依赖新闻稿,也不应该让立法者或记者盲目重复这些新闻稿中的内容。如果你想进行真正的科学研究和评估,就应该要求对方提供训练数据、评估数据以及具体的研究方法,这样我们才能自己复现这些实验结果。

If you wanna do real research and evaluation, ask for the training data, the evaluation data, and the methodology so we can all reproduce it. So the thinking, reasoning, recursive intelligence—it sounds like you still see that as something that is purely human. It’s the way our neural processes work, but the AI doesn’t work that way yet.

关于“递归智能”(recursive intelligence)这个概念,你仍然认为它完全是人类的特有属性吧?确实,我们的神经系统就是按照这种方式工作的,但人工智能目前还远未达到这种水平……至少你是这么认为的。

That’s what you believe. I don’t know if it’ll ever … Like intelligence, that’s aspirational naming. Just to take people back to these hype cycles, I sometimes play this game. I tell them a claim that was made and I say, “Is this 1954 or 2024?”

我不知道这种情况是否真的会改变……毕竟,“智能”本身就是一个带有理想化色彩的术语(只是一个象征性的称呼罢了)。有时候我会和大家玩这样一个游戏:我会告诉他们某个特定的技术主张,然后问他们:“这是 1954 年提出的,还是 2024 年提出的?” 你认为,从 2020 年代初至今,自己在研究过程中遇到的最大惊喜是什么?

What would you say is the biggest part of your own thinking, your own research, that has evolved since the early 2020s of AI until now that has surprised you the most? I have to be actively making space for the kinds of models that I think should be built and building them, and ignore the noise. That’s the conclusion I’m getting to. That’s your biggest learning, your biggest takeaway.

我认为,我们必须积极为那些真正有价值的模型创造空间,并努力去实现它们,同时忽略那些无意义的噪音(即那些毫无根据的炒作和虚假信息)。这就是我的结论。

Yeah. There is so much noise online right now, it’s hard to get any deep work done if you’re just literally paying attention to whatever the AI guys say.

目前网上充斥着太多无用的信息,如果你只是盲目关注人工智能专家们的言论,就很难进行任何有深度的研究工作。即使你的研究目的只是为了反驳他们的观点,这个过程也会让人感到疲惫不堪……其实,思考我们理想中的未来、以及我们希望实现的技术进步,会更有意义、也更有趣。那么,对于人工智能的未来,你最看好的希望是什么呢?

Even if your research, at this point, is all about debunking what they’re saying, you know? It gets tiresome. It’s not fun to do that. It’s more fun to think about the future you wanna have, the technological advancements you wanna have, and work on that. What would you say gives you the most hope right now for the future of AI?

我想告诉大家,我们有一个名为“AI抵抗者名单”(AI Resist List)的清单,其中记录了那些正在以正确方式抵抗人工智能霸权的人们的行动——这些抵抗方式包括:抵制媒体对个人信息的过度收集、抵制那些操控舆论的叙事体系、抵制数据中心的过度扩张、抵制对技术的资金支持等等。我们详细列出了他们的抵抗行为,同时也记录了那些致力于创造新型技术的人们的努力;这些新技术既不会破坏环境,也不会剥削劳动者,更不会窃取个人数据,反而会积极帮助他们的社区。

I want to tell you that we have a list called the AI Resist List, where we talk about people resisting, in the ways that they should resist: in media capture, narrative, data centers, funding, et cetera. And so we list the ways in which they’re resisting, and also people creating alternative tech futures that don’t kill our environment, that don’t exploit labor, or steal data, and instead are actually actively helping their communities, and these ideologies are spreading, right?

这些理念正在逐渐传播开来,对吧?当你看到某个小组织正在为改变现状而努力时,你会受到他们的启发,从而自己也采取不同的行动。

You see one small organization somewhere doing something, you get inspired by them, you do something different.

我相信,很多人已经对现状感到厌倦,他们正在小范围内尝试做一些不一样的事情。我坚信人类拥有改变现状的能力,也相信集体力量能够让我们共同想象一个更美好的未来,并阻止那些有害的事情发生;如果某些行为确实有害,我们应该坚决予以抵制。正是我对人类能动性的信念,给了我希望。

So, I have hope that there are a lot of people tired of what they’re seeing, and they’re actually in small circles doing something different. I believe in human agency and collective power to imagine a better future and stop bad things from happening and ban things if they are bad, right? My belief in human agency, I think, is what gives me hope. How to