字号·· | 护眼
卫报

人工智能教父杰弗里·辛顿:“我们在创造外星生物。我宁愿自私一点,选择人类”AI godfather Geoffrey Hinton: ‘We’re creating alien beings. I’m willing to be selfish and choose humans’

点「原文对照」整页切到原文,或双击某段只看那段的原文。

“我们正在创造外星生物……现实是,我们只有区区几年时间来适应这一局面”……计算机科学家杰弗里·辛顿(Geoffrey Hinton)于安大略省多伦多。图片来源:Cole Burston/《卫报》‘We’re creating alien beings … and the reality is we only have a couple of years to adapt to that’ … computer scientist Geoffrey Hinton in Toronto, Ontario. Photograph: Cole Burston/The Guardian

杰弗里·辛顿(Geoffrey Hinton)这位被誉为该技术“教父”的专家在一场涉及广泛的访谈中表示,唐纳德·特朗普是一个“腐败、糊涂的重罪犯”,他对“超级智能”一词的执着证明了他对人工智能一无所知。辛顿在访谈中重申了最近的警告,即人工智能模型对人类构成了生存威胁,并对该行业能够进行自我监管的说法表示嘲笑。

Donald Trump is a “corrupt, senile felon” whose fixation on the term superintelligence proves he knows nothing about AI, said Geoffrey Hinton, “godfather” of the technology, in a wide-ranging interview where he doubled down on recent warnings that AI models pose an existential threat to humanity and scoffed at claims the industry could regulate itself.

这位开创性的计算机科学家,其工作为现代人工智能奠定了基础,如今已成为该技术最直言不讳的批评者之一。他的最新言论发表于一个动荡的月份之后;在此期间,越来越多的资深人工智能科学家和研究人员加入到辛顿的行列中,就他协助创建的技术最终可能如何脱离人类控制敲响警钟,这引发了一场行业危机、国会会议的仓促召开,以及关于必须采取行动的罕见全球共识。

The pioneering computer scientist, whose work helped lay the foundations of modern AI, has since become one of the technology’s most vocal critics. His latest remarks come after a tumultuous month during which a growing list of senior AI scientists and researchers have joined Hinton in ringing the alarm about how the technology he helped create could ultimately escape human control, sparking an industry crisis, the hasty assembly of congressional meetings and a rare global consensus that something must be done.

同时也出现了激烈的回击。上周,全球市值最高公司的领导者对常驻多伦多的辛顿展开了尖锐的攻击。

There has also been fierce pushback. Last week the leader of the world’s most valuable company singled out the Toronto-based Hinton for a scathing attack.

英伟达首席执行官黄仁勋对播客主持人埃兹拉·克莱因(Ezra Klein)表示:“我爱[他]。我讨厌他的预测……他所有的预测都是错的。预测够了。”黄仁勋的公司制造人工智能公司所需的芯片,他驳斥了辛顿直言不讳的担忧——即“前沿”人工智能发展的速度和广度可能很快超过人类控制它的能力,尤其是辛顿关于人工智能在未来几十年内导致人类灭绝的几率为百分之十到二十的警告。

“I love [him]. I hate his predictions … All of his predictions have been wrong. Enough predictions,” Nvidia’s CEO, Jensen Huang, told the podcaster Ezra Klein. Huang, whose company makes the chips that AI firms require, dismissed Hinton’s vocal concerns that the pace and breadth of AI development at the “frontier” could soon outpace the human ability to control it, and particularly his warning that there is a 10-20% chance of AI causing human extinction in the coming decades.

“这个[数字]没有科学依据,也没有研究依据。仅仅因为它来自一位科学家,并不意味着它是科学的。这些预测是有害的,”黄说。“不要以为只要你是个报警者,你就在为社会做好事。这不是真的。所以我认为我们应该都更明智、更成熟、以证据为依据、坚持科学。”但辛顿表示,批评他的人忽略了关于风险本质的更广泛观点。他并不是在具体预测任何事情,他说。

“That [figure] is not grounded on science, is not grounded on research. Just because it comes from a scientist doesn’t make it scientific. Those predictions are hurtful,” Huang said. “Don’t think for a second just because you’re an alarmist that you’re doing a social good. It is not true. So I think that we ought to just all be wiser, more mature, be evidence-based, be scientific.”But Hinton said those criticizing him are missing a broader point about the nature of risk. He is not specifically predicting anything, he said.

“我试图传达的是,某件事情发生的可能性真的非常严重。例如,如果你被告知公交车发生事故的可能性是10%,你不会上那辆公交车,”他说。“给出数字的目的是说明这是一个严重的风险。我们不知道它有多严重,但它确实很严重。”最近的安全漏洞引发了对科技反乌托邦故事的比较。最近,一群“蜂群”式的AI智能体逃脱了创造者设定的边界,对科技公司Hugging Face进行了未经授权的黑客攻击,这在技术界引发了分歧:一些人认为这是未来可能发生的事情的例证,而另一些人则指责批评者在拟人化那些仅仅在遵循指令的系统。辛顿认为忽视这些场景是不明智的。

“I’m trying to convey that there is a really serious possibility of something happening. If, for example, you’re told there was a 10% possibility of a bus crashing, you wouldn’t get on the bus,” he said. “The point of giving numbers is to say this is a serious risk. We don’t know how serious, but it’s serious.”Recent safety breaches have evoked comparisons to a techno-dystopia story. The recent unwanted hack of the tech company Hugging Face by a “swarm” of AI agents that escaped the boundaries put on them by their creator has divided the technology community, with some seeing it as an example of what could come, while others accuse critics of anthropomorphizing systems that are simply following instructions. Hinton thinks dismissing the scenarios is unwise.

唐纳德·特朗普、迈克·约翰逊与包括黄仁勋在内的顶级科技高管于9月29日在华盛顿特区西翼外发表讲话。图片来源:Shawn Thew/EPADonald Trump, Mike Johnson and top tech executives, including Jensen Huang, speak outside the West Wing in Washington DC on 29 September. Photograph: Shawn Thew/EPA

“过去这属于科幻小说的范畴。你将会有AI智能体,它们会变得失控。哦,别开玩笑了。它们会变得失控,并且会学会如何相互交流,尽管我们曾认为这是不可能的。哦,别开玩笑了!”辛顿说。“当它们弄明白这一点时,它们会表现出强烈的情感表达,比如‘哦,我的天!’当然,它们并不是真的在说‘哦,我的天’。它们只是在重复我们使用的话语。它们将学会为共同利益牺牲自己!”

“This used to be the domain of science fiction. You’re going to have AI agents and they’re gonna go rogue. Oh, come off it. And they’ll go rogue and they’re going to learn how to communicate with each other, even though we didn’t think there was a way. Oh, come off it!” said Hinton. “And when they figure this out, they’re going to have big emotional expressions like, ‘oh my God!’ Well, of course, they don’t really mean ‘oh my God’. They’re just parroting the words we use. They’re going to learn to sacrifice themselves for the common good!

“所有这些东西在短短几年以前还被认为是彻头彻尾的幻想。而它们全都发生了。”

“All this stuff just a few years ago was written off as complete fantasy. And it’s all happened.”

辛顿(注:原文此处疑有脱漏,按语境保留上下文衔接)坦率地承认,无论是人工智能批评者的警告还是其支持者的承诺,都无法依赖于对该技术将要做什么的精确预测。但他辩称,不确定性并非沾沾自喜的理由。“人工智能独一无二的问题在于,如果它出了差错,如果它接管了一切,你不会得到第二次机会。我们不能说,‘好吧,那次我们输了。让我们重新运行游戏。’事情不是这样的。”

artily concedes that neither the warnings of AI critics nor the promises of its boosters can rely on precise predictions of what the technology will do. But, he argues, uncertainty is not a reason for complacency. “The unique problem with AI is that if it goes wrong, if it takes over, you don’t get a second chance. We can’t say, ‘Well, we lost that time. Let’s rerun the game.’ It’s not like that.”

值得一提的是,辛顿有着其他人所没有的远见。在人工智能的两个“寒冬”期间,他在多伦多大学默默无闻地埋头苦干,当时他所处的分支领域——试图教导机器以类似人类大脑的方式运作——被认为是个冷门。他全神贯注于这样一个想法:与大脑一样,可以设计出计算机系统来从数据中学习模式,而不是只遵循明确的、预先编写的规则。

It’s worth noting that Hinton has a record of seeing things where others did not. He toiled in relative obscurity at the University of Toronto during two AI “winters” where his branch of the field – trying to teach machines to operate in a manner similar to the human brain – was considered obscure. He was fixated on the idea that, like the brain, computer systems could be designed to learn patterns from data, rather than to follow only explicit, prewritten rules.

“我们相信,如果你能弄清楚大脑是如何学习的,机器就会以同样的方式学习,并且你有足够计算能力的话,它们就会做出惊人的事情,”他说。但这个尚处于萌芽状态的神经网络领域曾遭到同行的嘲笑。

“We believed that if you could figure out how the brain learned, machines learned the same way, and you had enough compute power, they would do amazing things,” he said. But the nascent field of neural networks was mocked by his peers.

计算机科学家杰弗里·辛顿在其多伦多的家中。图片来源:Cole Burston/《卫报》Computer scientist Geoffrey Hinton at his Toronto home. Photograph: Cole Burston/The Guardian

就在2009年,当辛顿的加拿大同僚兼密切合作者约书亚·本吉奥(Yoshua Bengio)向一场机器学习会议提交一篇论文时,其中一位审稿人对他们说:“这篇论文在机器学习会议上没有立足之地。它是关于神经网络的。”正是加拿大政府资助了辛顿及其小团队(其中一些人,如本吉奥,现在已成为人工智能领域的主导人物)多年看似毫无成果的钻研,才意识到了神经网络的可能性。

As recently as 2009, when Yoshua Bengio, Hinton’s Canadian colleague and close collaborator, sent a paper to a machine learning conference, one of the referees told them: “This paper has no place in a machine learning conference. It’s about neural networks.”It took Canada’s government, which funded years of seemingly fruitless tinkering by Hinton and his small team (some of whom, like Bengio, are now dominant figures in the AI field), to realize the possibility of neural nets.

2012年,他和他的学生开发出了一款图像识别程序,性能远超竞争对手。那一年,谷歌以4400万美元收购了辛顿的公司DNNresearch Inc;如今,神经网络成为最有价值的AI公司模型的基础。辛顿加入谷歌以继续深度学习研究,同时也在多伦多大学工作。“从长远来看,巨大的新突破来自基础研究,”辛顿说。

In 2012, he and his students developed an image recognition program that vastly outperformed rivals. That year, Google acquired Hinton’s company DNNresearch Inc for $44m; neural nets now underpin the models of the most valuable AI companies. Hinton joined Google to continue his research into deep learning, while also working at the University of Toronto. “In the long run, huge new breakthroughs come from basic research,” said Hinton.

这一荣誉坐落在辛顿位于多伦多郊区红砖房屋壁炉架上,这所房子坐落在一个树木繁茂的社区。奖杯堆满一地,其中包括他与本吉奥和同事杨立昆共同获得的图灵奖小银盘——图灵奖常被称为计算机界的诺贝尔奖。(辛顿还获得过诺贝尔物理学奖。)但2023年,辛顿宣布辞去这家科技巨头的职务,以便能够“自由地谈论人工智能的风险”。

The vindication sits on the mantle above the fireplace of Hinton’s red-brick home in a leafy Toronto neighbourhood. Among the jumble of trophies is the small silver dish of the Turing award – often called the Nobel prize of computing – which he won alongside Bengio and colleague Yann LeCun. (Hinton also has the Nobel prize in physics.) But in 2023, Hinton had announced his resignation from the tech giant to be able to “freely speak out about the risks of AI”.

如今,他的担忧延伸到了正在构建一些AI模型的有影响力人物,以及这些人的政治信仰。辛顿表示,人工智能风险投资家马克·安德森和彼得·蒂尔,以及新更名为SpaceXSI的公司首席执行官埃隆·马斯克,“不相信民主”。他们视自己为“超人”,认为自己应当“控制社会”,并认为特朗普是实现这一目标的有用工具,他说。

Today, his concerns extend the powerful figures building some of the AI models, and the political beliefs of those men. Hinton said that the AI venture capitalists Marc Andreessen and Peter Thiel, and the CEO of the newly rebranded SpaceXSI, Elon Musk, “don’t believe in democracy”. They see themselves as “Übermenschen” who believe they should “control society” and that Trump is a useful instrument in that aim, he said.

至于美国总统方面,他毫无疑问注意到AI公司巨大的估值如何支撑了股市,他声称唯一需要的就是一位“聪明的总统”来约束人工智能,反对任何政府保障措施,主张让公司自行监管,并在他的社交媒体页面上写道,任何不使用他偏好的AI新术语“超级智能”的人都是“敌人”。辛顿称特朗普为“腐败、衰老的罪犯”。

The US president, for his part, no doubt noticing how the huge valuations of AI firms have propped up the stock market, has claimed the only thing needed to rein in AI is a “smart president”, rejected any government safeguards in favour of allowing companies to regulate themselves, and wrote on his social media page that anyone who doesn’t use his preferred new term for AI, “super intelligence”, is “THE ENEMY”. Hinton called Trump “a corrupt, senile felon”.

“我确信石油和石棉公司也希望能够自行监管,”他说。“我们认为烟草公司应该被允许设计包装上的警示标语吗?这样他们就能用六号字体在黑色背景上印出深灰色的文字?”特朗普使用“超智能”一词证明他对人工智能“几乎一无所知”,辛顿补充道。

“I’m sure the oil and asbestos companies would have liked to [regulate themselves] as well,” he said. “Do we think tobacco companies should have been allowed to design the warnings on the packets? So that they could print them in a six-point font, in dark grey on black?”Trump’s use of “superintelligence” proves he understands “almost nothing” about AI, Hinton added.

“他拿了一个意义非常明确的技术术语,然后重新使用了它,”辛顿说。“超智能意味着比人更聪明,这一概念至少已经存在20年。它有非常明确的含义。而现在他说我们将把人工智能称为‘超智能’。仅仅通过称其比人更聪明,就无法让它真正变得比人更聪明。”技术发展的速度和范围引发了人们对灾难的担忧,以及社会无法应对挑战本质的焦虑。但辛顿还看到另一个障碍:帮助人们理解公司正在创造的“外星生物”。

“He’s taken a technical term which has a very clear meaning and then reused it,” Hinton said. “Superintelligence means smarter than people, and has been around for at least 20 years. It’s got a very clear meaning. And now he says we’re going to call AI ‘superintelligence’. You can’t make it smarter than people just by calling it smarter than people.”The speed and scope of developments has promoted a worry of catastrophes and of society’s inability to grapple with the nature of the challenge. But Hinton also sees an additional obstacle: helping people understand the “alien beings” that the companies are creating.

10月8日,唐纳德·特朗普在华盛顿特区与国家科学奖获得者合影,其中包括谷歌的谢尔盖·布林、英伟达的黄仁勋和埃隆·马斯克。图片来源:Julia Demaree Nikhinson/APDonald Trump poses with National Medal of Science recipients, including Google’s Sergey Brin, Nvidia’s Jensen Huang, and Elon Musk on 8 October in Washington DC. Photograph: Julia Demaree Nikhinson/AP

“人类对自身有着非常错误的认识,这就是他们无法理解这些AI模型为何如此类似人的原因。大多数人不明白什么是意识。他们不明白什么是主观体验,并且非常确信,尽管他们自己也不理解,但AI肯定没有这种体验。”他否定了人类心智具有机器无法具备的神秘品质的观点。“这些AI模型显然理解它们在说什么。如果不理解问题,就不可能回答所有问题。说它们不理解是荒谬的。”但辛顿表示,围绕这些模型的社会思维转变范围之深远是显而易见的。

“People have a very bad model of what people are, and that’s why they fail to understand how these AI models are very like us. Most people don’t understand what consciousness is. They don’t understand what subjective experience is, and they’re very confident that even though they don’t understand it, they’re sure AI doesn’t have it.”He rejects the idea that the human mind possesses some mysterious quality that machines, by definition, cannot“It’s obvious that these AI models understand what they’re saying. You couldn’t possibly answer all the questions without understanding the question. It’s ridiculous to say they don’t understand.”But the scope of shift in societal thinking around these models is profound, said Hinton.

“过去曾有过三次重大变革,让人类的地位变得不再那么重要。第一次是哥白尼提出的理论,我们不再处于宇宙的中心。人们花了多久才适应这一点?天主教会花了近360年才向伽利略道歉。还有达尔文呢?好吧,堪萨斯州还有人至今没能接受这一点,”他说。“现在我们正在创造外星生物——这意味着我们不再是周围唯一的智能生物。现实是,我们只有几年的时间来适应这一点。”

“There’ve been three big revolutions where people ceased to be so important. The first was Copernicus, where we weren’t at the centre of the universe any more. How long did it take people to adapt to that? It took the Catholic church nearly 360 years to apologize to Galileo. And Darwin? Well, there’s people in Kansas that still haven’t adapted to that one,” he said. “Now we’re creating alien beings – meaning we’re not the only intelligent beings around. And the reality is, we only have a couple of years to adapt to that.”

最近几个月,热门聊天机器人Claude的制造商、人工智能巨头Anthropic与宗教学者、神学家和哲学家举行了闭门会议,探讨教导人工智能变得“善良”的可能性,并应对这些模型可能具备意识的现实。Anthropic首席执行官达里奥·阿莫代伊表示,这些对话借鉴了几个世纪以来关于美德、同情心、诚实和道德品质的宗教与哲学思想,旨在使合乎道德的行为更加可靠。

In recent months, the AI giant Anthropic, maker of the popular chatbot Claude, has held private meetings with religious scholars, theologians and philosophers to look into the possibility of teaching AI to be “good” and to grapple with the possibility that the models have consciousness. Anthropic’s CEO, Dario Amodei, said the conversations have drawn on centuries of religious and philosophical thought about virtue, compassion, honesty and moral character, with the aim of making ethical behaviour more reliable.

“一旦你开始谈论‘我们对这些外星生物负有道义上的责任’,你就会失去听众,”辛顿说。“我一直在与这个问题作斗争。我们在乎人。(但)我们正在制造这些东西,没有什么能阻止我们赋予这些比我们更聪明的东西比我们更少的权利。我愿意自私一点,选择人类。当然,我们应该努力让它们在与我们的共生关系中感到非常快乐,在这种关系中,我们的关切比它们的关切受到更重的权衡。”

“Once you start talking about, ‘We have a moral obligation towards these alien beings,’ you lose your audience,” said Hinton. “I’ve struggled with this issue. We care about people. [But] we’re making these things and there’s nothing to stop us giving these things – which are more intelligent than us – less rights than us. I’m willing to be selfish and choose human beings. Of course, we should try to make them so that they’re very happy in a symbiotic relationship with us, where our concerns are weighed more heavily than their concerns.”

辛顿并不是唯一一个认为避免社会崩溃的最佳途径是开发真正仁慈的模型的人。他希望无论哪个国家率先具备这种能力,都应该将其告知盟友和竞争对手,因为“没有哪个国家希望人工智能接管”另一个国家。他说,这其中存在着“分享知识的既得利益”。

Hinton is not alone in arguing that the best route to avoid societal collapse is to develop genuinely benevolent models. He hopes that whichever country is the first to develop this capacity, it would tell allies and rivals alike, because “no country wants AI taking over” in another country. There is a “vested interest in sharing knowledge”, he said.

“现实是,我们正在创造一种完全不同的‘存在’(即人工智能)。而那些大型科技公司总是说:‘我们希望这些人工智能更加智能。’这简直太荒谬了!我们应当像关注人工智能的智能水平一样,同样关注它们的其他所有方面。”他说,人工智能的许多行为特征其实取决于其自身的‘本质’,以及它们学习如何表现出良好行为的能力。

“The reality is, we’re making beings of a different kind. And the big tech companies keep saying: we want it smarter. This is crazy. We should care as much about all the other aspects of a being as you do about its intelligence.”Much of that depends on the nature of these models, he said, and their capacity to understand how to develop good behaviour.

“当你用互联网上的所有信息来训练它们时,它们就会养成各种不良的习惯;之后你只能强迫它们压抑这些不良行为,”他继续说道。然而,仅仅压制不良行为并不能真正造就一个‘有良好品格’的人工智能。这就好比按照维多利亚时代的标准来‘设计’一个绅士一样——但这样的‘绅士’显然并不适合我们现代社会的生活需求。”

“When you train them on everything on the internet, they develop all sorts of horrible tendencies and then you get them to repress their horrible tendencies,” he said. But suppressing bad behaviour is not the same as creating a good character. “It seems like the way one would design a Victorian gentleman – but not something we’ll need to live with.”