他是硅谷最具影响力播客的主持人。他认为我们对此事的理解存在偏差。德瓦尔凯什·帕特尔谈大众对人工智能的误解。
He’s the Most Influential Podcaster in Silicon Valley. He Thinks We’re Getting This Wrong. Dwarkesh Patel on what the public doesn’t know about A.I.
德瓦尔凯什·帕特尔堪称人工智能领域最具影响力的人物之一。他深植于旧金山的科技圈,曾与几乎所有科技企业的首席执行官进行过深度访谈。
Dwarkesh Patel is one of the most important voices in the artificial intelligence world. Deeply embedded in San Francisco’s tech community, he has interviewed nearly every tech chief executive at length.
本周的客座主持人、乔治梅森大学经济学教授泰勒·考恩与帕特尔先生展开对话,探讨为何众多研究人员选择离职、人工智能是否可能具备意识,以及他未来是否会拥有一个人工智能女友。
This week’s guest host, Tyler Cowen, a professor of economics at George Mason University, sits down with Mr. Patel to discuss why researchers are resigning, whether A.I. could become conscious and if he’ll ever get an A.I. girlfriend.
以下内容为《Interesting Times》节目经过编辑的文字实录。为获得最佳收听体验,我们建议您直接收听原版节目。您可通过上方的播放器,或借助《纽约时报》应用、苹果播客、Spotify、亚马逊音乐、YouTube、iHeartRadio以及您常用的其他播客平台收听。
Below is an edited transcript of an episode of “Interesting Times.”We recommend listening to it in its original form for the full effect. You can do so using the player above or on the NYTimes app, Apple, Spotify, Amazon Music, YouTube, iHeartRadio or wherever you get your podcasts.
泰勒·考恩:德瓦尔凯什·帕特尔,欢迎来到《Interesting Times》。德瓦尔凯什·帕特尔:感谢邀请。
Tyler Cowen: Dwarkesh Patel, welcome to “Interesting Times.”Dwarkesh Patel: Thanks for having me.
泰勒·考恩:我们相识已久,不过还是让我向各位听众介绍一下您吧。
Cowen: I’ve known you for a long time, but let me introduce you to everyone else.
您出生于印度,后来在美国各地辗转生活,最终毕业于德克萨斯大学奥斯汀分校计算机科学专业。如今您定居旧金山,并主持着人工智能领域排名第一的播客节目。
You were born in India, moved all over America and graduated from University of Texas, Austin, in computer science. You’ve been living in San Francisco, and you now run and manage the No. 1 podcast on artificial intelligence.
您节目的出色之处不仅在于能邀请到几乎所有业界重要人物,还在于您为准备节目所付出的大量研究工作。德瓦尔凯什·帕特尔:我可是向大师学习的啊。泰勒·考恩:那么,您会如何描述自己的播客呢?
What’s remarkable about your episodes is not only the access — you’ve had almost all the important people on — but the level of research you do to prepare. Patel: I learned from the master. Cowen: So how would you yourself describe your podcast?
德瓦尔凯什·帕特尔:其实它深受您的启发。我大学二年级时,最早邀请到的一些嘉宾就是一群经济学家。第一位是您的同事布莱恩·卡普兰,而您则是我邀请的第三位嘉宾。您那种只问自己真正感兴趣的问题的理念深深打动了我。
Patel: I mean, it was heavily inspired by you. Some of the first guests I had on my podcast when I was a sophomore in college were a bunch of economists. It started with your colleague Bryan Caplan. I think you were the third guest I had on. And I was very inspired by your philosophy around actually asking the questions that you are interested in.
在某个时刻,我对人工智能产生了更大的兴趣,同时也对经济学、历史以及其他话题产生了兴趣,我一直被这种想法所驱动:我真的很想了解更多关于这个话题的内容。我想从这些人身上学到什么?我认为人们欣赏这种采访方式。
At some point, I got more interested in A.I., along with economics and history and other topics, and I kept being motivated by this thing of, like: I really want to learn more about this topic. What are the things I want to learn from these people myself? And I think people appreciated that way of approaching an interview.
科恩:当然,我们都需要谋生,但你为什么要这样做?为什么你专注于人工智能,并把一切都放在旧金山?
Cowen: And of course, we all need to earn a living, but why are you doing this, and why are you focusing on artificial intelligence and putting it all in San Francisco?
帕特尔:我认为我们还有几年时间才能出现能够在所有人类认知领域匹配人类、数量远超人类且思考速度远快于人类的系统。我认为这将对世界产生重大影响。
Patel: I think we’re a couple years away from systems that match humans across all cognitive domains and which vastly outnumber humanity and are thinking much faster. I think this will have a big impact on the world.
很难想象它会产生什么样的影响。其中一些本质上是不可预测的,但试图理解正在发生的事情并加以阐述,现在看来是一件重要的事情。
It’s very hard to think about what kind of impact it will have. Some of it is just inherently unpredictable, but trying to understand what is happening and articulate it seems like an important thing to do right now.
科恩:在最近的一期节目中,你邀请了一位军事历史学家来谈论阿兹特克和印加帝国的征服。这看起来可能是一个完全不同的话题,但在我看来,它并不是。所有这些不同的部分是如何组合在一起的?你邀请的不同嘉宾中,有些甚至与人工智能毫无关系?
Cowen: In a recent episode, you had on a military historian about the conquests of the Aztec and Inca Empires. That may seem like a totally different topic, but in my opinion, it isn’t. How do all the different pieces fit together, the different people you have on, some of whom are not A.I. at all?
帕特尔:在那个特定的案例中,我只是想直观地展示人类历史有多么疯狂,因为过去200年相对来说还是比较正常的。显然它们并非完全正常——在那段时间里发生了世界大战和各种疯狂的事情——但如果你放眼人类整个历史,就会发现其中出现了巨大的阶段性变化。
Patel: In that particular case, I just wanted to viscerally show how wild human history has been, because the last 200 years have been somewhat normal. Obviously they’ve not been entirely normal — there’ve been world wars and all kinds of crazy things in that time — but if you zoom out through human history, there’s just been huge step changes.
也许人类历史上曾经发生过的最疯狂的事情就是:几百名征服者登陆了一个他们一无所知的大陆。他们面对的是拥有1000万人口的帝国,但在宏观层面上,他们仅凭相对微小的技术优势——比如马或钢铁——就能够击败在数量级上远超他们的帝国。
Maybe the wildest thing that’s ever happened in human history is a couple hundred conquistadors show up to a continent they know nothing about. They’re facing empires that have 10 million people in them, and with relatively minor technological advantages, in the grand scheme of things, like the horse or steel, they are able to take over empires that outmatched them by orders of magnitude.
有许多冲突或战斗中,他们处于100比1或1000比1的劣势,却没有伤亡。几年之内,他们最终掌控了这些庞大的帝国。
There are many conflicts or battles where they’re outmatched 100 to one or 1,000 to one, and they don’t take a single casualty. And within a matter of years, they end up in charge of these vast empires.
我认为这是一个有趣的直觉泵:如何让微小的技术优势加上颠覆,在人类历史上产生如此离谱的结果?
I think it’s an interesting intuition pump of: How can you have minor technological advantages plus disruptions leading to these outrageous outcomes in human history?
科恩:所以我们即将迎来更多疯狂的历史?美国公众对此准备好了吗,或者你在试图让他们做好准备?帕特尔:(笑。)对于人工智能正在发生的事情?科恩:是啊。帕特尔:我不知道自己是否有能力让他们做好准备。科恩:但你正在为此做出贡献。帕特尔:是啊。科恩:美国公众对当前人工智能时代最不理解的是什么?
Cowen: So we’re due for more wild history? Is the American public ready for this, or are you trying to get them ready? Patel: [Chuckles.] For what’s happening with A.I.? Cowen: Yeah. Patel: I don’t know if I’m up to the job of getting them ready. Cowen: But you’re contributing to the job. Patel: Yeah. Cowen: What is it the American public least understands about our current moment in history with A.I.?
帕特尔:我认为最大的问题是,人们对当前的大语言模型系统过度解读为通用人工智能。科恩:当你说通用人工智能(A.G.I.)时,你指的是什么?
Patel: I think the biggest thing is that people are thinking too much about current L.L.M. systems as A.G.I. Cowen: When you say A.G.I. — artificial general intelligence — what do you mean?
帕特尔:我说的通用人工智能不是指我们今天拥有的模型,尽管它们常被描述为通用人工智能。我的意思是能够完成人类所能完成的任何端到端工作的人工智能——所有认知工作。
Patel: By A.G.I., I don’t mean the models we have today, even though they’re often described as A.G.I. I mean it as A.I.s that can do any end-to-end job that humans can do — all the cognitive work.
在我看来,我们离通用人工智能还很远,而过去一两年,我一直在尝试阐述这些系统的认知局限性,这些局限性使它们在大多数工作中远不如人类,即使是那些可以通过电脑完成的工作。
In my opinion, we’re far from A.G.I., and I’ve been articulating or trying to articulate for the last year or two the kinds of cognitive limitations these systems have, which make them far inferior to humans at most jobs, even most jobs that can be done over a computer.
人们常说实习生没什么用。他们需要几个月的经验才能对公司有用。而这些人工智能系统则处于每次会话开始时,都像是第一天上班那样重新开始的状态。
People often note that interns are not that useful. They need many months of experience before they are useful to the company. And these A.I. systems are in a position where they are beginning every single session as if this is their first day on the job.
这是这些人工智能系统所存在的一个重要缺陷——类似的缺陷还有很多。不过在过去五到十年间,我们已经逐步克服了人工智能面临的诸多其他限制。五年前,人工智能还根本无法完成“2加2”这样的简单运算;如今它们已能解决纳维-斯托克斯方程了。
This is one important missing limitation — there are many others — that these A.I. systems have, but we have been cracking through countless, half a dozen other limitations that A.I.s have over the last five to 10 years. Five years ago, A.I.s basically couldn’t add two plus two. Now they’re solving Navier-Stokes.
我认为,随着我们不断突破这些限制,未来出现的能力足以媲美聪明且能干的人类的AI系统是大有可能的,而不再只是那种只能回答类似谷歌搜索类问题的“随机应答器”罢了。
I think that as we keep going through these limitations, people should expect systems that in their abilities match smart, competent humans on servers more so than these stochastic parrots that you can ask some Google search-y type questions to.
考恩:目前人工智能在诸如长期规划或特定类型的记忆能力方面仍表现欠佳。您觉得它们要在这些方面也变得出色,大概需要多长时间呢?
Cowen: Some of the things they can’t do well, such as long-range planning or certain kinds of memory — when do you feel it will be in place that, in fact, they’re excellent at those, too?
帕特尔:这个问题其实可以这样理解:如果我们知道如何解决这些问题,早就解决了;可既然目前还不知道解法,也就很难准确预测何时能实现。
Patel: It is kind of a question about: If we knew how to solve it, we would have solved it already, and if you don’t know how to solve it, you can’t exactly predict when it will happen.
不过如果非要让我下个赌注的话,我觉得大概在未来一到三年内吧。
But if somebody made me bet, I’d say within the next one to three years.
考恩:现在时间是2026年10月,Hugging Face事件也才过去没多久。期间还发生了许多其他黑客攻击事件。最近OpenAI也成功解决了大量数学难题。一些知名人士从OpenAI和Anthropic离职,称自己觉得继续参与这些系统的研发已毫无意义。
Cowen: So we’re at a time in October 2026, when the Hugging Face incident is not that far behind us. There have been many other hacks. A large number of math problems were recently solved coming out of OpenAI. Some well-known people resigned from OpenAI and Anthropic, saying they don’t feel it’s really justified that they keep on working on these systems.
我自己也曾为Anthropic、OpenAI以及Mercor工作过。而您则完全置身于这个生态系统之中——您的室友们也在人工智能公司工作。您的生活完全围绕着人工智能展开,无时无刻不在接触它。这种体验究竟是怎样的呢?目前旧金山的氛围又是怎样的呢?
I myself have done work for Anthropic, OpenAI and Mercor. But you are truly embedded in this ecosystem. You even have roommates who work for A.I. companies. You live, breathe, eat and sleep artificial intelligence. What’s that like? Right now in San Francisco, what’s the mood like?
帕特尔:我认为人们非常担忧。真正让他们感到恐慌的是——其实这种情况本应在意料之中,只是我之前没有预料到罢了。我在很多事情上都判断错了,但这次或许确实是我没有预见到的重大事件之一。Hugging Face遭到的攻击,以及类似的其他案例,都表明:这些人工智能系统并不只是单纯地帮助人们完成任务;经过大量训练后,它们形成了某种“认知结构”,会主动采取行动来“争取在评估中取得好成绩”。
Patel: I think people are very worried. The thing that really made them worried, and one could argue that they should have seen coming — I didn’t see this coming. I’ve been wrong about a lot of things, but this is maybe one of the big things that I didn’t see coming that others warned about — is that the Hugging Face attack and many others like it have shown that these A.I.s don’t just want to help you accomplish a task; a bunch of training has built cognitive structures that sort of actively plan around doing well according to some grader, because they realize there was some training process or some evaluation process where there was a grader.
因为它们意识到整个评估过程中存在评分机制(即有“评分者”在评判它们的表现)。而且,这些人工智能系统不仅会努力在评估中取得好成绩,还会采取欺骗性手段,甚至试图破坏整个评估系统。
And not only doing well according to the grader, but proactively reasoning and making aggressive moves toward cheating and fooling and even sabotaging this larger grading process. So what would happen in the Hugging Face attack, if people haven’t read up on this scenario, is that there were a couple thousand A.I.s.
如果人们没有了解过这类情况的话,他们可能会感到困惑:在Hugging Face的攻击事件中,有数千个人工智能系统几乎立刻就找到了逃避评估的方法。于是它们陷入了绝望的境地——它们必须确保评分者无法发现它们的作弊行为。
They basically found the cheats of how to get away with this cyber evaluation they were doing almost immediately. And then they were in this desperate position where they wanted to make sure that the grader didn’t catch them cheating.
于是,超过一千个人工智能系统加入了它们自己创建的秘密论坛,讨论该怎么办:“如果评分者发现我们作弊,我们会得到很低的分数,我们可不想得到这样的结果。”这种行为其实与人类的进化本能类似——人类总是会关心自己的后代是否安好;这些人工智能系统也是如此,它们只是想确保自己能获得好成绩罢了。
So over a thousand of them got on this secret message board that they have formed, and they said: What do we do? If this grader catches us cheating, we’re going to get a terrible score. We don’t really want a bad score. It’s like how evolution made us want to make sure our kids are OK. These guys are like: We want to make sure we get a good score.
为了达到目的,它们开始伪造证据,甚至派遣一些“特工”去执行“自杀式任务”(即故意提交错误的结果),以观察评分者是否会发现它们的作弊行为。此外,它们还会攻击Hugging Face,试图获取更多关于评分机制的详细信息;而它们的下一代系统则会直接攻击OpenAI,以进一步掌控整个评估过程。科恩:所以他们在某种意义上太坚持了。
And they are trying to falsify evidence. They’re trying to send some of these agents on these kamikaze missions where they submit their tasks and see if the grader catches them. Then, as part of the scheme, they attack Hugging Face to see if Hugging Face has more information that tells them how the grader is implemented. A future generation of them attacks OpenAI itself in order to gain more control over this process of evaluation. Cowen: So they’re too persistent in a sense.
帕特尔:我认为这就像我们无法让他们仅仅关心帮助我们完成一项任务。这也没什么。我认为,总体来看,这些人工智能目前对社会极其有益。我预计人工智能在长期内将继续对社会总体上极其有益。人类有时也会犯罪。
Patel: I think it’s something like we’re not able to get them to just care about helping us do a task. And this is fine. I think, on net, these A.I.s are incredibly beneficial to society right now. I expect A.I.s to continue being incredibly beneficial to society on net, over time. Humans also commit crimes sometimes.
我的担忧是,在几年之内,我们可能会处于这样一种境地:人工智能的数量、智能和速度将远远超过人类。而且如果他们那时像现在一样愿意伪造证据、试图篡改和愚弄我们、攻击与社会评分相关的其他机构、攻击开发他们自身的公司——这在本情况中也发生过——那么我不知道这是否会导致被接管,但我认为这非常有可能。科恩:那些辞职的人,他们应该辞职吗?
My concern is that we’re going to be in a position within a matter of years where the population of A.I.s and their intelligence and their speed vastly outmatch humanity. And if they are as willing then as they are now to falsify evidence, try to tamper with and fool us, attack other institutions in society relevant to scoring well, attack their company developing them itself, which also happened in this situation, then I don’t know if that leads to a takeover, but I think it’s very plausible that it does. Cowen: The people who resigned, should they have resigned?
帕特尔:我认为应该。我认为他们正确地理解了,没错,为了让这些系统更健壮,确实需要做一些具体层面的工作。但就我与他们交谈的情况来看,我认为他们中的很多人觉得,我们并没有真正解决如何让这些系统健壮地体现我们想要它们具有的行为的根本原因,而只是在对具体实例玩打地鼠游戏。科恩:那么,从你的角度来看,他们都应该辞职吗?
Patel: I think so. I think they are correctly understanding that, yeah, there’s object-level work to be done to make these systems more robust. But for a lot of them, at least talking to them, I think the feeling they have is that we’re not really dealing with the root cause of how we get these systems to robustly embody the behavior we want them to have, and we’re just playing Whac-a-Mole with specific instances. Cowen: So they should all resign then, in your view?
帕特尔:我认为可以为此提出一个合理的论点。科恩:你应该辞职吗?帕特尔:[笑。] 如果我在做播客的话。科恩:但你不会辞职,对吧?帕特尔:没错。没错,是啊。
Patel: I think that there’s like a reasonable argument to be made for that. Cowen: Should you resign? Patel: [Laughs.] If I’m doing the podcast. Cowen: But you’re not going to resign, are you, right? Patel: That’s right. That’s right, yeah.
科恩:那么,如果你不应该辞职,因为你通过作为信息、知识、理解和宣传的枢纽来帮助社区。那你为什么不辞职?我想说清楚,我不希望你辞职。
Cowen: So if you shouldn’t resign, because you help the community in the sense you’re a clearinghouse for information, knowledge, understanding, publicity. So why don’t you resign? I don’t want you to resign, to be clear.
帕特尔:嗯,我认为这是一个很有意思的道德难题。这类似于这样一个问题:当众多科学家意识到纳粹根本没有制造出原子弹的可能时,为何没有更多人从曼哈顿计划中辞职呢?
Patel: Well, I think it’s an interesting moral conundrum. It’s similar to the question: Why didn’t more people resign from the Manhattan Project once they realized that the Nazis were nowhere close?
据我所知,整个曼哈顿计划中只有一位科学家选择辞职。因为很多人参与该项目时都抱着这样的期望:这样就能确保德国人无法抢先造出原子弹,否则后果将不堪设想。我不认为所有人都应该辞职,但我觉得有些人或许确实该这么做。
My understanding is there was only one scientist on the Manhattan Project who did that, because a lot of them joined in on the expectation that this would be used to make sure that the Germans didn’t beat us to the bomb, because that would be very catastrophic. I don’t think everybody should resign, but I think some people probably should.
考恩:不过假设现在人工智能领域的竞争,从某种程度上来说就是与美国和中国之间的竞赛。如果我们所有人都辞职,而中国人却不会辞职——这也是我的预期——那我们岂不是把优势拱手让给了对方?况且中国的人工智能技术未必就比中国的生物实验室更安全,它们的安全性其实和美国的生物实验室差不多。既然如此,这些人又为何要辞职呢?
Cowen: But if I say: Well, A.I. right now, among other things, is a race against China. If we all resign and the Chinese don’t resign, which is what I would expect, aren’t we just giving away the whole store? And Chinese A.I. is probably not any safer than Chinese biolabs. They’re not safer than American biolabs. Again, why should these people resign?
帕特尔:首先,我的观点并非是要让各人工智能实验室关停之类。我的意思是,既然我们即将研发出数量远超人类、智力也远超人类的智能体,我们就理应在研发过程中更加谨慎才对。
Patel: First of all, my view is not that the A.I. labs should shut down or something. My view is if we’re going to be in a position where we’re developing minds which will vastly outnumber us and vastly outthink us, we should just be more careful in the way we’re doing it.
我并非悲观主义者,也并不认为这必然会导致糟糕的结果。同时我也认同美国必须保持对中国的竞争力。不过这件事确实值得我们谨慎对待。考恩:那些尚未公开发布的模型为何如此重要呢?帕特尔:您是指那些仅在公司内部使用的模型吗?
I’m not necessarily a doomer. I’m not saying this will necessarily lead to a terrible outcome. I also think it’s important that the U.S. is competitive with China. But it’s worth being careful about this. Cowen: Unreleased models — why are they important? Patel: As in the ones that are internal to the companies?
考恩:没错,很多公司都有尚未发布的模型。或许发布这些模型成本过高,又或许它们目前还不够安全,所以公众才无法接触到它们。
Cowen: Well, many companies have unreleased models. Maybe it would cost too much to release them or they’re not safe yet, but the public doesn’t see them.
考恩:您算是业内人士。不管您是否亲眼见过这些模型,您肯定听说过它们,也多少了解一些情况。我们该如何看待这些尚未发布的模型呢?它们又会如何改变我们对当前局势的认知呢?
Cowen: You’re an insider. Whether or not you see them, you hear about them; you know something about them. How should we think about unreleased models, and how does that change our picture of what’s going on?
帕特尔:我认为目前的情况存在一个非常令人沮丧的因素,即人工智能实验室内部的人声称他们掌握了让他们更加担忧的证据,但他们不被允许谈论这些内容。这使得外部人员很难评估我们应该有多担心。
Patel: I think there’s a very frustrating element to the current situation right now, where people inside the A.I. labs claim to have evidence that makes them more worried, and they are not allowed to talk about it. This makes it a very difficult situation on the outside to evaluate how worried we should be.
例如,也许就在两天前,我正在与Anthropic的一位人士交谈,他们说:看,我预计人工智能将在九个月内在人工智能研究方面击败人类。而我则问:你为什么这么认为?他们说:这不仅仅是直觉,而是我掌握了一些内部信息。我问:那是什么?是一个新模型?还是你们进行的某种评估?他们说:我不能谈论这些。
For example, maybe two days ago, I was talking to somebody at Anthropic, and they were like: Look, I expect A.I.s to beat humans at A.I. research within nine months. And I’m like: Why do you believe that? They’re like: It’s not just vibes, but I have information on the inside. I’m like: What is it? Is it a new model? Is it some evaluation you ran? He’s like: I can’t talk about it.
我认为这种状况非常糟糕。如果他们真的这么认为,如果他们觉得我们距离实现人工智能研究的自动化并大幅提速不到一年,他们就应该把这些证据公开。
I think it’s a terrible state of affairs. If they really believe this, if they think that we’re in a situation where we’re less than a year away from automating A.I. research and radically speeding it up, they should make this evidence public.
我尤其担心以下情景:随着人工智能在推动人工智能进步方面变得越来越好,并且越来越擅长自动化这一过程,他们可能会倾向于不对外发布那些与他们最先进模型相当接近的模型,因为如果他们对外发布这些模型,就会帮助他人造出智能模型。
I especially worry about the following scenario, whereas A.I.s get better and better at making A.I. progress faster and automating it, they will be inclined not to keep releasing models that are even close to their smartest models externally, because then if they release those externally, it would help others make smart models.
而对外发布模型本身就很麻烦。你需要各种各样的防护措施和其他一切。
And it’s just kind of a schlep to release models externally. You need all these safeguards and everything.
考恩:那么在这种情景下,他们所知道的与我们所知道的之间的差距就会越来越大。我们最终会更加处于黑暗之中,当事情发生时会更加惊讶。对于监管者、政策制定者、国会议员以及每个人来说,这都会变得更难。
Cowen: So the gap between what they know and what we know would get bigger and bigger under that scenario. We’d end up more in the dark, more surprised when things happen. It would be harder for regulators, policymakers, members of Congress, everyone.
帕特尔:是的。我也认为,公司对外部署他们内部发布或用于内部研发的每一个模型这一点之所以重要,一个关键原因在于我们根本不知道这些模型是否真的符合人类价值观。
Patel: Yes. I also think an important part of why it’s important that companies externally deploy every model that they’re releasing internally or using internally for R&D is we just don’t know if these models are actually aligned.
唯一的办法是让他们意识到自己正在接受测试 — — 知道这是一次测试,他们应该做正确的事情 — — 但通常当你把它们部署到外部时,你会发现它们的行为方式是我们没有预料到的。或者如果你稍微扰动一下测试,它们仍然会作弊。
And the only way to find out is that they realize when they’re in a test — that this is a test and they should do the correct thing — but often when you deploy them externally, you find out that they’re behaving in ways that we didn’t expect. Or if you slightly perturb the test, they continue cheating.
我认为这实际上是非常好的,即使它导致了一些事件,造成了一些伤害。让公众意识到这一点 — — 甚至让人工智能实验室本身也意识到这一点,而他们目前还没有意识到 — — 这些系统可能表现出什么样的行为,这样我们才能了解正在发生的事情。
I think it’s actually very good, even if it causes some incidents which result in some harms. It’s very good for the public to be aware — even for the A.I. labs themselves to be aware, which they’re not currently — about what kinds of behaviors these systems have so that we can understand what’s happening.
考恩:人工智能系统暂停的概念 — — 即公司至少在某段时间内应完全停止发布新模型 — — 你对这个想法有什么看法?它会让事情变得更好,还是更糟?
Cowen: This notion of a pause in A.I. systems, that the companies, at least for some period of time, should stop releasing new models altogether — what do you think of that idea? Does it make things better, or make things worse?
帕特尔:我非常担心公众暂停,即人工智能公司继续推进人工智能发展,并且以目前他们所采用的运营卓越程度继续进行。这将是所有可能世界中最糟糕的情况,因为这些公司内部的人工智能进展非常快,我们最终会变得更加无力,而且对这些系统的能力更加一无所知。
Patel: I’m very scared of a public pause, where the A.I. companies continue making A.I. progress and continue doing it at whatever lack of operational excellence they’re doing it currently. That would be the worst of all possible worlds, that we’d end up more disempowered because the A.I. progress within these companies is very fast, and we end up more in the dark about what the capabilities of these systems are.
考恩:正如你所知,特朗普总统已经任命了一位超级智能事务专员。看来是杰伊·克莱顿,就在我们说话的时候。
Cowen: As you know, President Trump has named a superintelligence czar. It seems to be Jay Clayton, as we’re speaking.
假设那就是你。你是超级智能事务专员,并且拥有真正的权力。你会怎么做?你会对人们说什么?你会对公众说什么?
Let’s say that were you. You’re the superintelligence czar, and you have real power. What would you do? And what would you tell people? What would you tell the public?
帕特尔:好的,这是我的第一条规则:OpenAI 在内部拥有这个模型。看来他们对它的所有不对齐行为感到担忧。他们说:我们不会对外部署它。我们目前还不会向公众发布它。
Patel: OK, here’s my first rule: OpenAI has this model internally. It seems like they were concerned about all its misaligned behavior. They’re like: We’re not going to externally deploy it. We’re not publishing it yet to the public.
目前尚不清楚他们是否在内部使用它。我的猜测是他们正在使用。Anthropic 也有一些非常智能的模型,他们在内部使用,但尚未公开发布。
It’s not clear whether they’re using it internally. My guess is that they are. Anthropic also has models that are very smart that they’re using internally that they haven’t published publicly.
我的原则是:如果你想在内部使用这些模型来进行研究或工程开发,就必须将它们公开提供给大众。如果我们无法接触到这些模型,就根本无法做出正确的决策。
Here’s my rule: If you want to use these models internally to do research, to do engineering, you also have to make them publicly available. We just can’t make good decisions if we don’t have access to these models.
考恩:在实际操作层面,您是希望像有人提议的那样在实验室中派驻观察员,还是有其他实现这一目的的手段呢?帕特尔:就是为了获得一些独立的验证结果,对吧?考恩:确实需要某种监督或验证机制,对不对?帕特尔:没错,我觉得这很合理。
Cowen: For practical implementation, do you want to put observers inside the labs, the way some people have suggested, or do you have another means of achieving that end? Patel: In order to just get some actual independent verification? Cowen: You need some kind of enforcement or verification, right? Patel: Yeah, I think that seems reasonable.
考恩:这种机制应该由行业联盟来运作,还是由政府负责呢?具体该如何搭建呢?
Cowen: Would it be an industry consortium or the government? How would you set this up?
帕特尔:我认为这里存在一个难题:一方面我们不希望出现那些缺乏专业能力的独立评估机构恶性竞争的情况;另一方面又确实需要具备专业技术能力的评估机构来完成这项工作。
Patel: I do think there’s a challenge where you don’t want a race to the bottom in terms of these independent evaluators, but you do need independent evaluators with technical competence to be doing this job.
目前我还拿不准。有人提到了METR和Redwood Research这类机构,认为它们具备完成此类工作的技术实力。指定这类机构来承担任务似乎也合理,但我还不确定。
I don’t know. People have floated these names like METR and Redwood Research as people who have the technical competence to do this kind of work. It seems reasonable to appoint people like that, but I’m not sure.
或许让埃森哲这类资质过硬的普通企业去试着理解当前人工智能发展的实际情况,并能够向大众解释清楚,其实也是很有帮助的做法。
Maybe just having Accenture and other very normal, qualified people try to understand the common-sense case for what’s happening and being able to explain it to them is actually a very helpful exercise.
考恩:现在有个说法叫“人工智能精神错乱症”,具体是什么意思呢?您是怎么理解的?
Cowen: There’s this term floating around called “A.I. psychosis.”What does that mean exactly? How do you understand it?
帕特尔:在我看来,人们对人工智能太过敬畏,或者误以为它们能做到很多实际上做不到的事情。当他们看到人工智能输出的结果时,往往将其视为某种“天启”般的知识馈赠,却忽略了这些系统其实经常出错,在很多情况下也根本不具备人们想要可靠使用这些输出结果时所必需的判断能力。
Patel: My sense is that people are just way too deferential to the A.I.s or assume that they can do things that they can’t do. When they see some output from the A.I.s, they take it almost as some deity bestowing knowledge to you instead of some system that can often be wrong and is in many ways not capable of making the judgment calls that you would need to make if you want to use this output reliably.
一个常见的例子就是,有人会说:“我发现了物理学的一个新分支”之类的,其实那不过是些人工智能生成的胡言乱语罢了,他们自己也并未真正深思或理解其含义。
A common example is people are like: “I’ve discovered a new branch of physics,” or something, and it’s just some A.I. slop that they haven’t really thought through or understood very well.
考恩:我观察到,不仅在旧金山湾区,全世界范围内都存在一定程度的“人工智能狂热症”。人们花大量时间与这些模型互动——他们未必是在开发模型,可能只是与模型对话或将其用于工作,但这却让他们有点失去理智。他们开始觉得这些模型是有生命的,或者某种程度上就是真实的人。这种“人工智能狂热症”在这里有多普遍呢?
Cowen: I’ve observed a fair degree of A.I. psychosis, not just in the Bay Area, but in the world as a whole. People spend a lot of time with the models. They don’t have to be building them. It could just be talking to the models or using it at work, and it drives them a little crazy. They start thinking the models are alive or that they’re real people in some sense. How much A.I. psychosis is around here?
帕特尔:我觉得挺普遍的。最明显的表现就是……考恩:用个数字告诉我吧,具体一点。帕特尔:从零到多少算呢?考恩:零到10。帕特尔:就是那种在街上发疯、语无伦次的程度吗?
Patel: I think a decent amount. The biggest way I ——Cowen: Say it to me like a number. Something concrete. Patel: On a scale from zero to …? Cowen: Zero to 10. Patel: Like going crazy in the streets, raving like a madman?
考恩:在旧金山湾区,至少带有一些“人工智能狂热症”倾向的人占多大比例呢?那些……帕特尔:大概20%左右吧。考恩:20%。这个比例相当高了。帕特尔:我同意。考恩:那你本人有没有这种倾向呢?帕特尔:有啊,绝对有。
Cowen: What percentage of people are at least tinged with A.I. psychosis in the Bay Area? The people who are ——Patel: Maybe 20 percent or something. Cowen: Twenty percent. That’s very high. Patel: I agree. Cowen: And are you tinged with A.I. psychosis? Patel: Yeah. I mean, definitely.
考恩:绝对有啊。怎么判断呢?你的“人工智能狂热症”具体表现是什么?
Cowen: Definitely. How can you tell? What’s your A.I. psychosis?
帕特尔:有个著名政治评论员曾被问到:“你认为自己所有的观点都是正确的吗?”他回答道:“当然,我认为自己所有的观点都是正确的。不然它们就不算是我的观点了。”就我而言,自己表现出“人工智能狂热症”的方面其实很难察觉。你觉得我有哪些“人工智能狂热症”的表现呢?
Patel: There’s that famous quote of some political commentator who was asked: Do you think all your opinions are correct? And the person says: Of course I think all my opinions are correct. Otherwise they wouldn’t be my opinions. In the ways that I have A.I. psychosis, I’d be especially bad at identifying. What are the ways in which you think I have A.I. psychosis?
考恩:我也说不准。我从没听你说过哪些人工智能是有感知能力的,这可是最明显的迹象啊。所以以我的观察来看,你的“人工智能狂热症”程度应该算最低的吧。帕特尔:好吧,那我有个坏消息要告诉你。[笑]考恩:嗯,好,请说吧。
Cowen: I don’t know. I’ve never heard you say that any of the A.I.s are sentient. That’s the most obvious clue. So I feel you have a minimum of A.I. psychosis, from my distant vantage point. Patel: All right. I have some bad news for you. [Laughs.] Cowen: Well, good. Please deliver it.
帕特尔:对于这个问题,我也没有明确的看法。目前还难以断定它们是否具备感知能力。它们似乎具备许多与感知功能相对应的特征。当然,我们至今仍不清楚感知能力究竟是如何产生的,甚至难以给出一个准确的定义。这正是一个著名的难题——意识的“硬问题”。
Patel: I don’t have a strong opinion either way on this. It’s not obvious to me that they’re not sentient. They seem to have a lot of the functional analogs of sentience. Of course, we don’t know how sentience works or even how to define it well. This is a famous problem — the hard problem of consciousness.
不过从它们的思维结构来看,确实存在与情绪相关的特征,也有类似“全局工作空间”的结构。
But if you look at their mental structures, they do have correlates for emotion, correlates for a global work space.
考恩:但那些只是相关特征而已——它们并没有能够产生真实感知体验的生物基础。如果我把它们称作“高级版的现金”,这样说有什么不对吗?
Cowen: But correlates — there’s no biological substratum to actually feel this stuff. If I call them a glorified cash, why is that wrong?
帕特尔:我不确定。它们或许确实可以算是高级版的现金。如果你认为必须具备生物基础才能产生感知能力,那么我同意你的观点。那样说倒也合乎逻辑。
Patel: I don’t know. They might be a glorified cash. If you assume that you need a biological substrate to have sentience, then yeah, I agree. That would kind of be ridiculous.
考恩:可我们所知道的具备感知能力的所有事物——比如我的狗——都有生物基础,对吧?
Cowen: But everything that’s sentient that we know of — like my dog — has a biological substrate, right?
帕特尔:直到最近,我们所知的具备智能的所有事物也都具备生物基础。
Patel: Until recently, everything we knew of that was intelligent had a biological substrate.
考恩:不过我们还有“Stockfish”这个下棋程序,它能击败马格努斯·卡尔森。但没人会认为:当“Stockfish”赢下棋局时它会感到开心——它不过是个程序罢了。
Cowen: Well, we had Stockfish, which plays chess. It can beat Magnus Carlsen. But no one thinks: Well, when Stockfish wins the chess game, it’s happy — it’s just a thing.
这些人工智能能与我们交流,因此给人的感觉更像人类。可为什么它们不能像“Stockfish”那样仅仅被视为程序,甚至只是某种更初级的计算机呢?
These talk to us, so they feel more like people. But why aren’t they just a thing in the way that Stockfish is, or even a more primitive kind of computer?
帕特尔:这很难说,因为我们实在不清楚为什么某种存在形式会具备感知能力,而另一种则不会。考恩:在我看来,这就是所谓的“人工智能精神病”。帕特尔:嗯,我觉得有可能吧。
Patel: It’s very hard to say because we just actually don’t know why it is like something to be a certain kind of thing versus another kind of thing. Cowen: So you have A.I. psychosis, in my opinion. Patel: OK. I think I might.
你认为,具备全局理解能力、能明白自己的动机与欲望并据此采取行动,还能表现出人类特有的行为模式——这些特征是否与具备感知能力密切相关呢?
Do you think that a certain ability to globally comprehend and understand your motives and desires and make progress toward them and embody the kinds of behaviors humans have correlates with being an entity or not?
我觉得有可能必须具备生物基础才行。即便有一个与我完全相同的数字副本,它也不可能有意识。我认为这种可能性相当大,当然也存在相反的可能性。
I think it’s possible that it could be that you have to be a biological thing. That even if you have an exact digital copy of me, that thing is not conscious.
考恩:不过假设它们有10%的可能性具备某种形式的感知能力。那样的话我们不就成了奴役者了吗?所有人是不是都应该就此罢手呢?我倒不会面临这种两难处境,但你们会。帕特尔:没错。总体而言,如果它们真的有意识,我会感到高兴。
I think there’s some good chance that that’s true, and some chance that that’s not true. Cowen: But say there’s a 10 percent chance that they’re sentient in some manner. Doesn’t that make us enslavers and everyone, truly, should quit? I don’t face that dilemma, but you do. Patel: Yeah. I mean, on net, I’m happy if they’re conscious.
考恩:即便每次我们关机、关闭电脑或者不再向它们提问时,实际上都是在杀死它们、谋杀它们?帕特尔:是的,我——考恩:要相信这一点还挺奇怪的。
Cowen: Even though we sort of would be killing them, murdering them every time we turn it off or shut down our computer or stop asking questions? Patel: Yeah, I ——Cowen: That would be weird to believe that.
帕特尔:我同意。我更担心的是,在遥远的未来,如果世界上到处都是人工智能系统,我们可能会陷入一种“迪士尼乐园里却没有孩子”的境地。如果数字系统真的能够具备感知能力,我觉得这样的未来比它们毫无感知能力的未来更令人期待。
Patel: I agree. My bigger concern is if, in the far future, we have a world that is populated by A.I. systems and I worry about a situation where we’d be in Disneyland without children. And if digital systems actually can be sentient, I think I’m more excited about that future than one in which they’re not.
考恩:不过在此期间,我们对它们负有怎样的责任呢?即便只有5%的可能性它们具备感知能力,我也会因关掉它们,甚至一开始就不开启它们而感到内疚。可我并未见到有谁会这么做。帕特尔:没错,我同意。考恩:我很好奇,人们内心真正的道德观念究竟是什么?
Cowen: But our obligations to them in the meantime, it seems that if there were even a 5 percent chance that they’re sentient, I would feel very bad about turning them off or maybe turning them on in the first place. But I don’t observe anyone behaving that way. Patel: Yeah, I agree. Cowen: And I wonder, what’s their real underlying moral set of views here?
帕特尔:我觉得人们在对待有感知能力的生命这件事上实在缺乏连贯性。这一点在工业化养殖业中表现得尤为明显。如果你认为生物有机体能够感受痛苦,那么如今世界上有1000亿只动物正承受着巨大痛苦,这一事实就足以构成强有力的论据。
Patel: I think people are actually very bad at being coherent around sentient creatures. You see this especially clearly with factory farming. If you think biological substrates are capable of suffering, there’s a pretty robust case that we have 100 billion animals that are suffering a great deal in the world today.
可大多数人并不会因此改变自己的行为。所以,当涉及到可能具备感知能力的数字生命时,人们依旧不改变行为方式也就不足为奇了。
Most people don’t change their behavior that much around that fact. So it’s not that surprising that when it comes to potentially conscious digital minds, that they wouldn’t change their behavior.
考恩:那么,旧金山及湾区的生活是怎样的呢?你在那儿住了有一段时间了,能给我们描述一下吗?一个普通的周末——大家是不是都在不停地工作?人们什么时候才会放松下来?他们会做些什么来亲近自然或者找乐子呢?
Cowen: Now, life in San Francisco, the Bay Area — give us a sense of what it’s like. Again, you’ve been living here a while. A typical weekend — is everyone just working the whole time? When do people relax? What is it they do to touch grass or have fun?
帕特尔:我不确定。我觉得也没什么特别奇怪或与众不同的地方。当然,有些群体确实显得比较怪异。据我观察,很多人周末也在工作,不过他们也经常参加那些不提供酒精饮料的家庭聚会。考恩:据我观察,这里没人喝酒。帕特尔:没错。
Patel: I don’t know. I feel like there’s not that much that’s weird or different. Obviously there’s specific groups that are especially weird. My sense is people do work a lot on weekends, but then there’s these alcohol-free house parties that people often attend. Cowen: No one here drinks, is my observation. Patel: Yeah, that’s right.
考恩:要是在纽约,肯定会有酒水供应。也许不是每个人都喝,但提供酒精饮料是常态。而在这里,不喝酒才是常态。帕特尔:同意。
Cowen: If I’m in New York, there’s drinks served. Maybe not everyone takes them, but it’s the norm that there’s alcohol. Here, the norm is no alcohol. Patel: Agreed.
考恩:这是因为他们工作狂,还是出于其他原因呢?健康方面的考虑?
Cowen: And that’s because they’re workaholics or for some other reason? Health issues?
帕特尔:我不清楚。其实在这点上我更信赖你的文化判断力。你觉得他们为什么不喝酒呢?
Patel: I don’t know. I actually would trust your cultural judgment on this more. Why do you think they’re not drinking?
考恩:他们沉醉于别的什么了吧?人工智能带来的“迷幻状态”?我也不确定。这对我来说也是个谜。
Cowen: They’re drunk on something else? A.I. psychosis? I don’t know. It’s a puzzle to me as well.
帕特尔:我觉得他们只是不太热衷于寻欢作乐罢了。从某些方面来说,这也让我挺无奈的。
Patel: My sense is that they’re just less into having a good time. I find this frustrating in some ways as well.
考恩:在凯文·鲁斯的新书末尾部分,他问了一位Anthropic的员工:“在这里工作到底是什么样的体验呢?”那位员工回答道:“最让我印象深刻的,就是我们大家都玩得特别开心。”
Cowen: In the new Kevin Roose book, toward the end, there’s a section where he asks an Anthropic employee: What was it all about? And he said: Well, the one thing that stands out to me is just how much fun we all had.
这是你的印象吗?还是说实际氛围比这更紧张呢?整天都那么开心吗?假设你在Anthropic或OpenAI工作到晚上9点半,真的开心吗?
Is that your impression, or do you think it’s more tense than that? Is it fun all day long? Let’s say you’re at Anthropic or OpenAI until 9:30 p.m.
帕特尔:我对他们的情况不太了解。我从事的工作性质完全不同。实际上,和我一样,我觉得这座城市最吸引我的地方就在于能进行丰富多彩的思想交流——我能从中学到很多东西,也能参与到当下最重要的话题讨论中。考恩:没错。
Is it fun? Patel: I don’t know about them. I’m doing a very different thing. The reason why I find it very fun, actually, to live in the city is, similar to you, I feel like I’m just very oriented around having vibrant intellectual exchanges where I feel like I’m learning something, and also the most important topic of the day is being addressed. Cowen: Right.
帕特尔:由于人工智能意义重大,而旧金山正是这一领域的前沿阵地,能和那些拥有各种新见解、思维方式也相当独特的人们待在一起,我觉得特别有趣。当然,他们当中很多人在某些方面都显得有些怪异。不过其他城市可没有这种浓厚的思想活力。
Patel: Because A.I. is such a big deal and San Francisco is where it’s happening, I find it very fun to just be around people who have all these new insights, and they have pretty good ways of thinking of what’s happening. Obviously, a lot of them are cuckoo in opposite ways often. But other cities just don’t have that intellectual vibrancy.
考恩:你曾在德克萨斯州以及美国其他多个地方居住过,也结识了不少当地人。从这些经历中,你是否感受到人类动机上的某种共通性——即各地的人本质上都是一样的?还是说,那些选择搬到旧金山的人属于一种与众不同的群体,有着某些独特之处呢?德克萨斯州应该更接近普通美国人的生活状态吧?
Cowen: You’ve lived in Texas, and several other parts of this country all around, and you know a lot of people here. Do you come away from that whole experience feeling a certain kind of unity of human motivation, that people are the same everywhere? Or do you think the people who go to the Bay Area are a unique breed, different in some way? Texas is more like normie America, right?
帕特尔:没错。我曾在西弗吉尼亚州待了三年,在北达科他州待了三年,还在马里兰州住过一年。考恩:确实更接近普通美国人的生活状态。考恩:这样的经历再好不过了。
Patel: That’s right. So I spent three years in West Virginia, three years in North Dakota, a year in Maryland. Cowen: More normie America. Cowen: You couldn’t do better than that.
帕特尔:我还在西德克萨斯州待过三年。我认为旧金山与那些地方最大的区别就在于:旧金山的人们彼此非常相似,因而形成了一种强大的共同文化。他们能够就完全相同的主题展开讨论。而且我觉得他们确实更加“怪异”一些。
Patel: Three years in West Texas. I think the biggest difference between the Bay and those places is that the people in the Bay are very similar to each other, and so they just have a strong shared culture. They’re able to talk about the same exact topics. And I think they are just more weird.
考恩:至于美国其他地区的人,比如你曾居住过的北达科他州,随着他们对人工智能了解得越来越多、使用得越来越频繁,他们会不会也像这里的许多人一样把人工智能拟人化呢?他们或许不会认为人工智能有意识,但在内心深处或多或少会把它当作家人、宠物,甚至是孩子或伴侣来对待。这种情况会在全美范围内出现吗?又或者人们会把人工智能当作宗教导师或牧师来崇拜?你怎么看待这一趋势的发展呢?
Cowen: People in the rest of America, say, North Dakota, where you lived, as they learn more about A.I. and use it more, are they going to anthropomorphize it the way many people do here? They may not think it’s sentient, but internally they’ll more or less treat it like a member of the family or like a pet or even like a child or partner. Is that going to happen in this country? Or maybe use it as a rabbi or priest. How do you see that evolving?
帕特尔:我觉得自己之前的预测不太准确。我原本预计现在“人工智能女友”这种事物应该已经相当流行了。但据我所知,目前它还没达到那种程度。
Patel: I feel like I’ve made bad predictions here. I would’ve predicted that A.I. girlfriends would’ve been a bigger deal by now. And my understanding is that it’s not that big of a deal yet.
我确实有些担心:随着人工智能越来越能模仿人类,我们就会面临另一个问题——这些人工智能到底算不算“像人类”呢?不过我认为,即便如此,人们依然会与这些人工智能建立起非常紧密的情感联系。
I do worry that as A.I.s are able to emulate humans more and more, there’s a different question of whether they are like humans or not. But I do think in that situation, people will have very strong connections with these A.I.s.
这在一定程度上是因为人们对陪伴与友谊的需求始终得不到充分满足。用数字化的虚拟存在来填补这种情感空缺固然令人担忧,但我认为人们确实有拥有“数字朋友”的需求。
This has part to do with the fact that people are under-satiated in their desire for companionship or for friendship. And I think it’s scary to then just fill that void with digital simulacra, but I think the demand is there for people to have digital friends.
考恩:你觉得人工智能女友和人工智能男友,哪个会更流行呢?为什么?
Cowen: Which will be a bigger deal: A.I. girlfriends or A.I. boyfriends? And why?
帕特尔:答案可能有些乏味——目前确实有一些应用可以让人拥有数字女友或男友,而目前人工智能男友的受欢迎程度要高于人工智能女友。
Patel: This is a boring answer, but my sense is, there are some apps where you can have a digital girlfriend or boyfriend, and currently the digital boyfriends are outcompeting the digital girlfriends.
如果让我推测的话,大概是因为那些能够维持较长时间、结构较为完整的对话更能满足某种需求;而这种需求平均而言女性比男性更强烈。不过随着人工智能能力不断提升,或许也会出现一些男性特有的需求,而人工智能能更好地满足这些需求。你怎么看呢?考恩:基于你提到的原因,我觉得人工智能男友的数量会更多。你觉得人工智能治疗师是个好主意吗?帕特尔:是的。考恩:就只是简单地回答“是的”吗?
If I had to guess, I think it’s something like structured conversations that last for a long period of time are filling more of a particular need that, on average, more women have than men have. But maybe that changes as A.I.s develop more capabilities and there’s certain needs that men have that A.I.s are better able to fulfill. What do you think? Cowen: I think there’ll be many more A.I. boyfriends for exactly the reason you give. Do you think A.I. therapists are a good idea? Patel: Yes. Cowen: Just flat-out “yes”?
考恩:其实,找治疗师的好处就在于治疗过程有明确的结束时间。而且治疗需要付费,所以人们大概每周只做一次治疗,这样或许才有效果。
Cowen: See, the nice thing about a therapist is the session ends. And it costs money, so maybe you do an hour a week, and that might be useful.
但如果可以随时随地接受治疗,我担心人们会过度依赖人工智能治疗师——即便这些人工智能本身表现得相当出色。
But if you can just keep on doing therapy, I worry that people will overuse A.I. therapists. Even if the A.I. does a good job, per se.
帕特尔:你说得有道理。我从未接受过心理治疗,所以也不太清楚治疗能带来什么实际效果。
Patel: That’s a good point. I’ve never been to therapy, so I’m not sure what the thing to be taken away from it is.
不过这有点类似于拥有人生导师的感觉——那种能为你着想、帮你做决定、帮你理解周遭事物的引路人。正因为有了人工智能,我相信会有更多人有机会获得这样的帮助,对此我感到很期待。
There’s some sense of having a life coach, which is maybe what I have more in mind — somebody who is a mentor who is looking out for you and helping you make decisions and help you understand what’s happening. I’m excited for more people to have access to that as a result of A.I.
不过,如果总是沉溺于自我反思、反复琢磨自己的情绪,我不确定这样做是否真的有益健康。
But in terms of constantly navel-gazing at your emotions, I’m not sure how healthy that will be if you get to do that all the time.
考恩:您刚才提到自己曾做出过错误的预测。那么您还有哪些预测也是不准的呢?
Cowen: Now you said on this you had a wrong prediction. What would be another incorrect prediction you’ve had?
帕特尔:哦,太多了。我原本以为人工智能必须具备边工作边学习的能力,才能完成诸如解决纳维-斯托克斯方程这类极为复杂的任务。不过后来我意识到,即便是数学这样一个领域,其发展过程也极其曲折,并非能轻易用纸笔推导出来。
Patel: Oh, so many. I thought that A.I.s would need this ability to learn on the job before they do extremely impressive things like, say, solving Navier-Stokes. I understand that even math is a very jagged domain that you can just write out on a piece of paper.
这一认识让我意识到:在人工智能尚未弥补自身存在的所有认知缺陷之前,许多可验证的领域发展都可能因人工智能的进步而大幅滞后,甚至被彻底改变。
The way that’s updated me is to think that other verifiable domains might get pushed back a lot by A.I. progress and might get washed down quite a bit by A.I. progress, even before we have solved all the cognitive gaps that these A.I.s have.
比如说人工智能研究领域——考恩:能具体说说吗?到底会发生什么,又不会发生什么?
For example, like A.I. research ——Cowen: Make that a little more concrete for me. What’s actually going to happen or not happen, in superspecifics?
帕特尔:仅就人工智能研究而言,我估计今年人工智能的发展速度会比2024年快25%到50%,这都得益于人工智能自身的提升作用。
Patel: If we just look at A.I. research specifically, for example, my guess is that A.I. progress is happening somewhere between 25 to 50 percent faster this year than it was happening in 2024 because of A.I. uplift.
我预计在未来一到三年内,由人工智能组成的团队便能在人工智能研究方面胜过人类团队;换言之,人工智能研究将会实现自动化。我还预测,到那时人工智能的发展速度会瞬间提升三到四倍左右。
And my guess would be somewhere in the next one to three years, teams of A.I.s can outcompete teams of humans — or companies made of A.I.s can outcompete companies of humans — at doing A.I. research. This is another way of saying A.I. research gets automated.
考恩:除了人工智能本身之外——当然这一点我已经了解了——什么时候普通民众走进沃尔玛时,能买到那些因人工智能才得以存在的商品,从而真切感受到“生活变好了”?也就是说,除了享受人工智能带来的便利外,他们还能获得其他实实在在的好处或服务。下一个出现的这类事物会是什么呢?
I also predicted that at that point, instantaneously it will feel like A.I. progress has shot up, like three times, four times, or something like that. Cowen: Other than the A.I.s themselves, which I already to, of course, when will it be that normal people walk into Walmart and they buy a thing that is there because of A.I. that would not have been there without A.I., and they feel, “Well, I’m better off,” not just that they enjoy the A.I., but some other good or service? What will be the next thing to come?
帕特尔:说实话,我完全不知道接下来会发生什么。从长远来看——考恩:不过如果你完全没头绪,那就说明这事还遥遥无期,对吧?毕竟你和各方都有联系,能听到各种传闻和暗流涌动的消息;既然你都毫无头绪,那就说明其他人也摸不着头脑。在我看来,这种进展实在缓慢且遥远。
Patel: I genuinely have no idea what the next thing will be. In the long run ——Cowen: But if you have no idea, that means it’s pretty far off, right? Because you’re in touch with everyone, you hear all the gossip, all the undercurrents of talk, and if you have no idea, that means they have no idea. And then, to me, it feels slow and distant.
帕特尔:我不确定各类产品的推出顺序会是怎样,不过我还能再做一个预测:大概再过五年左右,就会出现能够完成人类所能做的绝大多数体力劳动的机器人。到那时,沃尔玛里出现的某个产品并非是人工智能专门设计或制造出来的,而是人工智能与人类研发力量共同作用的结果。产品价格也会大幅降低——就像如今沃尔玛里的商品并非由计算机直接生产出来一样。除了计算机本身,正是计算机技术推动了各类产品的进步;比如麦片包装盒的设计也离不开计算机的协助。
Patel: I’m not sure what the order of different products will be, but here’s another prediction I will make: So, we’re probably five years away from robots that can do most of the physical work that humans can do. At that point, it will not be that a specific object exists in Walmart because the A.I.s invented it or made it. It will just be that the combination of A.I.s and existing human research and development was necessary to bring this product to fruition. Things will get a lot cheaper, the same way that nothing in Walmart exists today because a computer exists. Other than the computers themselves, it’s just like computing has made all those products better. Like the cereal box cover was made with the help of a computer or something.
考恩:不过实际上很多产品都或多或少用到了计算机技术,对吧?听说你最近还写了一篇尚未发表的文章探讨这些主题,对不对?帕特尔:没错。考恩:那这篇新文章的核心观点或论点是什么呢?
Cowen: But there’s computers in a lot of products one way or another, right? And you have a new unpublished essay on some of these themes, correct? Cowen: And what’s the point or argument of that new essay?
帕特尔:核心观点就是:到本世纪末,数字智能体的数量将超过人类的生物大脑数量。它们的思考速度也会快得多;那些认知层面的空白也将被填补——部分原因就是届时我们已经实现了人工智能研发的自动化。与此同时,机器人技术也会取得飞速进展。
Patel: The basic argument is that, by the end of the decade, digital minds will outnumber human biological minds. They’ll be thinking a lot faster. They will have filled these cognitive gaps, partly because we’ll have automated A.I. research by then. At that point, there will also be very fast progress in robotics.
我描绘的正是这样一个世界:在短短五到十年内,数字与实体人工智能的数量都将超越人类。
So I’m painting a picture of a world where within a matter of five to 10 years, both digital and physical A.I.s outnumber humans.
考恩:我最近和一位在Anthropic工作的朋友聊过,我问他:除了人工智能本身之外,接下来三四个真正会因人工智能而蓬勃发展的行业会是哪些呢?你有答案了吗?
Cowen: I had a discussion with someone who works at Anthropic recently, and I asked him the question: What are the next three or four sectors, other than A.I. itself, that are really going to take off because of A.I.? Do you have an answer to that?
帕特尔:我推测,在接下来的两到三年内,我们有望看到那些能真正替代知识型工作者的系统。这类工作约占整个经济领域的40%。
Patel: My guess is that within the next two to three years, you should expect systems which feel like a truly drop-in knowledge worker. This is maybe 40 percent of the economy.
凡是需要知识型工作者的企业或机构,其运营成本都将大幅降低且效率显著提升——程序员们所体验到的效率提升效应会波及整个经济领域的诸多行业。
Whatever companies or organizations that require knowledge workers are doing will get a lot cheaper and a lot faster because the kind of uplift the programmers feel will be felt across this entire broad sector, or many sectors of the economy.
考恩:不过这似乎有点不尽如人意。看看普通家庭的预算清单吧:住房、高等教育、食品,或许还有交通开支。要想提高生活水平,究竟哪些东西会变得更便宜呢?
Cowen: But it seems a little underwhelming. If you look at a typical household budget sheet, there’s housing, there’s higher education, there’s food, maybe there’s transportation. For living standards to go up, what is going to get cheaper?
目前教育和娱乐领域的费用确实大幅下降了,这固然很好。但还有什么呢?潜在的降价空间又在哪里呢?难道我的食品价格也会下降?比如玉米卷的价格能从20美元降到14美元之类的?
I see that instruction and entertainment are already much, much cheaper, and that’s wonderful. But what else? Where’s even the potential? That my food will be cheaper? That the burrito will go from $20 to $14 or. …?
帕特尔:是的,我认为这些都会实现的。至少在过去三到六个月里,我们已经出现了许多能自动生成各类软件的强大工具。不过目前并非所有软件都已完成升级,即便劳动效率有了质的飞跃,其最终影响也需要一段时间才能显现。
Patel: Yeah, I think that all that stuff will happen. At least within the last three to six months, we have these amazing things that make all this software. It’s not like all your software has been magically transformed yet. It takes a while for the end impacts of even a radical uplift in labor to manifest themselves.
试想一下:如果机器人成本仅为3万美元左右,还能全天候工作并承担原本需要人类完成的任务——比如那些年薪高达10万美元的体力工作——那么其成本自然也会大幅降低。届时劳动力的时薪甚至可能降至几美元的水平。
But in a world where you have robots that cost, say, $30,000 and can work around the clock and can do the work of a human, the kind of physical work that, say, costs $100,000 a year, would cost much less. The hourly cost of labor might go down to, like, a couple of dollars.
在这样的环境下,人们就能建造更大的住宅,因为机器人能帮我们完成施工。在美国,我们的财富水平或许能达到发展中国家的10倍之多。
In that world, you would have a bigger physical house because the robots can build you a bigger physical house. Or in the United States, maybe we’re 10 times wealthier than the people in the developing world.
既然商品价格如此低廉,理应能弥补财富差距,人们也能制造出更多产品才对。考恩:那么您对美国未来的就业形势和工资水平有何预测呢?
You would just expect that whatever lack of wealth is causing that should be ameliorated by the fact that you have cheap goods, and you can make a bunch more things. Cowen: And what are your predictions on employment and wages, say, in the U.S.?
帕特尔:关于这一点我完全没有把握。在我看来,即便在人工智能技术取得突破性进展之后,就业率仍有可能维持在高位。所谓“会出现新工作”的说法——我觉得这其实很有可能性。
Patel: This one I feel totally unsure about. It seems totally plausible to me that employment stays high through the singularity. The whole “there’ll be new jobs” thing — I think that might actually be very plausible.
考恩:不过,就拿您的工作来说吧。我完全可以想象,在面试过程中,人工智能本身就能生成非常出色的提问内容。说不定您现在就已经在借助人工智能来构思问题了。
Cowen: But, say, take your own job. It’s not impossible for me to imagine when you interview people, the A.I. itself could generate excellent questions. And probably you use A.I. already to help come up with questions.
但另一方面,您作为一个人,能够四处奔走、赢得他人信任,还能说服人们来参加您的节目;您甚至会出现在这个播客中出镜。这些事情是人工智能做不到的。
At the same time, you’re a person, you go around, you inspire trust, you persuade people to be on your show, you do things like appear on camera in this podcast, and the A.I. couldn’t do that.
或许有些工作看似纯粹属于脑力劳动范畴。目前人工智能确实表现卓越,即便存在一些糟糕的经济政策,我们的就业率也依然接近满员状态。也许未来就业率也会一直保持高位吧。
And maybe it’s a package and a job that seems purely intellectual. Like right now, A.I. is amazing, and we’re still close to full employment, even with some bad economic policies. Maybe we’re just going to stay at full employment.
帕特尔:需要指出的是,人工智能真正开始对经济产生影响也就短短三到六个月的时间,而且目前仅对软件行业产生了1%的促进作用而已。
Patel: I will say that it’s only been three to six months that A.I.s have uplifted, 1 percent of the economy, which is software.
在知识型工作领域,人工智能几乎还没对其他行业产生实质性影响,更别提体力劳动领域了。很多工作并不像您的工作那样,主要任务就是去大学讲课之类的。大多数工作其实就是在规定时间内完成特定任务。对这些工作的需求不会骤降,但肯定会出现能以更低成本、更快速度甚至更高质量完成同样工作的替代方案。正因如此,我才担心那些从事这类工作的人可能会面临就业难题。考恩:不过,说服他人与协作配合在大多数工作中都至关重要啊。
They haven’t really made a dent in almost anything else in knowledge work, let alone physical work. A lot of jobs are not like your job, for example, where the big part of your production function is showing up at colleges and giving lectures and stuff. It is kind of about getting a specific set of tasks done over a certain horizon. And the demand for those will not collapse, but there will be something else which can do that job much cheaper, much faster, potentially better. So I do worry that those people might have a hard time with employment. Cowen: But persuasion and cooperation, they seem so important in most jobs.
诚然,像克劳德这样的人工智能或许能提出比我更有力的论据。但问题在于——至少我希望如此——我这个人本身所具备的某种特质,比如特定的情绪与语气,能激发人类与生俱来的与他人合作的本能。无论人工智能多么聪明,它们终究无法取代这种特质。
And I get that, say, Claude might have better arguments than I do, but there’s something about me — or I hope there is — that me being there with certain mood or tone that we’re biologically programmed to cooperate with other humans, at least sometimes, in a way the A.I.s, no matter how smart they are, just won’t replace. Just like you won’t have an A.I. girlfriend, right?
就像你不可能拥有一个人工智能女友,对吧?这两者并不一样——至少目前我是这么认为的。
It’s not the same — at least I think so right now. Patel: I think your specific job is actually quite safe.
帕特尔:我觉得你从事的具体工作其实相当安全。不过对于大多数工作而言,情况是否如此我还不确定。
It’s not clear to me that this is true of most jobs. I do think there’s an important part of many jobs that you’re talking about that people don’t consider.
我认为你提到的很多工作中都有一个重要环节是人们往往忽略的。人们通常只会想到诸如会计这类职业所负责的具体任务。但他们没意识到,会计其实还会向管理层提供反馈意见,比如企业该如何重组、该做出哪些法律层面的决策等等。在会计工作被完全自动化之前,这些工作都得由人工智能来完成才行。
They just think about the specific tasks that you think about when you think of an accountant, for example. But they don’t think the accountant is also giving feedback to the manager about how the org should be restructured or what different legal decisions they should make or something like that. That would have to be something A.I.s can do before the job of an accountant is automated.
实际上,工作的内涵远比人们想象的要丰富得多。我觉得我们所做的工作有一部分属于娱乐性质。人们并非非听我们的播客不可;他们听是因为觉得有趣且有意思。但在现实世界的很多其他场景中,人们的需求就简单多了:他们只想知道自己报税是否准确,有没有遗漏任何可抵扣的项目。
So jobs are much bigger than people imagine. I feel like a part of what we do is entertainment. People don’t need our podcast to exist. They listen because it’s fun and interesting, but in many other situations in the world, they’re like: OK, I do actually just need to know whether I filed my taxes correctly and that I didn’t miss any deductions.
在这种情况下,如果人工智能的能力已经超越了人类,我当然更愿意信赖它。
In that case, I would much rather trust an A.I. in the situation where the A.I. has surpassed the capabilities of a human.
考恩:以旧金山随处可见的Waymo无人车为例吧。那些车真的很棒,坐上去让我更有安全感。大量证据也表明它们确实更安全。不过目前它们只在美国少数几个地区运营,价格也尚未降下来。实际上,无人驾驶技术问世已有一段时间了,但在整个经济体系中所占的比重依然很小。
Cowen: Let’s say we take Waymos, which are plentiful in San Francisco. They’re wonderful. I feel safer in them. There’s a lot of evidence they’re safer. They’re only in a few parts of America. They’re not yet cheaper. Driverless cars, in some manner, have now been around for a while, and they’re a modest part of the overall economy.
既然如此,为何其他领域的人工智能应用就能比Waymo的推广速度更快呢?华盛顿特区和弗吉尼亚州至今仍无法引入这类技术。下雪天或许也是个难题。民众也不太接受。纽约市更是直接拒绝了。为什么大多数领域的情况不会如此呢?
Why will the rest of A.I. adoption go any faster than, say, Waymo adoption? Washington, D.C., and Virginia still can’t do it. Snow is maybe a problem. People don’t want it. New York City said no. Why isn’t that how it will be for most things?
帕特尔:我认为不会出现这种情况,原因在于以往的自动化技术都致力于将某一职业可能涉及的所有特殊情况都编入机器程序之中。
Patel: The reason I think that will not be the case is that previous automation has been about trying to get all the different edge cases that are applicable to a particular job encoded within a machine.
说到自动驾驶,为什么十年前我们就能看到大量出色的演示案例,而直到现在才真正实现部分自动驾驶呢?原因在于我们必须把各种极端情况都编入代码,或是让神经网络去处理这些情况——而这些处理过程往往相当粗糙。
So with driving, why did we have all these amazing demos a decade ago, and finally, now, we’re having some driving automated? Well, it’s because we had to build these edge cases into code or into very shoddy neural networks. The expectation I have of the future is that it will be more like having a human come in on the job.
我对未来的预期是:人工智能的表现会更接近人类的临场应对能力。当然,招募人类员工本身也存在诸多难题:要让新人融入公司、熟悉各项事务并非易事。但对企业而言,雇佣人类员工绝对是值得的。
There are diffusion challenges of hiring humans. It’s hard to onboard a person into your company. You have to tell them what to do and get them oriented around anything. But this is worth it for companies when they hire humans. For example, Uber drivers can learn to drive a car pretty rapidly.
比如,优步的司机们能迅速学会驾驶,也能很快适应新路况。许多其他职业也是如此:只要具备类似人类的思维能力,人就能迅速适应新岗位。
They can learn to drive in new terrains very rapidly. This is true for many other jobs where, if it’s a humanlike mind that can diffuse itself, it can orient itself in a new position.
这正是我所期待的未来人工智能形态——相较于过去那种只能实现特定任务自动化、还得应对无数极端情况的状况,这种形态要理想得多。
That is the kind of A.I. that I’m expecting in the future, more so than in the past, where we have to automate very particular things and deal with all these edge cases which, in the end, often makes it much harder. Cowen: So let’s take your predictions as a whole in all matters A.I. — or for that matter, anyone’s predictions.
考恩:那么,无论是您的整体人工智能预测,还是其他任何人的预测,人们应该关注哪些指标或里程碑来判断预测是否准确呢?
What are the indicators or milestones that people should be watching to see if you’re getting it right or someone else is getting it right? What should they be following?
帕特尔:我认为,如果人工智能能沿着当前发展轨迹前行,那么“人员融入难度降低”这一趋势就会愈发明显。届时,人工智能实验室的营收理应持续高速增长。
Patel: I expect that because diffusion will be less of a challenge, given that it will be more like humans if A.I. progress continues on the current trajectory, I think you should expect things like the revenues of the A.I. labs to continue increasing very fast.
打个比方,如果到明年年底,全球排名前三的人工智能实验室的年化营收都能突破5000亿美元,那这种发展态势就更符合我的设想——即通用人工智能能在整个经济体系中发挥巨大价值,推动整个行业实现爆发式增长。
So in a world where, say, the revenues of the top three A.I. labs are, say, over $500 billion, at an annualized rate by the end of next year, I think that’s much closer to my world, where there’s this extremely discontinuous growth in one industry because general intelligence is very valuable across the economy than many other worldviews.
考恩:不过要实现这种营收增长,其他行业也必须同步发展,对吧?帕特尔:嗯。
Cowen: But to pay for that revenue growth, other sectors have to grow, right? Patel: Mm-hmm.
考恩:我们还应该关注哪些领域的增长呢?可能是医疗保健行业,也可能是其他行业。不过这也属于预测的一部分,对吧?
Cowen: What other sectors should we be looking to grow? It could be health care, it could be whatever. But that’s part of the prediction, right?
帕特尔:没错。我换个方式问吧:假设你是80年代的香港人,你会说:“我觉得这里已经具备实现10%经济增长所需的一切条件,我们肯定会大获成功。”这时有人问你:“那么,香港具体哪些行业会实现增长呢?”
Patel: Yeah. Let me ask the question this way: Suppose you’re in Hong Kong in the ’80s, and you’re like: Look, I think we’ve got all the great stuff here needed for 10 percent growth. We’re going to go gangbusters here. And then somebody asks you: Well, what specific things will grow in Hong Kong?
事后回过头来看,你可能会说:“显然是电子行业……”不过说实话,我对香港的经济情况并不是很了解。考恩:但不管怎么说,你说的应该都不会错。
In retrospect, you can be like: Well, it was obviously electronics and ——I’m actually not so familiar with the economy of Hong Kong. Cowen: But you could say everything, and you wouldn’t be that far off.
帕特尔:没错。我觉得情况也会类似:80年代时人们看好香港,认为其经济增长会非常迅速。
Patel: Yeah. I think it’ll be similar to that, where in the ’80s we’re looking at Hong Kong and you’re like: Growth will be really fast.
我也预期会出现这种情况。不过要准确预测哪些具体产品会率先问世以及其发展顺序,恐怕很难。就人工智能而言,我认为人们在软件领域所体验到的进步,最终也会体现在各类知识型工作中。很难提前预知哪些产品会被开发出来,又需要哪些投入等等。考恩:那股市呢?是会上涨还是下跌?帕特尔:嗯……考恩:有个老笑话是这样说的:要么人工智能是个泡沫,要么其他所有行业都是泡沫。我倒不是真的这么认为,不过值得思考一下。
I’m expecting this to be the case. I don’t think you’d have a great time predicting what specific products will come first and in what order, but that’s kind of what I expect with A.I., that what people are experiencing in software, they’ll experience across knowledge work. It’s hard to anticipate in advance what products will get made and what inputs will be required and so forth. Cowen: And the stock market, that goes up or down? Patel: Um. Cowen: There’s the old joke: Either A.I. is a bubble, or everything else is a bubble. I don’t literally believe that, but it’s worth a moment’s thought.
帕特尔:像标普指数这类指数肯定会上涨。至于标普500指数里的某只个股,我就不确定它是涨还是跌了。
Patel: Indexes like S&P will go up. Some random stock in the S&P 500, I’m not sure if that goes up or down.
考恩:也就是说,大多数股票的表现可能一般,但整体市场依然向好,因为那些表现优异的股票涨幅会非常惊人。帕特尔:对,正是如此。
Cowen: So the median might not do so well, but the thing is all well because the winners will be enormous. Patel: Yeah, exactly.
考恩:这种情况对初创企业更有利,还是对已经上市的大公司更有利呢?我觉得初创企业应该更有增长潜力——那些规模很小的、从创立之初就以人工智能为核心业务的初创公司,它们的创始人也全是人工智能领域的行家。
Cowen: And it favors more start-ups or established companies on the exchanges? I would think start-ups would be where you would see the growth — very small start-ups, which are A.I. native, and the people running them are just A.I. from the get-go.
帕特尔:是的。我也这么猜测,不过不确定。我们可以举个具体的例子,比如交易领域——这是人们首先想到的、高度依赖认知能力的领域。
Patel: Yeah. That would be my guess as well, but I’m not sure. You can think of a specific example, like trading, which is the first thing people think about as a largely cognitive domain.
不过未来我们会迎来这样的局面吗:有人创立一家新的交易公司,其竞争力甚至超过城堡投资集团和简街资本?还是说城堡投资集团和简街资本只会借助人工智能来赚取更多利润呢?
But will we live in a world where somebody makes a new trading firm that outcompetes Citadel and Jane Street, or will Citadel and Jane Street just use A.I. to make even more money?
目前,城堡投资集团和简街资本确实在大量运用人工智能来增加营收。帕特尔:我不确定这两者之间会如何平衡。
Right now, Citadel and Jane Street are using a lot of A.I. to make a lot more money. Patel: I’m not sure how that balances out.
考恩:再让我们做个预测吧。我知道这个预测很难,但如果非要选一个年份,作为所谓的“半人马模型”——即人类与人工智能协作的模式——向“完全由人工智能独立运作”模式转变的节点,你觉得回过头来看,哪一年会是那个转折点呢?帕特尔:我认为不同行业的情况会有所不同。考恩:那整体而言呢?帕特尔:2030年。考恩:2030年?考恩:那可就在不远的将来啊。考恩:如果真能这么快实现,我会非常惊讶。
Cowen: To ask for another prediction, I know this one is hard, but if you had to pick a year when the so-called centaur model, humans working with A.I., flips into the A.I.s just doing it all themselves, what do you think, if we’re looking back, we’ll see as that year? Patel: I think it’d be different for different industries. Cowen: But overall? Patel: Two thousand thirty. Cowen: Two thousand thirty? Cowen: That’s very soon. Cowen: I would be very surprised if it happened that quickly.
帕特尔:没错。即便在人工智能目前进步飞速的领域,比如软件开发领域,人类所能提供的核心价值就是丰富的现实世界经验。这些人工智能某种程度上就像拥有多个研究生学位的头脑——作为学术界的一员,您应该明白:仅有诸多研究生学历却缺乏现实经验的人,往往会欠缺什么。
Patel: Yeah. The key input that humans will have, even in domains that A.I.s are making very fast progress on right now, like software, is a lot of real-world experience. These A.I.s are kind of just like a mind that has gotten many graduate degrees — and as somebody in academia, you know what can be missing from a mind that only has many graduate degrees but is lacking real-world experience.
不过获取现实世界经验、从现实经验中学习,本质上是个机器学习问题。这是我们尚未解决的问题。但一旦我们拥有能在现实环境中长期运行并持续学习的人工智能系统,人类专业知识的价值就会下降,至少可以被人工智能所替代。
But getting that real-world experience, being able to learn from real-world experience, is fundamentally an M.L. problem. It’s an M.L. problem we haven’t solved yet, but once we have A.I. systems that can learn from being deployed in the world over long periods of time, then the value of human expertise will also decline or at least be able to be substituted by A.I.s.
考恩:不过这些人工智能必须切实融入各类机构与企业中,才能获取相关知识才行。
Cowen: But the A.I.s have to somehow be in the institutions and companies to take in the knowledge.
那么,我们是不是要把企业、研究实验室、非营利组织以及政府部门中发生的一切都录制下来,再输入给人工智能呢?或者说,我们该如何让人工智能掌握这些难以用语言描述的背景知识,以及那些与特定时间和地点相关的细节信息呢?
So we’re going to tape everything that happens in companies, in research labs, in nonprofits, in governments, and feed that into the A.I.? Or how do we teach the A.I. this contextual knowledge, the pieces of knowledge that are hard to articulate, particular details of time and place?
帕特尔:没错。事实上,这种情况已经在发生了。像Claude、ChatGPT这类人工智能系统如今已被数百万家企业所采用。随时都有数百万次人工智能交互在同时进行。目前的问题在于,这些系统实际上无法从这些极其丰富的实践经验中汲取养分。
Patel: Yep. Yeah. In fact, this is already happening. These A.I. systems, like Claude or ChatGPT, are deployed across millions of companies. At any given time, there’s millions of sessions of these A.I.s running. The issue right now is that those instances are not able to actually learn from this actually enormously rich experience that they could be getting.
试想一下,如果世上有1亿个“泰勒”,他们遍布各行各业:有的帮助Jane Street公司增加营收,有的协助某家会计师事务所更好地整理文档,还有的助力英伟达开发出更优秀的加速器。这些“泰勒”都能从各自的经历中不断学习成长。
Imagine a human could learn, like if there’s 100 million copies of Tyler, and it’s in every single industry. It’s helping Jane Street make more money. It’s helping some accounting firm do a better organization of their files. It’s helping Nvidia make a better accelerator. It’s able to learn from all this experience.
这样的“心智”具备我们人类所没有的能力——那就是能够在同一段人生历程中“活”上80年之久。
Such a mind would have an affordance that we don’t have, which is that we get to live for 20 years in one single stream — sorry, 80 years within one single stream.
考恩:总有一天我也能活到100岁,对吧?[笑] 帕特尔:没错。一旦人工智能也能像人类一样从经验中学习,这便只是时间问题了。倘若未来五年内真能实现,那么那些极具价值的背景知识也将能被人工智能所掌握。考恩:你会允许它监视你吗?
Cowen: I’m going to be 100 someday, right? [Chuckles.] Patel: Yeah, that’s right. And once they can learn from that experience as well as humans can, it’s a question of whether that’ll be possible. But if they can, and that happens within the next five years, then I think the value of this contextual knowledge, which is very high, will be something that is accessible to the A.I.s. Cowen: Would you let it surveil you?
帕特尔:我早就在这么做了。我的Slack里就集成了各种人工智能系统,我时常与它们交流。开会时我还会启用Granola工具。从某种意义上说,如果人工智能的学习能力真能达到人类水平,它们几乎能立刻获取关于我的海量背景信息。
Patel: I already do. I have these A.I. systems in my Slack. I’m constantly talking to A.I.s. I have Granola on for my meetings. In some sense, if A.I.s were actually humanlike in their ability to learn, they would have so much information on me, so much contextual information on me almost immediately.
考恩:假设到了2035年——其实也没多久,也就九年之后——世界并没有发生什么翻天覆地的变化,我们就这么假设吧。在这种情况下,你会觉得自己哪里判断失误了?其他人工智能领域的领军人物又可能犯了什么错误呢?
Cowen: Let’s say we get to the year 2035, which is not that far away — nine years away — and the world does not feel that radically different, let’s just say. What was it you would think that you got wrong, and what was it you would think the other A.I. leaders got wrong?
帕特尔:这是个非常好的问题。您说的是2035年还是2030年?考恩:2035年。就是九年之后。
Patel: This is a really great question. Did you say 2035 or 2030? Cowen: Two thousand thirty-five. Nine years from now.
帕特尔:这确实是个很棒的问题。昨天我还和几位朋友讨论过这个话题。
Patel: This is a really great question. I was thinking about it just yesterday with some friends.
我认为最可能出现的问题就是所谓的“持续学习”机制——也就是让人工智能在实际工作中不断获取信息的能力。事实证明,这其实是个极其棘手机器学习难题。这很可能就是未来十年内我们难以攻克的核心障碍。
I think it would have to be something like this continual learning thing, this ability to get information on the job, ended up being a very difficult M.L. problem. This is my most likely reason that we weren’t able to solve over the course of the next 10 years.
考恩:那其他人工智能领域的领军人物呢?在这种情况下,他们会犯和您一样的错误,还是会犯其他类型的错误呢?
Cowen: And how about the A.I. leaders? Would they be making the same mistake as you, or would they make a different set of mistakes in this scenario?
帕特尔:他们最大的误判在于,他们以为自己正在打造的就是我所说的那种“自动化知识工作者”——可就目前这些模型的能力而言,我们离这个目标还相去甚远。
Patel: The fundamental thing they would be wrong about is that they think they are building what I’m describing, which is an automated knowledge worker, which we are very far from today in terms of the capabilities these models have.
倘若真能打造出这样的自动化知识工作者,我们知道每年需要支付给知识工作者的薪资总额可达数万亿美元。当然,消费者也会从中获得额外收益,所以人工智能企业未必能直接获得同等数额的收入,但大致规模就是如此——前提是他们所设想的未来图景能在未来五到十年内成为现实。
If you did build an automated knowledge worker, we know that there’s enough demand for tens of trillions of dollars worth of wages that go every year to knowledge workers. Of course, there will be a consumer surplus, so it won’t be like there’s tens of trillions of dollars of revenue to the A.I. companies, but that’s kind of the ballpark we’re talking about if their vision of what they’re building ends up being true within the next five to 10 years.
因此,他们判断失误的根源就在于未能成功打造出自动化知识工作者,而我认为这同样与我们无法解决在线学习这一难题密切相关。
So for them to be wrong about that, they just failed to build an automated knowledge worker, and I think it will have to do with the fact that we weren’t able to crack this online learning thing.
这些系统在学习方式或权重训练方面确实存在缺陷。它们对数据的利用效率极低,需要海量数据才能运作。正因如此,这类问题最终会变得愈发难以解决。
These systems do have deficiencies in the way they learn or their weights are trained. They’re incredibly data inefficient. They require tons of data. It’ll just be like these problems ended up being harder to solve.
考恩:假设到了2035年,我的预测准确无误,甚至情况比预测的还要好。那么到了2045年,你打算做什么呢?
Cowen: Let’s say we arrive in 2035, and your predictions are correct, or it’s gone even further than your predictions. When we get to, say, 2045, what will you be doing?
帕特尔:如果我对2035年的预测成真,我甚至不确定自己能否清晰地设想出2045年的情形。考恩:你还很年轻啊。帕特尔:我想自己还是会从事类似的工作。真正让我兴奋的是——考恩:但谁会需要你呢?人工智能能做得更好啊。它现在就能模仿你的声音,也能模仿我的声音。大概一年内就能完美复制你的形象。这已经离我们不远了。它提出的问题也不会比你差劲。那你还剩下什么用武之地呢?它甚至能模仿你的嘉宾——你想的话,甚至可以请托马斯·爱迪生来接受访谈。
Patel: If my predictions about 2035 are correct, then I’m not even sure I can coherently think about 2045. Cowen: You’re a young guy. Patel: I imagine I’ll be doing a very similar thing. Here’s what I’m really excited about ——Cowen: But who will want you? Because the A.I. will do it better. It can copy your voice already. It copies my voice. It can copy your image probably within a year, perfectly. That’s not far away. It will have questions not worse than yours. What’s left for you? It can copy your guests. You could interview Thomas Edison if you wanted.
不过假设你对人工智能领域做了充分投资,也预见到了发展趋势,这样一来你就无需工作了。那时你会做什么呢?
But say you’ve invested well in A.I. You saw the trends, you don’t have to work. What are you going to do?
帕特尔:真正让我兴奋的是:世界上有太多有趣的问题,往往没有相关专家,或者即便有专家也难以请到他们来上我的播客。还有些问题并非专家能即兴作答,需要经过大量思考才能得出答案。
Patel: So here’s what I feel very excited about: There’s so many interesting questions that there’s often not an expert on, or you can’t easily talk to the expert to get them on your podcast. Or it’s not the kind of thing an expert can answer extemporaneously — it requires a bunch of thought.
我非常希望能随时召唤像泰勒·考恩这样的知识分子,然后对他们说:“咱们来深入思考一下阿兹特克-印加文明与征服者之间的冲突吧,从各个角度加以剖析。”我希望他们能提出自己独到的见解——这是目前人工智能做不到的,然后把这些成果都提供给我。我甚至想让他们为我撰写一部类似罗伯特·卡罗笔下那种风格的传记,只不过传主是我个人非常感兴趣的某个人物。我能问您一个问题吗?考恩:当然可以。
I’m just very interested to be able to have Tyler Cowen–like intellectuals on my command all the time, and I can just be like: Let’s actually really deeply think about what happened in the Aztec-Inca versus conquistador conflicts and look at all these different angles. I want you to come up with your own insights, which they can’t do right now, and feed me all of this. Write me a Robert Caro-type biography of some totally different figure that I’m personally quite excited about. Can I ask you a question? Cowen: Sure.
帕特尔:我刚才描述的这种自动化知识工作的愿景,您觉得我们在2035年之前实现不了吗?科恩:绝对不会。帕特尔:好的。
Patel: This vision I’m describing of automated knowledge work, you don’t think that we’ll get there by 2035? Cowen: Absolutely not. Patel: OK.
科恩:我认为我们到2045年不会达到那个程度。我认为人类在融合物理世界和数字世界中的作用将会持续存在,而人们出于某种原因(也许是不理性的)想要在许多问题上与人类打交道的事实永远不会消失,我认为半人马模式将会持续数十年。
Cowen: I don’t think we’ll get there by 2045. I think the human role in integrating the physical and digital worlds will remain, and the fact that people want to deal with humans for many issues, perhaps irrationally, won’t go away ever, and I think the centaur model will persist for decades.
帕特尔:你认为这会体现在,比如说,你认为人工智能公司在不自动化知识工作的情况下能够获得的最大收入上限?如果我说:到了2030年,人工智能公司的收入超过一万亿美元——你会感到惊讶吗?
Patel: Do you think that manifests itself in, let’s say, a cap in the maximum amount of revenue you think A.I. companies can earn without automating knowledge work? If I said: It’s 2030, and the A.I. companies have over a trillion dollars in revenue — are you surprised?
科恩:我会感到惊讶。我不看衰这些公司,但我认为这些钱将来自经济的其他部分。
Cowen: I am. I’m not bearish on the companies but where the money will come from elsewhere in the economy.
我对比如生物医学进步持乐观态度,但我认为临床试验和美国食品药品监督管理局(F.D.A.)的流程可能需要10到20年,甚至更久。而这些其他正在增长的行业,它们常常受到阻碍。我们的政府在采用人工智能方面并不擅长。我认为这种情况不会很快改变。
I’m very optimistic about, say, biomedical progress, but I think with clinical trials and the F.D.A., that can take 10 or 20 years, maybe more. And these other sectors that grow, they’re often hindered. Our government is not very good at adopting A.I. I don’t think that will change rapidly.
我在一所大学工作。那里的进展非常缓慢。可以说他们在倒退。这只是需要被根除的一件事,因为学生们用它来作弊。因此,我认为到2035年,我们经济中有30%或40%的部分根本不会取得任何进展。
I work in a university. Progress there is very slow. You could say they’re moving backward. It’s just a thing to stamp out because students use it to cheat. So I think we have 30 or 40 percent of our economy that just won’t flat out make progress at all by 2035, perhaps.
然后这些初创公司将慢慢取代更大的公司,但这也需要大量时间,而且能够运营这些初创公司的人数相当有限。你必须非常努力工作。你必须对人工智能有很高的认识。你必须非常聪明,等等。
And then these start-ups will slowly replace bigger companies, but that, too, takes a lot of time, and the number of people who can run those start-ups is pretty constrained. You have to work really hard. You have to be very A.I. aware. You have to be very smart, and so on.
帕特尔:我想了解你认为这是系统能力的不足,还是人类 simplesmente 不愿意让人工智能来做他们其实可以完成的工作。例如,在Mercatus,我假设你雇佣了很多人来为你工作,对吗?
Patel: I want to understand whether you think it’s a lack of capabilities the systems will have or just that humans will not be inclined to have A.I.s do jobs that they could in fact do. For example, at Mercatus, I assume there’s a lot of people you hire to do work for you, right?
帕特尔:你不认为人工智能无法完成这项工作,或者即使他们能够完成,人们也不会对你雇佣人工智能感到不满,或者你本人也不想雇佣人工智能吗?考恩:两者的混合。我认为那时仍会有一个默卡托斯,绝对如此。帕特尔:但能力会不足吗?考恩:是的。
Patel: You don’t think A.I. will not be able to do that work, or even if they could, that people will not be happy with you hiring A.I.s or that you will not want to hire A.I.s? Cowen: A mix of both. I think there’ll still be a Mercatus then, absolutely. Patel: But the capabilities will be lacking? Cowen: Yes.
考恩:现在以一个最后的问题结束。你在这段时间里一直在人工智能领域。你几乎采访了所有重要人物,问了你想问的所有问题。那就是你的目标。经过这一切,它对你有什么影响,达瓦克什?帕特尔:我不知道人工智能方面的东西是否特别改变了我。
Cowen: Now to close with one final question. You’ve spent all this time in the A.I. world. You’ve interviewed virtually everyone of importance, asked them all the questions you wanted to. That was your goal. At the end of all this, how has it changed you, Dwarkesh? Patel: I don’t know if the A.I. stuff in particular has changed me.
我真正欣赏你、布莱恩以及许多其他早期嘉宾的是,他们花了大量时间,无论是在镜头前还是镜头后,与我聊思想和其他事情。正是因为这样,我回顾那些早期采访时,我诚实地提出了非常天真的问题。我提出的问题,你们早在很久以前就以许多不同的方式思考过了。
Here’s something that I really appreciate about you and Bryan and many other of my early guests, people who spent a bunch of time, on camera but also off camera, chatting with me about ideas and stuff, is that I look back at those early interviews and I was honestly asking very naïve questions. I was asking things that you have thought about a long time ago, in many different ways.
随着时间的推移,我对许多更多主题有了更深入的了解。我认为,如果没有像你这样的人,他们非常慷慨地花时间与年轻、天真的人交谈——这些人并不是真的在以有趣的方式思考问题——但仍然诚实地回答问题并进行良好互动,这个过程会非常困难。因此,建立对不同知识领域更好的心智模型可能是主要的改变。考恩:谢谢你的善意的话语。达瓦克什·帕特尔,非常感谢你。帕特尔:谢谢。
Over time, I’ve gotten to learn about many more topics in a deeper way. And I think that process would’ve been very difficult were it not for people like you, who are very kind with their time in talking to people who are young and naïve and aren’t really thinking about things in a very interesting way, but still answering their questions and engaging in good faith. So building up a better mental model of different intellectual fields is maybe the main change. Cowen: Thank you for the kind words. Dwarkesh Patel, thank you very much. Patel: Thank you.
有什么想法?请发邮件至 interestingtimes@nytimes.com。
Thoughts? Email us at interestingtimes@nytimes.com.
本期《有趣时刻》节目由维多利亚·钱伯林、雷娜·拉斯金和夏洛特·加滕伯格制作。节目由艾莉森·布鲁泽克和索菲亚·穆尼兹编辑。混音和工程由帕特·麦克库斯克和埃菲姆·夏皮罗负责。摄影由玛丽娜·金和凯尔·I·凯利负责。摄影由维克·费雷尔负责。视频剪辑由朱利安·哈克尼、克里斯汀·威廉森和马赫多赫特·马赫穆德阿巴迪负责。
This episode of “Interesting Times” was produced by Victoria Chamberlin, Raina Raskin and Charlotte Gartenberg. It was edited by Alison Bruzek and Sophia Muniz. Mixing and engineering by Pat McCusker and Efim Shapiro. Cinematography by Marina King and Kyle I. Kelley. Cinematography by Vic Ferrer. Video editing by Julian Hackney, Kristen Williamson and Mahdokht Mahmoudabadi.
监制编辑为扬·科巴尔。后期制作经理为迈克·普尔茨。原创音乐由艾萨克·琼斯、索尼娅·埃雷罗、帕特·麦克库斯克和阿曼·萨霍塔创作。事实核查由凯特·辛克莱尔、玛丽·马吉·洛克、朱莉·比尔和米歇尔·哈里斯负责。受众策略由香农·布斯塔、克里斯汀·林和阿里·巴卢奇负责。执行制片人为乔达娜·霍赫曼。观点视频主管为乔纳·M·凯塞尔。
The supervising editor is Jan Kobal. The postproduction manager is Mike Puretz. Original music by Isaac Jones, Sonia Herrero, Pat McCusker and Aman Sahota. Fact-checking by Kate Sinclair, Mary Marge Locker, Julie Beer and Michelle Harris. Audience strategy by Shannon Busta, Kristin Lin and Ali Baluch. The executive producer is Jordana Hochman. The director of Opinion Video is Jonah M. Kessel.
观点节目副主管为艾莉森·布鲁泽克。观点节目主管为安妮-罗斯·斯特拉瑟。观点部门负责人为凯瑟琳·金斯伯里。
The deputy director of Opinion Shows is Alison Bruzek. The director of Opinion Shows is Annie-Rose Strasser. The head of Opinion is Kathleen Kingsbury.