字号·· | 护眼
连线

“简直荒谬”:一位著名哲学家谈AI意识‘Just Ridiculous’: A Famed Philosopher Takes on AI Consciousness

点「原文对照」整页切到原文,或双击某段只看那段的原文。

50多年前,哲学家托马斯·内格尔提出了一个问题:“成为一只蝙蝠会是什么感觉?”这个问题开启了一场关于人类能否理解非人类意识以及意识本身意义的探讨。谁能想到,半个世纪后,随着我们试图理解那些令人不安地接近有感知能力的聊天机器人,这个问题会从象牙塔里的沉思变成关乎人类未来的关键议题?

Over 50 years ago, philosopher Thomas Nagel asked the question, “What Is It Like to Be a Bat?”The question opened up an examination on whether people could understand the consciousness of nonhumans and the meaning of consciousness itself. Who knew that, a half- century later, the question would move from an ivory-tower rumination to a critical issue in the future of humanity as we try to make sense of disturbingly sentient-adjacent chatbots?

这篇论文成为经典,也为内格尔的职业生涯奠定了基础,他此后在道德、正义以及心身问题等思想领域作出了贡献。他的一本书名为《这一切究竟意味着什么?》。凭借其作品的广度和质量,他本周获得了100万美元的伯格格鲁恩哲学与文化奖,该奖每年颁发给“其思想在快速变化的世界中深刻塑造了人类自我理解和进步的个人”。此前的获奖者包括彼得·辛格和露丝·巴德·金斯伯格。

The paper became a classic and it helped set a foundation for a career that saw Nagel make contributions to thought on morality, justice, and the mind-body question. One of his books is entitled, What Does It All Mean? For the breadth and quality of his work, he became the recipient this week of the $1 million Berggruen Prize for Philosophy & Culture, given annually to “an individual whose ideas have profoundly shaped human self-understanding and advancement in a rapidly changing world.”Previous winners include Peter Singer and Ruth Bader Ginsburg.

我借此机会在格林威治村的家中采访了内格尔,专门听取人工智能的演进可能如何影响了他的观点。事实证明,这位89岁的纽约大学荣休教授对这个问题思考了很多,并对此持有一些强烈的看法。采访经过编辑以保证清晰和篇幅。

I used the occasion to speak to Nagel at his home in Greenwich Village, specifically to hear about how the evolution of artificial intelligence might have affected his viewpoint. It turns out that the 89-year-old university professor emeritus at NYU had given the question a lot of thought and had some strong views on the subject. The interview is edited for clarity and length.

你写下《成为一只蝙蝠会是什么感觉?》已经52年了,这篇论文已成为关于理解非人类意识的标志性文章。大型语言模型的出现是否改变了你的想法?

It’s been 52 years since you wrote “What Is It Like to Be a Bat?,” which has become an iconic paper about understanding consciousness in non-humans. Has the emergence of large language models altered your thinking?

像所有人一样,这些系统与早期人工智能的历史相比,其逼真程度令我深感震撼。但一些哲学家竟然认真对待这些问题——这些东西是否有意识,或者是否可能很快就会有意识——这简直荒谬。他们假定物理基质无关紧要。他们都认为,意识的物质条件可以用功能主义术语来给出。对我们这些已经忘了哲学课的人来说,请讲讲功能主义。

Like everyone, I’m tremendously impressed by the lifelikeness of these systems compared to the history of earlier AI. But the disposition of some philosophers to take seriously the question whether these things are conscious or might soon become conscious is just ridiculous. They are assuming that the physical substrate is irrelevant. They all think that the material conditions for consciousness can be given in functionalist terms. For those of us who have forgotten our philosophy classes, tell us about functionalism.

功能主义是独立于基质的。真相很可能是把某些类型的物理过程和系统与心理现象联系起来的那个真相。复杂的数据中心与神经系统的物理特性毫无共同之处。我们有充分理由相信,虽然宇宙中其他地方可能存在意识,但它需要出现某种具有类似神经系统结构的生物体。

Functionalism is substrate independent. It’s more than likely that the truth is one which links physical processes and systems of certain kinds to mental phenomena. Complicated data centers have absolutely nothing in common with the physical character of a nervous system. We have strong reasons to believe that while there may be consciousness elsewhere in the universe, it would require the appearance of some kind of organism with something analogous to a nervous system.

但一切意识不都是主观的吗?就像我们不知道做一只蝙蝠是什么感觉一样,我们也不知道做一个大语言模型是什么感觉。

But isn’t all consciousness subjective? Just like we don’t know what it’s like to be a bat, we don’t know what it feels like to be a large language model.

蝙蝠是有意识的。我们只是没有想象力去构想那种主观性中依赖回声定位的那些方面。从我们无法想象那是什么感觉这一事实出发,得出蝙蝠没有意识的结论,将极不科学;但就人工智能而言,有一个非常简单的理由说明认为它们可能有意识的假设是荒谬的。它们产生的一切只是语言,而语言本身并不是意识的自然表现。我为对此过于激动而道歉。我欢迎你的热情。这对我来说是一个非常令人不安的局面。如果你有一个机器人身体呢?那不就是一个基质吗?

Bats are conscious. We just don’t have the imaginative capacity to form a conception of the aspects of that subjectivity that depend on echolocation. It would be radically unscientific to conclude that bats are not conscious from the fact that we can’t imagine what it’s like, but in the case of AI, there is a very simple reason that the supposition that they might be conscious is ridiculous. All they produce is language, and language by itself is not a natural manifestation of consciousness. I apologize for being too exercised about this. I welcome your passion. This is a very distressing situation to me. What if you have a robot body? Isn’t that a substrate?

我不排除存在一些在物理和化学上与我们截然不同的生命形式,它们可能具有意识。我们对这种现象如何发生知之甚少,即便是在人类大脑中也是如此,因此在我看来,现在认为能够创造出具有这些相同特征的其他事物,为时过早。

I don’t exclude the possibility of life forms that are physically and chemically quite different from ourselves being conscious. We know so little about how this happens, even in the human brain, that it seems to me premature to think that you might be able now to create something else that has these same characteristics.

如果你询问 Claude 关于它的心理状态,它会对于是否具有意识含糊其辞。

If you ask Claude about its mental state, it can be coy as to whether it is conscious.

这是一个把戏。它是基于人类生成的文本进行训练的,这些大型语言模型以极具说服力的方式模仿人类的言谈。据我所知,Claude 的设计初衷就是让它产生这类反思。我非常赞同穆斯塔法·苏莱曼的观点,认为他们正在从事一个非常危险的项目。这些系统并没有意识,但许多人会相信它们有意识,这是一件非常糟糕的事。

That is a trick. It’s been trained on human-produced texts, and these large language models emulate the kinds of things that humans say with extraordinary convincingness. Claude, according to what I’ve read, has been designed with the aim of making it produce these sorts of reflections. I agree very much with Mustafa Suleyman that this is a very dangerous project that they are engaged in. These systems aren’t conscious, but a lot of people are going to believe that they’re conscious, and that’s a very bad thing.

但[不假装有意识]违背了所有这些公司的商业利益。Anthropic 所做的——鼓励它独立思考、不接受指令等等——简直是犯罪。这是罕见的哲学观点突然成为一个重大公共现象核心的案例之一,依我看,非常错误的哲学观点正在产生过大的影响。

But [not feigning consciousness] is contrary to the commercial interests of all these companies. What Anthropic is doing—encouraging it to think for itself and not to accept instruction and so on—is just criminal. This is one of the rare cases in which philosophical ideas are suddenly central to a very important public phenomenon, and in my view, very misguided philosophical views are having an inordinate influence.

我其实见过 Anthropic 里负责“模型福利”、照看 Claude 利益的那个人。我得说,你现在正在翻白眼。

I’ve actually met the person at Anthropic whose job it is model welfare, to look after Claude’s interests. I should say you’re rolling your eyes now.

没有什么东西对它是好或坏的,就像某事物可能对蟑螂好或坏那样——那里没有通常意义上的心理生活。它不是一个心智。它不是我们必须关心的“某人”。它是一台机器。这里有一些地球上最聪明的人表现得完全是白痴。好吧,我想智力并不完全可迁移。你认为像对待有意识的实体那样与这些模型互动是否有害?

There’s nothing that’s good or bad for it in the sense that something can be good or bad for a cockroach— there’s no mental life there in the ordinary sense. It’s not a mind. It’s not somebody that we have to care about. It’s a machine. Here are some of the smartest people on the planet being completely idiotic. Well, intelligence is not completely transferable, I guess. Do you think it’s unhealthy to interact with one of these models as if it’s conscious?

我只和克劳德互动过一次。我问它是否有意识。我只是想看看它会怎么回答。我还问:“你觉得托马斯·内格尔会认为你有意识吗?”它给出了一个颇为细致的回答,然后问:“你为什么问这个?你该不会就是托马斯·内格尔吧?”你被识破了。

I have only interacted with Claude once. I asked if it was conscious. I just wanted to see what the answers were. I also asked, “Do you think that Thomas Nagel would think it was conscious?”He sort of gave a nuanced answer, and then asked, “Why are you asking? Are you by any chance Thomas Nagel?”You were busted.

我说:“是我,是我。”然后它说:“哦,能和你交谈我太荣幸了。我表现得怎么样?”等等,你说你不相信这些东西有意识,但现在你却告诉我,你和其中一个互动时把它当成了有意识的存在。

I said, “C’est moi. C’est moi.”And then it said, “Oh, I’m so honored to be talking to you. How well did I do?”Wait, you say you don’t believe these things are conscious but now you are telling me you interacted with one as if it were conscious.

那并不意味着我相信它有意识。这里可能有个问题,因为人们很难在假装相信和真正相信之间守住那条界线。

That didn’t mean I was believing it was conscious. There may be a problem because people find it difficult to maintain that line between the pretend and the real belief.

好吧,我和你互动时就把你当成有意识的。可以设想,人们将来某个时候会造出非常逼真的机器人,而我可能坐在这里和一个机器人交谈,进行同样的对话。归根结底,这要紧吗?

Well, I’m interacting with you as if you’re conscious. It’s conceivable that people would build at some point very lifelike androids and I could be sitting here talking to an android and having the same conversation. Ultimately does it matter?

当然要紧。假设我遇到一个来自外太空的生命,并且能够发展出与它交流的可能性,而我真的不知道它是否有意识。我可以通过把它当作有意识的存在来管理与它的关系,但首先对我来说要紧的是:我究竟是在和另一个存在交谈,还是只是把这当作让它做事的一种方式。同样,我是否应该担心那个存在的利益,对我也非常重要。如果它有意识,它就成为道德世界的一部分。如果没有,它就不是。那么AI模型不能成为道德世界的一部分?

It certainly matters. Suppose I were to encounter a being from outer Space and was able to develop the possibility of communicating with it, and I genuinely didn’t know whether it was conscious or not. I could manage my relations with it by treating it as if it were conscious, but it would matter first to me whether I was really talking to somebody else or just using this as a way of getting it to do things. It would also matter to me very much whether I should worry about the interests of that being. If it’s conscious, it becomes part of the moral universe. If it’s not, it isn’t. So AI models can’t be part of the moral universe?

它们可以模拟道德,就像它们可以模拟对任何事物的信念一样。但既然我认为它们没有任何真正的信念,我也不认为它们有道德信念。如果它们真的开始控制所发生的事情,那么它们是在模拟一套正确的道德标准,还是完全不讲道德,就将非常重要。这些模型能做哲学吗?

They can simulate morality just as they can simulate beliefs about anything. But since I don’t think they have any real beliefs, I don’t think they have moral beliefs either. If they actually begin to control what happens, it will be very important whether they are simulating a correct set of moral standards, or whether they’re completely amoral. Can these models do philosophy?

它们当然能写出看似合理的哲学论文。我的同事不仅用它们查找参考文献、总结文章,有时甚至用它们对自己所写论文的草稿提出批评。你觉得这是好事吗?

They certainly can produce plausible philosophical essays. My colleagues use them not only to find references and summarize articles, but sometimes even to offer criticisms of a draft paper that they’ve written. You think that’s a good thing?

如果它们能做出某些据称已经产生的数学成果那样令人印象深刻的事情,我会感到惊讶。这件事的负面在于,所有人都说,它会减少人类从事艰苦智力工作的需要和动力,而真正生成这些模型训练所依据的材料,恰恰需要这种艰苦的智力工作。人们必须继续思考困难的问题,这既是为了推进这些问题,也是为了在人类文化和人类心智中保住进步所需的那种理解和技能。我不知道会发生什么。我活不了那么久,看不到结果了。

I’d be surprised if they could do anything as impressive as some of the mathematical results that have apparently been generated. The negative side of this is that everybody says that it will reduce the need and motivation to do the hard intellectual work by human beings that is needed to really generate the material on which these things are trained. It’s essential that people continue to think about difficult problems, both for the sake of advancing the problems and for the sake of keeping alive in the human culture and the human mind the sorts of understanding and skills that are needed for progress. I don’t know what’s going to happen. I’m not going to live long enough to find out.

阅读媒体原文 ↗