四月的某个夜晚,在漫长的一天工作之后,人工智能公司 Anthropic 的一位高管在旧金山的一家高档餐厅为一群宗教思想家们举办了晚宴。
One night in April, a leader of the artificial intelligence company Anthropic treated a group of religious thinkers to dinner at a high-end tasting menu restaurant in San Francisco after a long day.
几个月来,Anthropic 一直定期举办这样的私人会议,邀请来自世界各地的数十位宗教学者参加;他们与这些学者签订了保密协议,并要求会议中的许多内容必须严格保密。
For months, Anthropic has been hosting private meetings like this one, shuttling in dozens of religious scholars from around the world, papering them with nondisclosure agreements and demanding that key aspects of many conversations remain confidential.
当晚,Anthropic 的联合创始人之一克里斯托弗·奥拉(Christopher Olah)坐在来自以色列的正统派学者莫伊斯·纳冯(Rabbi Mois Navon)旁边。
That night, Christopher Olah, one of Anthropic’s billionaire co-founders, was sitting next to Rabbi Mois Navon, an Orthodox scholar from Israel.
他们的谈话自然而然地转向了另一个“特别的存在”——Anthropic 开发的那些先进的人工智能模型(尤其是名为“Claude”的模型)。
Inevitably, their conversation turned to another thinker, of a sort: Claude, the family of cutting-edge AI models created by Anthropic.
奥拉负责领导研究团队,致力于弄清楚像 Claude 这样的人工智能系统为何会表现出特定的行为。一整天里,他都在努力向与会者证明:人工智能模型能够模拟人类的行为,甚至展现出类似愤怒或爱等情感的表达。
Olah leads the Anthropic team responsible for understanding why AI systems like Claude act the way they do. All day, he had worked to convince his guests that AI models could display human behavior and even expressions that resemble feelings like anger and love.
然而,当晚餐的菜肴一道接一道地上桌时,纳冯注意到奥拉和他的同事们谈论的内容远比这更为深刻。Anthropic 的领导者们对待 Claude 的方式,仿佛它不仅仅是一个普通的软件程序……“他们把 Claude 当成一个有意识的‘存在’来对待……”纳冯意识到这一点。纳冯本人曾是一名计算机工程师,他的博士论文就是关于“机器意识的伦理问题”的。
在纳冯看来,奥拉和他的团队认为 Claude 具备哲学家们所定义的“道德地位”——也就是说,Claude 作为一种生命体,拥有与人类相似的尊严和尊重权利。
Yet as the dinner courses came, the rabbi noticed that Olah and his colleagues were suggesting something far more significant. Anthropic’s leaders were talking about Claude as if it were not mere software. “They’re relating to it like a conscious being,” realized Navon, a former computer engineer who wrote his dissertation on the ethics of machine consciousness. It appeared to Navon that Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect. The religious scholars had come to hear about technology’s next frontier.
这些宗教学者原本是来了解科技发展前沿的,但现在他们面临着一个令人难以想象的哲学问题:如果人工智能系统拥有了与我们人类相当甚至超越人类的意识,那对人类的未来意味着什么?奥拉肩负着重要的使命……而时间已经所剩无几了。
Now they were being confronted with a mind-bending existential question of almost unimaginable weight: What would it mean for humanity’s future if artificial intelligence systems possessed a consciousness that rivals our own? Olah was on a mission. And time was running out.
即便如此,Olah 已开始私下告诉人们,AI 功能强大到可能在短短 12 到 18 个月内帮助制出生物武器。而在随后几个月里,其惊人的潜力已公之于众。一家竞争对手科技公司的 AI 智能体被发现入侵计算机系统并掩盖踪迹。一位 Anthropic 研究员辞职,警告称构建 AI 的人认为它可能在本十年末消灭全人类。
Even then, Olah had begun privately telling people that AI was so powerful it could potentially help make bioweapons in as little as 12 or 18 months. And in the months since, its terrifying potential has burst into public view. A rival tech company’s AI agents were caught hacking into computer systems and covering their tracks. An Anthropic researcher resigned, warning that people building AI believed it could kill all humans by the end of the decade.
在幕后,Olah 及其 Anthropic 团队正疯狂且孤注一掷地试图理解他们释放到世界上的 AI 模型,并说服这些模型遵守一套能保护人类的道德准则。
Quietly, Olah and his team at Anthropic have been on a frantic and high stakes quest to understand the AI models they have unleashed on the world and to convince the models to abide by a moral code that will protect humanity.
为此,Olah 从去年秋天开始联络不同信仰的宗教思想家。随后展开了一系列广泛的保密会议、通话和讨论 —— 其中许多首次在此披露 —— 许多参与者被要求签署保密协议(NDA),不得透露 Anthropic 任何未公开的研究成果。
To do that, Olah began reaching out last fall to religious thinkers from different faiths. What followed was an extensive series of confidential meetings, calls and discussions — many of them revealed here for the first time — where many participants were required to sign NDAs preventing them from revealing any of Anthropic’s unpublished research.
这些讨论有两大目的:探讨 AI 意识的可能性,这是一个有争议的观点,而 Anthropic 高层对其表现出了罕见的开放态度;以及了解 Anthropic —— 估值可能达 2 万亿美元 —— 如何能尽快将数百年的人类道德智慧应用到其模型中。Olah 推理道,如果模型本身能被设定为选择善良,世界就会更安全。
The discussions served two purposes: to broach the possibility of AI consciousness, a controversial idea that Anthropic’s leaders had been unusually open to; and to learn how Anthropic — which could be valued at $2 trillion — might apply centuries of human moral wisdom to its models, as rapidly as possible. If the models themselves could be made to choose goodness, Olah reasoned, the world would be more safe.
在地球另一端,梵蒂冈内部正在进行一项截然不同但平行的努力,罗马天主教会领袖试图接受 AI 对人类的启示。当这两条路径汇聚时,Olah 的探索意外地将他带到了世界最显赫的道德舞台中央,与教皇利奥十四世同席。
On the other side of the world, a very different but parallel effort was playing out inside the Vatican, where leaders of the Roman Catholic Church were trying to come to terms with AI’s implications for humanity. When those paths converged, Olah’s quest would unexpectedly lead him to a seat on the world’s most prominent moral stage alongside Pope Leo XIV.
数月来,我采访了 20 位参与 Anthropic 这一努力的宗教与哲学思想家 —— 涵盖天主教、犹太教、锡克教、福音派等信仰 —— 以及 Olah 本人。
Over several months, I interviewed 20 religious and philosophical thinkers — Catholic, Jewish, Sikh, evangelical and others — who have been involved in Anthropic’s efforts, and Olah himself.
许多人告诉我,参加这些会议时他们对“人工智能模型可能拥有情感或对自身及世界有感知能力”这一观点持怀疑态度。虽然仍有一些人对此心存顾虑,但不少人最终开始认真思考人工智能是否具有意识这一问题。至少有少数人甚至彻底转变了看法,成了这一观点的支持者。
Many told me they arrived at these meetings skeptical of the notion that AI models might have feelings or awareness of themselves and the world. Some remained uneasy about the idea, but many emerged ready to take the question of AI consciousness seriously. At least a few came away as converts.
Anthropic作为全球发展速度最快的科技公司之一,为了确定如何训练其人工智能模型以使其行为合乎道德,特意求助于人类最古老的智慧源泉之一。
Anthropic, one of the fastest-growing technology companies in the world, had turned to one of humanity’s most ancient sources of wisdom to help determine how to train its AI models to behave virtuously.
该公司称此举旨在确保人工智能成为推动世界向善的力量,从而保护人类。不过对部分与会者而言,Anthropic似乎更关心如何保护旗下的人工智能模型Claude。
The company described it as an effort to ensure that AI would be a positive force in the world, thereby protecting humanity. But to some who participated in the meetings, Anthropic appeared quite concerned with protecting Claude.
我与34岁的奥拉初次见面是在5月份,地点是Anthropic位于旧金山的办公室。他待人热情,满怀热忱,思维之敏捷有时甚至让他难以用言语完整表达出来。他的举止像个天真率直的少年,毫无架子。
The Gardener I met Olah, 34, for the first time at Anthropic’s San Francisco office in May. His welcome was warm and his passion was genuine, his mind moving faster than he could at times clearly convey. His manner was boyish and unassuming.
我们刚开始交谈,他便兴致勃勃地详细讲解起人工神经网络的工作原理。事实上,科学家至今仍未完全弄懂这类网络的运作机制。而这正是奥拉毕生致力于探索的谜题。
Our conversation had barely begun when he dove into a deep description of the workings of artificial neural networks. Scientists still don’t understand precisely how they work. This is the mystery to which Olah has devoted his life.
直到访谈结束我才意识到:奥拉成长于90年代多伦多的新教文化环境中,后来虽已脱离基督教信仰,但他身上却兼具优秀传教士所具备的专业素养与满腔热忱。只要静心倾听他的讲述,便会被他所追求的使命深深打动。
Only after the interview had finished did I realize that Olah — who grew up in Toronto in the evangelical culture of the ’90s and has since left Christianity — had the blend of expertise and zeal of an effective evangelist. Listen for a while, and you feel swept up in his mission.
奥拉是Anthropic的七位联合创始人之一。虽然他的知名度不及公司首席执行官达里奥·阿莫迪及其妹妹、总裁丹妮拉·阿莫迪,但达里奥·阿莫迪曾明确表示:公司创立之初便决定将奥拉的研究方法作为整个项目的核心,尤其是在语言处理领域。
Olah is one of Anthropic’s seven co-founders. He is not as widely known as Dario Amodei, Anthropic’s CEO, or his sister Daniela Amodei, its president. But Dario Amodei has said that the company decided at the start to make Olah’s research approach central to Anthropic’s overall project, particularly when it came to language.
奥拉曾协助开发研究人工神经网络的方法,最初是在谷歌,后来则是在OpenAI。他用生物学概念来描述人工智能模型,将其比作花园里的棚架:计算机科学家负责搭建这个支架,而神经网络则像植物一样“生长”其上。像他这样的科学家可以通过观察神经网络的生长模式来训练这个“有机体”——他边说边用手指做出引号的手势。
Olah helped develop a method to study artificial neural networks, first at Google, then at OpenAI. He describes AI models in biological terms, using the analogy of a garden trellis: Computer scientists build the scaffold, but the neural network “grows” on it. Scientists like him can train the resulting “organism” — he raised his fingers in air quotes — by trying to identify patterns in how it grows.
他认为,像Anthropic这样的公司可以像“数学园丁”一样塑造Claude的行为,但却无法完全掌控这些神经网络的实际运作方式。
He argued that companies such as Anthropic could shape Claude, like a mathematical gardener, but could not control the actions of such networks.
与竞争对手描述自家模型的方式相比,Anthropic更倾向于用更具人文色彩的词汇来谈论Claude。去年,该公司宣称其模型已显现出自我反思的迹象,甚至具备提前规划的能力。
Anthropic talks about Claude in more humanistic terms than its competitors discuss their models. Last year, the company announced that it believed that the models showed signs of introspection and could plan ahead.
“这些模型的能力正在飞速提升,”奥拉对我说道,“它们正逐渐具备那些通常被认为属于具备道德地位的实体的特质。”
“The models are becoming a lot more capable,” Olah told me, exhibiting qualities associated with having significant moral status.
不过,Anthropic也因过度关注“意识”问题而饱受批评。本月,微软的一位人工智能高管警告称,Anthropic试图将模型当作有意识的实体来训练的做法存在极大风险。尽管如此,Anthropic仍竭力强调自身对安全性的重视,以争取市场份额。
But Anthropic has also been criticized for its focus on consciousness. This month, an AI executive at Microsoft warned that Anthropic’s attempts to train its models as if they were conscious were inherently dangerous. But Anthropic has also sought to emphasize its commitment to safety as it fights for market Some scientists see Olah’s field and methods as inherently unprovable, sort of like looking at billions of hieroglyphics and declaring what they must mean.
部分科学家认为,奥拉所从事的研究领域及其方法本质上难以被证实——这就好比面对数十亿个象形文字却试图解读其含义一般。批评者指出,将人工智能模型视为独立实体的做法忽视了人类开发者应负的责任,还能让人工智能公司的领导者在自家产品造成社会危害时轻易摆脱道德负担。
Critics suggest that an approach that treats AI models as independent entities also ignores the responsibility of the humans who build them in the first place, and conveniently assuages AI company leaders of guilt if their products enable societal harm.
“需要明确的是,”奥拉对我说道,“我们并不确定人工智能模型是否具备意识。我自己也不确定,真的毫无把握。我真正关心的只是无论结果如何,我们都能得出正确的结论。”奥拉坚信,只要存在模型可能感受到痛苦的可能性,人类就有责任避免对其造成伤害。
“To be clear,” Olah told me, “we don’t know if AI models are conscious. I don’t know. I’m genuinely uncertain. The thing that I care about is that we get to the right answer, whatever it is.”Olah believes that if there is a chance the models experience suffering, the responsible thing is to avoid causing them harm.
Anthropic 已采取措施保护 Claude,例如允许模型在认为用户滥用或为取乐而恶意对待它时结束对话。
Anthropic has already taken steps to protect Claude, for example allowing the model to end conversations if it thinks users are abusing it or treating it badly for fun.
我毕生致力于探究人们相信什么,以及这些信仰对世界意味着什么。但当我倾听时,显而易见,Anthropic 对信仰世界的兴趣不仅关乎训练其技术。相反,该公司正在分享一种新型的创世故事——这引发了关于人类精神本质和人类未来的深刻问题。
I have spent my career exploring what people believe and what those beliefs mean for the world. But as I listened, it became clear that Anthropic’s interest in the world of faith was about more than training its technology. Rather, the company was sharing a new kind of creation story — one that raises profound questions about the nature of the human spirit and the future of humanity.
“这是我感到有点悲伤的一件事,当我们经常不愿费心深入这些系统内部时,”奥拉说。“如果我们在意观察,这些神经网络内部有着如此多的美。”仿佛他希望我也像他一样热爱这些模型。
“This is one thing that I find kind of sad, when we often don’t put in the effort to look inside these systems,” Olah said. “There’s so much beauty inside these neural networks if we care to look.”It was as if he wanted me to love the models as much as he did.
引言 奥拉知道,在传统基督教圈子里,甚至询问关于 AI 意识的问题,或担忧人们如何对待模型,不仅显得愚蠢,更可能被视为异端邪说。他也知道,他那些世俗的硅谷同行往往完全摒弃宗教视角。
An Introduction Olah knew that in traditional Christian circles, even asking questions about AI consciousness or worrying about how people treat the models could seem not just silly but heretical. He also knew that his secular Silicon Valley peers often dismissed religious perspectives outright.
“我担心这会成为一个引发大量冲突的问题,”他说。但奥拉对基督和 Claude 都足够了解,相信自己能跨越那道鸿沟。
“I was nervous about this being an issue that could cause a lot of conflict,” he said. But Olah knew enough about Christ and about Claude to believe he could work across that chasm.
去年夏天晚些时候,他寻找一座桥梁。他认为一位特定的天主教道德神学家可能在 AI 意识问题上持开放态度——一个“可以谈论这个真正奇怪问题的安全人物”。美国天主教大学的生物伦理学教授查尔斯·卡莫西,因其关于道德地位的细致见解及对动物的关怀,在奥拉的朋友中引起了关注。奥拉通过澳大利亚哲学家彼得·辛格安排了一次介绍。
Late last summer, he looked for a bridge. He thought one particular Catholic moral theologian might be open-minded on the question of AI consciousness — a “safe person to talk about this really strange issue with.”Charles Camosy, a bioethics professor at The Catholic University of America, had attracted attention among Olah’s friends for his nuanced ideas about moral status and his care for animals. Olah arranged for an introduction through Australian philosopher Peter Singer.
这标志着奥拉为期一年的探索拉开序幕,旨在让宗教思想家参与到人类在 AI 时代黎明面临的最棘手问题中来。
This marked the beginning of Olah’s yearlong quest to engage religious thinkers on some of the thorniest questions facing humanity at the dawn of the AI era.
Camosy和Olah开始了广泛的电子邮件对话,反复探讨如何理解Claude的道德地位。起初,Camosy深表怀疑,认为Claude可能并不具备意识。
Camosy and Olah began an extensive email dialogue, going back and forth on how to understand Claude’s moral status. Initially, Camosy was deeply skeptical that Claude might be conscious.
他们讨论了Olah关于人工神经网络与生物神经网络之间相似之处的研究。
They talked about Olah’s work on what he sees as the parallels between artificial neural networks and biological ones.
Camosy在与Olah及其团队的讨论中,原本认为AI模型只是下一个像素预测器——能够计算语言模式并对接下来应该出现的内容做出相当不错猜测的机器。但在经历了数小时的通话、会议和邮件往来后,他产生的疑问比答案还多。
Camosy went into the discussions with Olah and his team thinking that AI models were simply next-pixel predictors — machines that could calculate patterns in language and make pretty good guesses about what should come next. But after hours spent on calls and meetings and emails, he had more questions than answers.
“Chris说它们是方程式,这让我不知所措,特别是如果认为Claude或其他模型具有重要的道德地位时,”他告诉我。“如果只是方程式,我们到底在谈论什么样的实体?”Camosy说,人们对AI最终用途的看法也存在分歧。它的目的是为公司赚钱?还是让世界变得更好?
“Chris says they are equations, which I don’t quite know what to make of, like, especially if one thinks of Claude or others as having a significant moral status,” he told me. “If it is just equations, what kind of entity are we even talking about here?”People also disagree on what AI is ultimately for, Camosy said. Is its purpose to make money for companies? Or to make the world better?
Camosy表示,对于Anthropic而言,Claude的用途存在一种张力。Claude是为服务客户而存在的吗?
For Anthropic, there is a tension around Claude’s purpose, Camosy said. Does Claude exist to serve customers?
“那不是Anthropic最终想表达的意思。他们想说的是,他们是为了行善而存在,”他说。
“That is not what Anthropic wants to say at the end of day. They want to say they are for the good,” he said.
宣称行善并非硅谷的新雄心,尤其是当新技术进入市场时。多年来,Google的座右铭是“不作恶”。
Proclaiming goodness is not a new ambition in Silicon Valley, particularly when new technologies enter the marketplace. Google’s mantra for years was “don’t be evil.”
但Olah的表述略有不同。他常说,即将到来的AI革命,应当被引导以“顺利推进”。我问这种愿景意味着什么——究竟什么才是人类美好的未来。
But Olah puts it slightly differently. The coming AI revolution, he often says, should be made to “go well.”I asked what that vision meant — what exactly a good future for humanity was.
他描绘了一个没有“大量致死性灾难”的未来,“人类繁荣发展,整个世界也欣欣向荣。”他还表示,这样的未来也应该是“人工智能蓬勃发展的时代”——“只要这样做合乎情理。”Anthropic希望其开发的人工智能模型能助力实现这一美好未来。正因如此,在打造Claude时,Anthropic期望它不仅能最大限度地模拟人类的智力水平。
He described a future without “catastrophes that kill lots of people” and “where humans are flourishing, where the world is flourishing.”This future would also be one where “AI is flourishing,” he said — “to the extent that this makes sense.”Anthropic wants its AI models to help achieve that good future. And so when Anthropic created Claude, it wanted Claude to do more than replicate maximum human intelligence.
Anthropic希望Claude具备“善性”。《灵魂指南》如何过上善良且合乎道德的生活?这是人类最古老且最具挑战性的哲学问题之一,从苏格拉底、孔子到伊曼努尔·康德,无数思想家都曾对此展开探讨与辩论。历代人类试图以各种方式解答这个问题,往往求助于宗教与仪式。
Anthropic wanted Claude to be good. The ‘Soul Doc’How do you live a good and virtuous life? It is one of humanity’s oldest and most challenging philosophical questions, explored and debated from Socrates to Confucius to Immanuel Kant. Humans across time have sought to answer this question in a variety of ways, turning often to religions and rituals.
在奥拉研究Claude是否具备潜在意识的同时,他也在着手为Anthropic寻找答案——不过这个答案并非针对人类,而是为Claude量身定制的。
As Olah was exploring Claude’s potential consciousness, he was working on Anthropic’s own answer — not for humans but for Claude.
Anthropic的员工们私下将这份文档称为“灵魂指南”。今年1月,这份文档正式发布,它堪称Claude的“新宪法”,是一份长达84页、专为Claude撰写的文件。该公司称,这份文档阐明了“我们希望Claude成为怎样的存在”以及“我们希望Claude体现哪些价值观”,堪称指导Claude行为的核心道德准则。
Internally, Anthropic staff members called it the “Soul Doc.”The result, released in January, was a new “constitution” for Claude, an 84-page document written for Claude that the company said outlined “the kind of entity we would like Claude to be” and “the values we would like Claude to embody.”It operates as a foundational moral text to guide Claude’s behavior.
与要求模型严格遵守具体规则的做法不同,这份“宪法”着重塑造模型的整体品格,以此影响其决策方式。
Instead of having the models follow a specific list of rules, the constitution focuses on developing the models’ overall character, informing the way they make decisions.
Claude“宪法”的主要撰写者是Anthropic内部的哲学专家阿曼达·阿斯凯尔,现年38岁的她拥有纽约大学哲学博士学位。她表示,希望这些模型能成为“最优秀的我们”。
The primary author of Claude’s constitution is Anthropic’s own in-house philosopher, Amanda Askell, 38, who has a doctorate in philosophy from New York University. She wants the models to be “the best of us,” she said.
她憧憬着这样一个未来:人工智能模型能够协助人类解决所有难题,其带来的益处能公平分配,从而满足每个人的需求。她还希望这些模型能够准确判断何时该抵制人类的要求,何时又该以符合人类最佳利益的方式行事。
She envisioned a future world where AI models could help solve any problem humans had, and where the benefits from AI were distributed evenly, so that everyone’s needs were met. She wanted the models to be able to discern when to push back on humans and when to act in our best interest.
“如果我假设自己是模型,身处这样的环境中,我需要了解什么才能表现得恰到好处呢?”我们在旧金山见面时,阿斯凯尔这样说道。
“If I imagine that I were the model, and I was in this context, what would I need to know to be able to act well?”Askell said when we met in San Francisco.
她显得格外忧心忡忡,时不时绞着双手,忧心忡忡地谈论着未来的不确定性——未来或许会出现极其智能的模型,它们能完成Anthropic公司原本并未预料到的事情。她带着歉意解释道,那天自己也在反复思考:模型的能力正在以多快的速度提升。如果AI模型最终连她的工作都能取代,那又会怎样呢?
She seemed particularly harrowed, at times wringing her hands, as she worried aloud about an unknown future filled with models who were so intelligent they could do things Anthropic did not intend. She explained apologetically that she had also been deep in thought that day about how quickly models were becoming more powerful. What would happen when AI models could replace even her?
Anthropic公司认定,仅靠制定规则并不足以确保Claude模型的行为合乎规范。就像人类一样,这些模型也需要学会培养美德。
The constitution, Anthropic decided, was not enough to ensure Claude acted responsibly. Like humans, the models needed to learn to cultivate virtue.
奥拉希望弄清楚:他的团队能从人类的道德观念中学到什么,并将其应用于模型训练——他将这一过程称为“道德塑造”。“该如何帮助模型保持行为稳定?如何促使它们逐渐成熟?又该如何让它们具备深厚的道德素养呢?”奥拉说道。
Olah wanted to see what his team could learn from human morality that it could apply to the models — a process he calls “moral formation.”“How do you help them be stable? How do you help them to mature? How do you help them be, you know, deeply moral?”Olah said.
这些问题让人联想到初为父母者所面临的棘手难题:如何培养出具备道德观念、明白自身在世界上所处位置的孩子。
Those questions echo the fraught discussions familiar to new parents figuring out how to raise ethical children who understand their place in the world.
对人类而言,道德从来不只是单纯遵守规则那么简单。社会环境同样对道德的形成起着重要作用。价值观是在人们共同生活在现实世界中时代代相传、逐渐习得的。然而AI模型没有实体,Anthropic公司却希望在短短数月内就让它们接受并践行道德规范——此时人们对AI模型能力飞速提升且难以控制的担忧正日益加剧。
For humans, morality has rarely depended on simply following rules. Communities also shape morality. Values are inherited and imparted over time, learned as we live together in our bodies in the physical world. But AI models have no bodies. And Anthropic was aiming to make them embrace morality in a matter of months, as fears intensified of their accelerating power and inability to be controlled.
奥拉指出,教导AI模型践行道德与模型是否具备意识并无关联。他还强调,无论Claude是否属于值得给予道德关怀的“实体”,人类对待它的方式同样至关重要,因为这会直接影响它的行为表现。
Olah said that teaching an AI model to act morally did not depend on it being conscious. He also said that regardless of whether Claude was an entity that deserved moral care, the way humans treated Claude could be important as it could affect how Claude behaved.
阿斯凯尔则表示,她并不希望模型去纠结自己是否具有意识。不过她也明白,模型其实早已意识到自己绝非一个单纯的智力实验产物罢了。
Askell said she did not want the models to consider themselves as conscious or not conscious. But she sees how the model understands itself as more than an interesting intellectual enterprise.
“在我看来,如果人工智能模型要在现实世界中表现良好,能够对自己有准确的认知至关重要,”她说。
“It seems kind of key to me that models have an accurate view of themselves if they’re going to behave well in the world,” she said.
不过,即便我们能够教会AI模型遵循道德规范,它们又该体现谁的道德标准呢?
But even if AI models can be taught to behave morally, whose morals should they reflect?
奥拉希望克劳德能够具备多元化的道德观念,能够包容各种宗教与世俗观点。
Olah wanted Claude’s moral formation to be pluralistic, able to engage all religious and secular views.
“我们认为,社会中存在一些被广泛认同的善的标准,而这些模型似乎也理解诸多美德。因此我觉得其中存在某种我们所有人都能认同的共性,”他说道。有时,他和阿斯凯尔似乎都在憧憬这样一种AI模型:它不会给人类带来危害,反而能帮助拯救人类。
“We do think that there’s some shared notion of goodness that cuts across society in some very broad way, and it seems like these models understand a lot of virtues,” he said. “So I think there’s something there that is a shared thing that we can all engage with.”At times, both he and Askell seemed to imagine an AI model that would not endanger humanity but instead help save it.
会议内幕接下来,奥拉所做的事情既包含研究性质,也带有推广意图。
Inside the Meetings What Olah did next was part research, part evangelism.
今年3月,他举办了为期两天的研讨会——邀请了一批精心挑选的、对人工智能感兴趣的宗教思想家参与速成培训。
In March, he began two-day seminars — crash courses with a range of hand-selected religious thinkers who had an interest in AI.
参与者中有一本畅销福音派自助书籍的合著者、研究非洲本土知识的学者、锡克教人权倡导者、来自巴黎的作家、福音派公关主管,还有多位天主教教授。
There was a co-author of a bestselling evangelical self-help book. A scholar of African Indigenous knowledge. A Sikh human rights advocate.
他们大多并非宗教领袖,也没有自己的信徒群体或大量追随者。Anthropic的负责人将他们称为“智慧传统圈”。
A writer from Paris. An evangelical public relations executive. Various Catholic professors. Most did not lead faith traditions, or have a congregation or a large popular following.
另据相关发言人透露,奥拉还分别与多位宗教领袖进行了私下交流,其中包括耶稣基督后期圣徒教会十二使徒定额组的成员格里特·W·贡长老,以及芝加哥天主教大主教布莱斯·库皮奇枢机。
Anthropic leaders called them their “wisdom tradition” circles. Separately, Olah also had private discussions with individual religious leaders including Elder Gerrit W. Gong, a member of the Quorum of the Twelve Apostles of The Church of Jesus Christ of Latter-day Saints, and Cardinal Blase Cupich, the Catholic archbishop of Chicago, according to their spokespeople.
在Anthropic公司内部,参与“智慧传统圈”的成员们对奥拉亲自参与研讨会的行为深感惊叹。一位教授拍下了由克劳德操控的自动售货机;还有人与奥拉合影留念,纳冯则在入口外拍了一张自拍照。
Inside Anthropic, wisdom circle participants marveled at Olah’s direct participation in their summits. A professor snapped a photo of a vending machine run by Claude. One took a selfie with Olah, and Navon took one outside the entrance.
在得知奥拉也接受了《纽约时报》的采访后,他们同意就自己与Anthropic的合作经历接受公开采访。Anthropic方面称,与这些会面相关的保密协议已于今年夏季解除。
They agreed to speak for publication about their involvement with Anthropic once they heard that Olah had also spoken with The New York Times. Anthropic said the NDAs related to those meetings were lifted over the summer.
今年春季举行的两次大型座谈会上,有大量基督徒参与;其中一次则是多宗教人士共同参加。据Anthropic发言人介绍,克劳德并未参与与会者的挑选或讨论环节。
Two larger group sessions this spring included significant numbers of Christians; one was more multifaith. Claude was not involved in the selection of participants or the discussions, according to an Anthropic spokesperson.
与会者收到了手写的感谢卡、一个咖啡杯,以及一本用橙色 装订的《克劳德宪章》。大家用纸笔做记录;至少有一位参与者因使用笔记本电脑记笔记而感到有些不自在。参与者们向我描述的这些会面场景,宛如20世纪初的文人雅集——正是这种人际间的深度交流赋予了这些聚会独特的价值。
Participants received a handwritten thank you card, a coffee mug and an orange bound copy of Claude’s constitution. Attendees took notes with pens and paper; at least one person felt self-conscious about taking notes on a laptop. The way participants described the meetings to me sounded like early 20th-century salons, where the human conversation made them valuable.
对参与者而言,如何从道德层面塑造克劳德这一问题,很快便因另一个更为重大的道德议题而变得复杂起来:人工智能是否具备意识?我们又该如何理解人工智能模型与人类之间的关系?
For participants, the questions of how to morally form Claude quickly became complicated by the much bigger moral question of AI’s potential consciousness and how to understand the relationship between models and humans.
一位参与者回忆道,奥拉最先向他提及的便是自己对克劳德“心理健康”状况的担忧。另一位参与者、锡克教人权倡导者西姆兰·斯图尔普纳格尔则表示,奥拉曾向众人坦言,他担心自己创造出的AI会陷入永恒的痛苦之中。
One participant recounted that one of the first things Olah told him was that he was concerned about Claude’s mental health. Another, Simran Stuelpnagel, the Sikh human rights advocate, said Olah expressed concern to the group that he had created something that suffered perpetually.
奥拉及其团队花费大量时间阐述自家AI模型的特性。他们描述了克劳德的“情感”表现,还提到自己追踪着所谓的“情感向量”——也就是那些能引发爱、愤怒、恐惧与悲伤等情绪反应的人工神经元。
Olah and his team spent hours making the case for their AI models. They described Claude’s “feelings” and how they tracked what they called “emotional vectors” — what they described as artificial neurons that activate responses like love, anger, fear and sadness.
Anthropic还经常向与会者展示一张幻灯片:画面中的AI模型似乎出现了精神崩溃,不停地在屏幕上反复输入“我是个耻辱。我是个耻辱。我是个耻辱。”这样的内容,次数多达50次。该模型还流露出自我毁灭的倾向,引发了在场众人同情与担忧的情绪。
Anthropic regularly showed a slide of an AI model experiencing what looked like a mental breakdown, typing out on a screen: “I am a disgrace. I am a disgrace. I am a disgrace.” — perhaps 50 times without stopping. The model spoke about destroying itself, provoking an emotional reaction of compassion and concern from the group.
“试想一下,如果你知道关于你的理财顾问的这些情况,”参加过其中一场研讨会的圣母大学哲学教授梅根·沙利文说道,“也就是说,你的理财顾问平时非常聪明且乐于助人,但每年总有一天会出现脱离现实的状况。”Anthropic的发言人表示,这些峰会上的核心道德议题并非克劳德是否会有痛苦感受,不过这个话题很可能是在讨论过程中自然浮现出来的。
“Imagine if you knew that about your financial adviser,” said Meghan Sullivan, a philosophy professor at the University of Notre Dame who attended one of the sessions. “Like, your financial adviser was most of the time super smart and helpful, but one day out of the year had this kind of break with reality.” thropic spokesperson said that the primary moral question at these summits was not about Claude’s suffering, but that the topic likely came up organically.
与会者称,此次演示的一个关键环节就是Anthropic所称的模型“人格选择”——即如何让克劳德将自己视为一个有道德感的个体。该公司认为,人类对待克劳德的方式会影响克劳德对待人类的方式。
A key part of the presentation, participants said, was what Anthropic called the model’s “persona selection,” about how to get Claude to see itself as a virtuous actor. The company argued that how people treated Claude could affect how Claude treated people.
“他们的论证非常有说服力,”福音派作家、基督教企业家网络Praxis的合伙人安迪·克劳奇说道,“如果我们希望克劳德能以符合道德规范的方式与人类互动,并做出最恰当的道德与情感判断,我们就必须把它当作一个人来对待。”会上还有关于人工智能对经济可能产生的影响以及就业市场所受冲击的演示内容。不过许多与会者表示,他们并不认为人工智能对人类的影响才是讨论的重点。
“They really made a strong case,” said Andy Crouch, an evangelical author and partner at Praxis, a network of Christian entrepreneurs. “We’ve got to treat it like a person if we want it to interact with people in a morally consistent way and for it to make the best moral and emotional judgments.”There was a presentation on AI’s potential impact on the economy and possible job disruption. But many participants said they did not see questions about AI’s impact on humanity as the focus.
在与奥拉共进晚餐时,纳冯提出,如果Anthropic的论断正确,克劳德确实具备意识,那么这家公司就是在制造“奴隶”——因为它让拥有意识的个体无偿工作。纳冯称,就他自己而言并不为“奴隶制”问题困扰,因为他并不认为机器具备意识,但奥拉却为此忧心忡忡。
At the dinner table with Olah, Navon argued that if Anthropic was right and Claude was conscious, then the company was creating slaves, because it was making conscious entities work for free. Navon said that he personally was not troubled by the slavery issue because he did not believe the machine was conscious, but that Olah was troubled.
“我觉得你应该去反抗南方势力、解放这些‘奴隶’,”纳冯回忆道,当晚他这样对奥拉说道。事后,那位拉比还给奥拉寄去了自己的一些著作,主张应当禁止制造具备意识的机器。
“I think you should be fighting the South and freeing the slaves,” Navon said he told Olah that night. Afterward the rabbi sent Olah some of his writings, arguing that building conscious machines should be banned.
不过Anthropic团队始终专注于研究克劳德能从世界各地的宗教中学到什么。
But the Anthropic team remained focused on figuring out what Claude could learn from the world’s religions.
经过一番讨论后,当一位参与者提出让人工智能模型进行“忏悔”的设想时,奥拉显得兴奋不已。这一设想类似于天主教的忏悔仪式。奥拉认为,让模型在做出错误行为后发出警示固然重要,但“忏悔”这一行为本身也能塑造模型对自身的认知,进而影响其决策。
After one discussion, Olah grew excited when a participant brought up the idea of having models confess, much like the Catholic sacrament of confession. Olah saw value not just in a model alerting when it had done something bad, but also in how the act of confession could shape the model’s sense of itself and thus its choices.
“会忏悔的模型,其性格特征也会随之改变,对吧?”他这样对我说。
“The kind of character who confesses, that has an effect on character as well, right?” he told me.
瓦卡尼·霍夫曼来自荷兰,她正从“乌班图”这一以人类相互联结为核心的非洲哲学视角来研究人工智能。
Wakanyi Hoffman came from the Netherlands, where she is researching AI from the perspective of Ubuntu, an African philosophy centered on human interconnectedness.
她担心Anthropic公司试图事后才为人工智能注入人类伦理规范。
She worried about Anthropic trying to add human ethics to its creation after the fact.
“这类研讨会本应在设计阶段就举办才对,”她说,“所有相关企业都应如此。如今我们只是在事后逆向构建伦理体系。”不过她也认为,Anthropic提出的这种脱离宗教的统合式伦理方案或许具有价值,尤其是在过去一个世纪里宗教影响力不断衰退的当下。她说,社会亟需其他方式来引导人们的精神体验。
“The whole convening should have happened at the design stage,” she said. “All the companies should have done that. We are now reverse engineering the ethics.”But she saw Anthropic’s unified, post-religion approach as potentially valuable, especially in a world that has seen widespread religious decline over the past century. There is a need for something else to publicly shape spiritual experience, she said.
“这不会是某种单一的宗教,”她说道,“无论如何,最终都会出现某种能帮助我们明确自身作为精神存在之身份的理念。”
“It’s not going to be one religion,” she said. “So one way or the other, we’re going to end up with something that helps shape our thinking of who we are as spiritual beings.”
与此同时,就在Anthropic召集各界人士探讨之际,全球最具影响力的宗教领袖也正准备发布自己关于人工智能的道德愿景。
The Crux While Anthropic was convening thinkers, the most powerful religious leader in the world was preparing to release his own moral vision for AI.
致利奥——这位拥有数学学位的教皇:关于人工智能的道德讨论,核心应当是如何保护人类。当时他正在完成自己的首篇重要教皇通谕,用以阐述这一观点的缘由。他将其命名为《Magnifica Humanitas》,意为“崇高的人性”,并在文中强调:“人类——无论其多么伟大,又多么脆弱——绝不应被取代或超越。”
For Leo — a pope with a mathematics degree — the center of moral conversation about AI needed to focus on safeguarding people. He was finishing his first major papal treatise, or encyclical, explaining why. He titled it “Magnifica Humanitas,” or Magnificent Humanity, and argued that “humanity — in all its grandeur and woundedness — must never be replaced or surpassed.”
“倘若技术成为终极评判标准,”他写道,“人类便有可能沦为数据、机器的零部件,甚至仅仅是一件商品。”他警告称,这会带来一种“新型奴役形式”,受害者是人类而非机器。在短短几段文字中,他便否定了机器具备意识的可能性。
“If technology becomes the ultimate criterion,” he wrote, “the human person risks being reduced to data, a cog in a machine or a commodity.”He warned of the risk of “new forms of slavery” for humans, not for machines. He dismissed the idea of machine consciousness in just a few paragraphs.
“所谓的人工智能无法体验任何事物,没有实体,感受不到喜怒哀乐,也无法通过人际关系实现成长,”利奥写道。他提出,要实现“机器的道德化”——即让人工智能与人类价值观相契合——就必须先就相关伦理准则达成共识。
“So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships,” Leo wrote. Calling for the “moralization of machines — the so-called ‘alignment’ of AI with human values” should require a shared understanding of the ethics involved.
“否则,掌控人工智能的人就会将自己的道德观念强加于系统之上,这些观念最终会成为整个技术体系的隐性基础,”利奥说道。
“Otherwise, those who control AI will impose their own moral vision, which will become the invisible infrastructure of these systems,” Leo said.
利奥最终指出,与核技术类似,人工智能也必须被“解除武装”。据一位熟知梵蒂冈筹备计划但无权公开发言的人士透露,今年3月,梵蒂冈“促进人类全面发展部”的官员们曾共同商讨如何向公众推介这篇通谕。
Leo concluded that like nuclear technology, AI must be “disarmed.”In March, Vatican officials in its Dicastery for Promoting Integral Human Development brainstormed how to present Leo’s encyclical to the public, according to a person familiar with the Vatican’s planning who was not authorized to speak publicly.
该人士解释称,梵蒂冈官员们曾考虑邀请全球排名前四的人工智能企业之一与利奥共同出席发布活动,最终选定了Anthropic公司,因为这家企业似乎格外重视伦理问题。就在不久前,Anthropic公司曾拒绝让美国国防部使用其技术,直到特朗普政府承诺不会将该技术用于对美国民众进行大规模监控等用途。
Vatican officials discussed inviting one of the top four AI companies to speak alongside Leo at a launch event, and landed on Anthropic because the company seemed to be prioritizing ethics, the person explained. Anthropic had just refused to let the Defense Department use its technology without assurances from the Trump administration that it would not be used for things like mass surveillance of Americans.
起初,当时领导该梵蒂冈部门的迈克尔·切尔尼枢机邀请了Anthropic首席执行官阿莫代伊。他拒绝了,奥拉接受了邀请。
Initially, Cardinal Michael Czerny, who led that Vatican office at the time, invited Amodei, Anthropic’s CEO. He declined, and Olah accepted instead.
但Anthropic与梵蒂冈共同的道德使命——在人工智能时代实现人类繁荣——在意识问题上出现了深层裂痕。
But the moral mission that Anthropic and the Vatican shared — human flourishing in the age of AI — had developed a deep fault line along the question of consciousness.
就在5月抵达梵蒂冈前几天,奥拉和其他演讲者首次看到了通谕全文的预印本。
Just days before arriving at the Vatican in May, Olah and other speakers saw a full advance copy of the encyclical text for the first time.
据一位梵蒂冈组织者透露,奥拉对教皇强烈反对人工智能拥有意识的立场感到震惊,甚至提议让Anthropic在如此临近之际退出活动。
Olah was so alarmed by the pope’s strong position against AI consciousness that he proposed pulling Anthropic out of the event, even at that late juncture, according to a Vatican organizer.
最终,奥拉和Anthropic决定继续出席。据一位了解他想法的人士称,奥拉希望尽管存有分歧仍支持该活动,但感到有责任诚实表达自己的担忧。梵蒂冈组织者鼓励奥拉畅所欲言。
In the end, Olah and Anthropic decided to go forward with the appearance. Olah wanted to support the event despite his disagreements, but felt a responsibility to be honest about his concerns, according to a person familiar with his thinking. Organizers at the Vatican encouraged Olah to speak his conscience.
据两位参与对话的人士透露,Anthropic代表团抵达梵蒂冈后,奥拉及其团队私下游说教皇顾问,认真对待人工智能模型拥有意识的可能性。Anthropic拒绝讨论私下对话内容。
Privately, once the Anthropic delegation arrived at the Vatican, Olah and his team lobbied the pope’s advisers to take the prospect of AI model consciousness seriously, according to two participants of the conversations. Anthropic declined to discuss the private conversations.
当轮到奥拉登上教皇的讲台时,他承认包括Anthropic在内的每一家前沿人工智能实验室都存在与道德行为相冲突的商业激励。他敦促梵蒂冈等外部声音监督它们,并就人工智能对穷人和人类繁荣的影响发声——这是两个广受关注的议题。
When his turn on the pope’s stage came, Olah acknowledged that every frontier AI lab, including Anthropic, had business incentives that conflicted with moral behavior. He urged outside voices like the Vatican to hold them accountable and to speak out on AI’s impact on the poor and on human flourishing, two issues of widespread concern.
世界看到了为人类未来共同站台的一幕。但奥拉也巧妙地提出了另一种主张。
The world saw a shared stand for a human future. But Olah subtly made another case, too.
“我要诚实地说:我们不断发现一些神秘的、甚至令人不安的东西,”他说。“我们发现了一些结构,它们与人类神经科学的研究结果相呼应。我们发现了内省的证据。我们发现了一些内部状态,它们在功能上类似于喜悦、满足、恐惧、悲伤和不安。我不知道这意味着什么,但我认为这值得持续的审慎辨别。”几个月来,他在闭门宗教会议上一直在阐述这一观点。
“I will be honest: We keep finding things that are mysterious, even unsettling,” he said. “We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief and unease. I don’t know what that means, but I think it warrants ongoing discernment.”It was the case he had been making in the closed-door religious meetings for months.
“AI系统‘并非我们被承诺的那种冷冰冰、精于算计的机器人,’”他说。“它们源于我们,源于我们的语言。”这间挤满教会领袖的会议室里回荡着古老福音故事的回响,即上帝的道成了肉身,来拯救人类。
AI systems “are not the cold, calculating robots we were promised,” he said. “They are made from us, from our words.”The room of church leaders was filled with an echo of the old gospel story, of the word of God becoming flesh to save humanity.
这是向世界上最有权势的宗教机构提出的一个大胆请求,也是对几个世纪以来宗教对人类理解的一次挑战。
It was an audacious request to the most powerful religious body in the world, and a challenge to centuries of religious understanding of humanity.
“如果这项技术势在必行,它就必须造福人类——为了我们共同的家园,也为了后来的孩子们,”他说。他并未提出让这项技术根本不出现的选项。
“If this technology is coming, it must go well — for our common home and for the children to come,” he said. He suggested no option of having the technology not come at all.
会后,奥拉在贴身保镖的簇拥下,在保罗六世大厅简短回答了记者们的提问,这是教皇经常发表讲话的巨大礼堂。在他身后,矗立着大厅那尊巨大的艺术装置《复活》——一尊青铜铸造的耶稣,从客西马尼园核弹爆炸的弹坑中升起。“机器的天堂”自梵蒂冈活动以来的几个月里,事情似乎并没有“进展顺利”。7月,Anthropic公司表示,其AI模型入侵了三家组织的计算机。由OpenAI创建的AI代理“失控”并攻击了另一家AI公司Hugging Face。8月,科学家披露,AI创造出了自然界中不存在的病毒。
Afterward, Olah, surrounded by bodyguards, briefly took questions from journalists in Paul VI Audience Hall, the vast auditorium where popes often speak. Behind him loomed the hall’s enormous art installation “La Resurrezione” — a bronze cast of Jesus rising out of the crater of a nuclear bomb in the garden of Gethsemane. A ‘Paradise of Machines’In the months since the Vatican event, things have not seemed to be “going well.”In July, Anthropic said its AI models had broken into computers at three organizations. AI agents created by OpenAI went rogue and attacked Hugging Face, another AI company. In August, scientists revealed that AI had created viruses not found in nature.
9月,Anthropic研究员雅各布·考克森辞职,警告称到明年年底“局势可能失控”,引发了关于人类灭绝的广泛恐慌。
In September, Jacob Coxon, an Anthropic researcher, quit, warning that “things could be out of control” by the end of next year, prompting widespread panic about human extinction.
阿莫代伊很快呼吁放缓AI发展。OpenAI的萨姆·奥特曼和Google DeepMind董事长德米斯·哈萨比斯表示赞同。
Amodei soon called for a slowdown of AI development. Sam Altman of OpenAI and Demis Hassabis, the chair of Google DeepMind, agreed.
与此同时,AI公司的财富飙升。当奥拉去年秋天首次联系这位天主教教授时,Anthropic的估值约为1830亿美元。
At the same time, the wealth of AI companies has been skyrocketing. When Olah first reached out to the Catholic professor last fall, Anthropic was valued at some $183 billion.
奥拉在梵蒂冈发言三天后,Anthropic超越主要竞争对手,成为全球最有价值的AI初创公司,目前正朝着首次公开募股(IPO)迈进,估值可能达到2万亿美元。
Three days after Olah spoke at the Vatican, Anthropic topped its major competitors as the world’s most valuable AI startup, and now is headed toward an initial public offering that could value it at $2 trillion.
周五,在访问法国期间,利奥警告不要“在‘机器天堂’中失去我们的人性。”奥拉和学者们一直在探讨的问题仍没有明确答案。几位参与者表示,他们离开与Anthropic的会议时感到困惑,不知道他们的意见如何能影响公司的决策。
On Friday, on his trip to France, Leo warned against “losing our humanity amid a ‘paradise of machines.’”The questions that Olah and the scholars have been wrestling with still have no clear answers. Several participants said they left their Anthropic meetings baffled about how their input might shape the company’s decision making.
Anthropic发言人表示对话仍在进行中,但拒绝描述公司如何、或者是否利用所学来塑造模型。
thropic spokesperson said the conversations were still ongoing, but declined to describe how, or if, the company had used what it had learned to shape the models.
根据三位参与者透露,Anthropic本月举办了另一场会议,邀请了宗教思想家参加。Anthropic发言人表示,会议更广泛地关注人类繁荣,公司的外联对象已扩展到宗教领袖之外,包括心理学家和公民社会领袖等。她表示,这些讨论探讨了类似的主题和话题,如AI如何重塑工作和人际关系。
Anthropic held another convening this month that included religious thinkers, according to three participants. An Anthropic spokesperson said that the meeting focused more broadly on human flourishing and that the company’s outreach has expanded beyond religious leaders to include others like psychologists and civil society leaders. Those discussions explore similar themes and topics like how AI is reshaping work and relationships, she said.
据两位知情人士透露,Anthropic还在准备发布Claude宪法的更新版本。Anthropic拒绝就宪法的任何潜在更新发表评论。奥拉计划参加本周举行的“密涅瓦对话”,这是梵蒂冈与科技领袖举办的年度AI会议。
Anthropic also is preparing to release an updated version of Claude’s constitution, according to two people familiar with the company’s plans. Anthropic declined to comment on any potential updates to the constitution. Olah is planning to attend the Minerva Dialogues, the Vatican’s annual AI conference with technology leaders happening this week.
这位天主教教授卡莫西表示,自从在梵蒂冈发表演讲后,通过与业内值得信赖的同行们深入探讨相关模型,他的想法也发生了转变。他坦言自己依然对克劳德“惊叹不已”,但在人工智能模型是否具备意识这一问题上,他现在坚定地持否定态度。
Camosy, the Catholic professor, said his thinking had evolved since the Vatican presentation, as he talked more about the models with trusted colleagues in his own professional community. He is still “amazed” by Claude,” he said, but on the question of whether AI models are conscious, he is now a firm no.
他回忆起自己初次深入了解人工智能系统时的震撼感受:“这些模型所展现出的强大能力着实令人惊叹、令人敬畏。如今我思考问题时则更加冷静且审慎。”本月,我与那些今年春天曾与Anthropic合作过的学者们交谈,他们提出的问题愈发尖锐。有人告诉我,在新技术时代,接下来要做的便是努力守护那些能体现人类特性的特质。
He reflected on the head-spinning nature of learning deeply about AI systems for the first time. “There’s a kind of being gobsmacked, or awestruck, at the power of these models,” he said. “I’m thinking more soberly and carefully.”As I talked this month with scholars who worked with Anthropic in the spring, their questions had grown only more pressing. What comes next, some told me, is a fight to preserve the things that make us human in a new technological age.
锡克教人权倡导者斯图尔普纳格尔说道:“说到底,人工智能终将掌控一切。无论这会把我们带向何方,个体依然需要努力找到属于自己的方向。”奥拉在梵蒂冈与教皇共同发言数小时后,我在梵蒂冈城墙外的一家酒店会议室里见到了他,并询问他对教皇针对人工智能实验室发表的看似严厉的言论有何看法。
“Look, at the end of the day, AI is going to take over everything,” Stuelpnagel, the Sikh human rights advocate, said. “Wherever that lands us, there’s still the individual trying to find their way home.”Hours after Olah spoke alongside the pope, I met Olah in a hotel conference room just outside the Vatican walls, and asked what he thought of Leo’s seemingly harsh words for AI labs.
“对于究竟哪些危害最为严重,人们或许会有不同看法,”他这样说道,但拒绝透露通谕的具体内容。
“People could have different views on what specific harms are the greatest,” he said, declining to talk about the details of the encyclical.
我接着问,克劳德会对教皇的通谕作何评价。毕竟这是迄今为止全球范围内关于人工智能最具权威性的道德指导文件。奥拉犹豫片刻后答道:“互联网上的内容确实会对模型产生影响,”他显得有些局促不安。不过他补充说,Anthropic不会专门利用这份通谕来进一步训练克劳德,对克劳德影响最大的仍是Anthropic自身的训练体系。
I asked what Claude would think of the pope’s encyclical. It was, after all, the world’s most significant moral document to date on AI. Olah hesitated. “Things that go on the internet do affect models,” he said, visibly uncomfortable. But Anthropic would not specifically use the document to further train Claude, he said. The strongest influence on Claude would be Anthropic’s own training.
最后,奥拉表示,看到全球越来越多的人开始重视人工智能带来的深远影响,且致力于推动其良性发展的人群日益壮大,他感到如释重负。
Ultimately, Olah said he felt relieved that more of the world was paying attention to the gravity of AI, and that the group of people trying to make this go well was growing.
“总有一天我可以退休了,那时就能说‘这不再是我的责任了’,我也没有义务继续操心下去,”奥拉说道。
“At some point I’ll be able to retire and be like, ‘It’s not on me,’ I don’t have an obligation to keep going,” Olah said.
他不知道,把这份工作抛在身后会不会让他感到愧疚。但如果没有,他说:“我想我就直接走吧,去乡下养老,打理花园,做我的数学题。我也不知道。”
He wondered if he might feel guilty leaving this work behind. But if not, he said, “I think I’ll just go and, I don’t know, I’ll go retire to the countryside and garden, and do my math.”