字号·· | 护眼
纽约时报

Anthropic的IPO将改变人工智能的未来The Anthropic I.P.O. Will Change the Future of A.I.

点「原文对照」整页切到原文,或双击某段只看那段的原文。

我花了一年时间与Anthropic的员工交谈。以下是他们的信念。

I Spent a Year Talking to Anthropic Employees. Here’s What They Believe.

想象一家人工智能公司即将启动史上规模最大的IPO。背后是无数个不眠的星期、充满漏洞的训练运行、多年在没有产品和利润的情况下争取投资者和客户的煎熬。它将不再是一家拼劲十足的初创公司,而是整个硅谷艳羡的对象。你或许会期待如释重负与庆祝,香槟四溢的盛会,锃亮的跑车驶入公司停车场。

Imagine an artificial intelligence company that is about to launch the biggest I.P.O. in history. Behind it are the sleepless weeks, the training runs full of bugs, the yearslong grind of wooing investors and customers without a product or profit. It will no longer be a scrappy start-up but the consummate envy of Silicon Valley. You might expect relief and celebration, champagne-drenched galas and shiny sports cars rolling into the office parking lot.

在Anthropic的投资者路演上,确有这种欢腾的迹象。为Anthropic叩开公开市场大门的华尔街银行家们大力宣传一场与首席执行官达里奥·阿莫代伊的对话,配以电子音乐制作人Fred Again的音乐,房间里装点着充满未来感的Anthropic周边。但在Anthropic总部,气氛并非一片欢腾。

On Anthropic’s investor roadshow, there are hints of such jubilance. The Wall Street bankers ushering Anthropic into the public markets hyped a conversation with the chief executive, Dario Amodei, with music by the electronic producer Fred Again and a room adorned with futuristic Anthropic swag. But euphoria is not the mood back at Anthropic headquarters.

在首次公开募股前的几周里,Anthropic的员工——他们亲切地自称“蚂蚁”——一直忧心于如何放慢人工智能的发展。在最近一连串失控智能体事件之后,整个行业比以往任何时候都更显末日气息。

In the weeks leading up to the initial public offering, employees of Anthropic — who affectionately call themselves “Ants” — have been preoccupied with concerns about how to slow down A.I. Following a recent spate of rogue agent episodes, the industry has felt doomier than ever.

即便最热忱的技术乐观主义者也收敛了欢呼。群聊和聚会转而讨论如何把控前沿的节奏:领先的人工智能实验室怎样才能停止竞赛,就模型开发的速度达成合作,确保它们在向世界发布模型之前有时间让模型变得安全。

Even the most ardent techno-optimists have quieted their cheers. Group chats and gatherings instead debate pacing the frontier: how the leading A.I. labs can stop racing and agree to cooperate on the speed of model development, ensuring they have time to make the models safe before releasing them to the world.

对许多“蚂蚁”来说,艰难的部分才刚刚开始。他们看到递归自我改进正迫在眉睫,即当前的人工智能模型自主构建下一代模型。由于这一潜在突破,Anthropic的联合创始人杰克·克拉克以及独立预测机构“人工智能未来项目”都提前了超人类人工智能研究员的可能到来日期。他们现在预计是在2027年。

For many Ants, the hard part is just beginning. They see recursive self-improvement, in which current A.I. models autonomously build the next generation, looming on the horizon. Because of this potential breakthrough, Anthropic’s co-founder Jack Clark as well as the independent forecasting group A.I. Futures Project have moved up the likely arrival date for superhuman A.I. researchers. They now anticipate it being in 2027.

一旦人工智能模型能够制造其继任者,它们就更有可能逃脱人类的控制。没有人为这个未来做好准备,但也没有人停止——恰恰相反。Anthropic将其首次公开募股资金的首要支出优先事项,定为购买更多算力,以便在竞争对手之前实现递归自我改进。

Once A.I. models can build their successors, they are even more likely to escape human control. Nobody is ready for this future, but nobody is stopping — rather the opposite. Anthropic’s first spending priority with its I.P.O. cash will be buying more compute to reach recursive self-improvement before its rivals do.

过去两年里,我与数十名“蚂蚁”交谈过,从公司创始人到年轻研究人员,再到厌倦的前员工。我发现,他们致力于让人工智能发展良好,态度出乎意料地认真。但他们的担忧也十分独特且封闭,往往与大多数人对人工智能的担忧相去甚远。

I have spoken to dozens of Ants during the past two years, from company founders to young researchers to jaded ex-employees. I have found them surprisingly earnest in their dedication to making A.I. go well. But their concerns are also idiosyncratic and insular, often far removed from the concerns that most people hold about A.I.

他们不把Anthropic视为一家普通的营利性公司。因此,随着美国经济和我们个人的财务命运与“蚂蚁”们捆绑在一起,了解他们的信念至关重要。

They do not view Anthropic as a normal for-profit company. So as the U.S. economy and our personal financial fates become yoked to the Ants, it’s critical that we understand what they believe.

与他们认为即将到来的超级智能形成对比,即将到来的发薪日在许多Anthropic员工看来显得狭隘。一些员工对首次公开募股不以为然,将其比作又一轮融资。

Juxtaposed with what they presume will be the imminent arrival of superintelligence, the upcoming payday seems parochial to many at Anthropic. Some employees describe the I.P.O. with a shrug, likening it to just another funding round.

对于一群年轻人来说,这是一种奇特的态度,而他们中的许多人即将在一夜之间成为千万乃至亿万富翁。Anthropic的员工并非普通的科技从业者。他们从早到晚坐在旧金山市中心一间简陋且安保严密的办公室里的一排排桌前,桌上常见的书籍包括理查德·罗兹的《原子弹的诞生》,以及公司内部读物的平装打印本,比如阿莫代伊的《充满爱意的机器》和内部哲学家阿曼达·阿斯凯尔的《克劳德的宪法》。

This is a peculiar attitude for a crowd of young people who, in many cases, are about to become deca- and centimillionaires overnight. The Ants are not your average tech workers. They spend morning to night at rows of desks in the bare, high-security offices in San Francisco’s downtown, where the common books on display include Richard Rhodes’s “The Making of the Atomic Bomb” and paperback printouts of company writings, such as Mr. Amodei’s “Machines of Loving Grace” and the in-house philosopher Amanda Askell’s “Claude’s Constitution.”

没有人被强制要求执行科技行业已司空见惯的“996”工时(早9点到晚9点,每周六天),但相当多的Anthropic员工出于自己狂热的投入仍然这样做。许多人挤在普通的合租公寓里,举办家宴时几乎没人喝酒;当我问他们如何花掉丰厚的薪水时,他们一脸茫然,即使我给出提示:“也许更新一下衣橱?”“考虑过坐商务舱吗?”钱是次要的。最常见的IPO话题,是讨论如何将新近可变现的股票分配到他们的慈善捐赠账户中。

No one is compelled to work the grueling “996” hours (9 a.m. to 9 p.m., six days a week) that have become common in the tech industry, but a good number of Ants do so anyway out of their own feverish devotion. Many cram themselves into modest group apartments, host house parties where few, if any, drink and blank when I ask how they spend their lavish salaries, even with my prompts: “A wardrobe update, maybe?”“Ever considered flying business class?”The money is secondary. The most common form of I.P.O. talk involves discussions of where to allocate the newly liquid stock in their charitable donation accounts.

作为一家公司,Anthropic深受有效利他主义的影响,这种理念认为富人有义务与世界分享自己的成果。新员工入职时会被邀请承诺将一定比例的股权捐给慈善机构,Anthropic会配捐其中一部分。主要捐赠对象包括致力于动物福利、全球健康以及人工智能对齐与安全研究的慈善机构。许多人已开始向政治竞选大额捐款。

As a company, Anthropic has been strongly influenced by effective altruism, which holds the belief that the wealthy are obliged to share their fruits with the world. New hires are invited to pledge a percentage of their equity to charity when they start, which Anthropic matches part of. Top targets include charities dedicated to animal welfare, global health and research on A.I. alignment and safety. Many have begun giving big to political campaigns.

至于他们自己的股票,我交谈过的Anthropic员工中很少有人计划在IPO后立即出售。“人工智能泡沫”的概念在Anthropic似乎并不存在。员工们预计自己的股权最终会值多得多。

As for their own stock, few of the Ants I spoke to planned to sell right after the I.P.O. The notion of an “A.I. bubble” seems not to exist at Anthropic. Employees expect their equity to eventually be worth much, much more.

Anthropic清晰地体现了当代人工智能令人费解的悖论。那些在拓展人工智能能力方面贡献最大的研究科学家,往往也最坚持强调其风险。Anthropic是在大规模自动化等经济影响方面发声最响亮的公司,但它同时也是人工智能企业级代理的主要供应商,而这些代理有可能取代白领工作者。

Anthropic crisply embodies the mind-rending paradox of contemporary A.I. The research scientists who’ve done the most to extend A.I. capabilities are often the most insistent about its risks. Anthropic is the most vocal company when it comes to economic impacts such as mass automation, yet it’s also the leading vendor of the A.I. enterprise agents that threaten to replace white-collar workers.

这种立场看起来可能虚伪且自私自利,既方便又绕来绕去。Anthropic的IPO招股书中有一句话令人翻白眼:“我们对世界产生积极影响的能力,与我们作为一家成功公司的覆盖范围和地位成正比。”公司领导层对风险直言不讳,但对解决方案却没那么坦率。当被问及为什么要孕育一种他们怀疑具有存在性危险的技术时,他们的回答本质上就是:相信我们吧,因为我们是好人。

Such a stance can seem duplicitous and self-serving, both convenient and convoluted. “Our ability to have a positive impact on the world scales in relation to our reach and stature as a successful company,” reads one eye-roll-inducing statement in Anthropic’s I.P.O. prospectus. While the company’s leaders have been candid about the risks, they have been less forthcoming about their solutions. When asked why they should birth what they suspect is an existentially dangerous technology, their answer amounts to, essentially, just trust us because we’re the good guys.

Anthropic即将成为21世纪最重要的公司之一。如果它想管理我们的国家、我们的经济和我们的生活,Anthropic就必须做更多努力来赢得公众的信任。我们需要一份关于它将如何安全地构建人工智能的计划,而不能只依赖少数人的判断。

Anthropic is about to become one of the most important companies of the 21st century. If it wants to steward our country, our economy and our lives, Anthropic must do much more to earn the public’s trust. We need a plan for how it will build A.I. safely, and not just reliance on a few people’s judgment calls.

阿莫代伊似乎坚信两点:第一,强大的人工智能可能毁灭人类。第二,我们唯一的生存机会,就是创建一家极其成功、吞噬市场的科技公司,抢先构建出这种人工智能。

Mr. Amodei seems to hold two convictions very strongly: First, that powerful A.I. might destroy humanity. Second, that our only shot at survival is to create a supremely successful, market-eating technology company to build that A.I. first.

Anthropic过去一直坚称自己并不争夺第一名。在2022年的一档播客中,阿莫代伊表示,Anthropic“不应该抢跑,也不应该试图构建比其他机构大得多的模型”。他反而暗示,公司将以人工智能发展的快速跟随者身份开展安全研究。一些投资者表示,他当时告诉他们,公司打算研究这项技术,但不推进能力前沿。

Anthropic used to insist that it was not gunning for first place. In a 2022 podcast, Mr. Amodei said Anthropic “shouldn’t be racing ahead or trying to build models that are way bigger than other orgs.”He suggested instead that the company would do safety research as a fast-follower of A.I. development. The company intended to study the technology without pushing capabilities ahead, some investors said he told them at the time.

2022年夏天,Anthropic因内部对安全问题的担忧,决定不向公众开放其人工智能聊天机器人Claude。这让OpenAI——其已获悉Anthropic的内部原型——得以设计类似产品,抢先推出ChatGPT,并在消费者认知度上取得决定性领先。

In the summer of 2022, Anthropic decided not to make Claude, its A.I. chatbot, available to the public over internal concerns about safety. This allowed OpenAI — which had gotten wind of Anthropic’s internal prototype — to design a similar product, rush ChatGPT out the door first and gain a decisive lead in consumer mindshare.

在Anthropic的公司传说中,这段往事被奉为选择原则而非利润的典范。但它可能也是对阿莫代伊的一次战略警醒,他称这一牺牲是“一次性”的。阿莫代伊似乎越来越清楚地看到,成为人工智能竞赛的领导者如何在收入、招聘、融资和开发下一代人工智能模型方面带来复利式的好处。一旦商业人工智能产品已经推出,阿莫代伊似乎就不愿甘居第二。

In Anthropic company lore, this saga is put on a pedestal as an example of choosing principle over profit. Yet it may have also been a strategic wake-up call for Mr. Amodei, who called the sacrifice a “one-off.”Increasingly, Mr. Amodei seemed to see how being the leader in the A.I. race conferred compounding benefits in terms of revenue, recruiting, fund-raising and developing the next-generation A.I. models. Once commercial A.I. products had been introduced, Mr. Amodei seemed unwilling to settle for second place.

Anthropic在竞赛方面无疑非常擅长。尽管它在初创岁月里奋力复制OpenAI的语言和推理模型,但此后Anthropic在编程智能体方面走在了前列。

Anthropic is certainly very good at racing. Although it spent its toddler years scrambling to replicate OpenAI’s language and reasoning models, Anthropic has since led the charge on coding agents.

说Claude Code重塑了软件工程这门学科并不夸张。程序员过去逐行手写代码,如今许多人用通俗英语指挥成群的AI智能体。自Claude Code发布以来,Anthropic的季度收入已超过OpenAI,在2026年第三季度飙升至170亿美元。如今,《财富》500强中超过一半是Anthropic的六位数客户。

It is not an exaggeration to say that Claude Code reinvented the discipline of software engineering. Where programmers used to write lines of code by hand, many now use plain English to command fleets of A.I. agents. Since the release of Claude Code, Anthropic’s quarterly revenue has surpassed OpenAI’s, surging to $17 billion in the third quarter of 2026. Today, more than half of the Fortune 500 are Anthropic’s six-figure customers.

凭借这样的成功,阿莫代伊更新了他的变革理论——他称之为“向上竞赛”。他认为,只有打造最先进的人工智能系统,Anthropic才能以更安全的方向影响整个行业。这一战略内含三个基本假设。

With this kind of success, Mr. Amodei has updated his theory of change — he calls it a “race to the top.”Only by building the most advanced A.I. systems can Anthropic influence the whole industry in a safer direction, he argues. Three underlying assumptions are baked into this strategy.

第一,Anthropic假定人工智能竞赛本身不可避免,因此没有任何单一行为者能够阻止超级智能被建造出来。建造一台能够匹敌或超越人类认知能力的机器,其经济回报如此巨大,以至于必定会有人去追求它。如果不是阿莫代伊和他的团队,那就是萨姆·奥尔特曼,或者马克·扎克伯格,或者埃隆·马斯克。

First, Anthropic assumes the A.I. race itself is inevitable, so no single actor can stop superintelligence from being built. The financial rewards for building a machine that can match or exceed humans’ cognitive abilities are so great that someone is guaranteed to pursue it. If not Mr. Amodei and his team, then Sam Altman, or Mark Zuckerberg or Elon Musk.

第二,Anthropic假定自己是人工智能可能的守护者中最负责任的一个。在公开场合和内部,阿莫代伊喜欢把OpenAI和中国政府当作竞争中的假想敌。它们被视为在构建强大人工智能模型并将其商业化方面与Anthropic同样卖力的行为者,但会远不如Anthropic负责任——往好了说是鲁莽,往坏了说是威权。从近期的证据来看,最严重的失控智能体黑客事件确实出自OpenAI。而如果你认为Claude是道德价值观最好的人工智能,那么扩大Claude的市场份额本身就可以被视为一种美德。

Second, Anthropic assumes it is the most responsible of A.I.’s possible stewards. In public and internally, Mr. Amodei likes to invoke OpenAI and the Chinese government as competitive boogeymen. These are viewed as actors that are pushing just as hard as Anthropic on building and commercializing powerful A.I. models, but that would be far less responsible — reckless at best, and authoritarian at worst. Looking at the recent evidence, the most severe rogue agent hacks have indeed come out of OpenAI. And if you think Claude is the A.I. with the best moral values, then expanding Claude’s market share could be construed as a virtue in itself.

第三,Anthropic假定人工智能的“赢家”将对人工智能生态系统和美国政府获得巨大影响力。通过商业上的成功,Anthropic旨在向业界表明,注重安全可以对生意有利、对科学有利,也有利于吸引顶尖人才。

Third, Anthropic assumes the “winner” of A.I. will gain tremendous influence over the A.I. ecosystem and the U.S. government. Through commercial success, Anthropic aims to show the industry that being safety-minded can be good for business, good for science and good for attracting top talent.

一些Anthropic员工提到了一些举措,例如将人工智能安全审计员嵌入公司的工程团队——这一决定促使竞争对手做出了类似的承诺。此外,该公司表示将利用其商业实力倡导人工智能监管,正如它在争取加州S.B. 53法案时所做的那样。该法案提高了对领先人工智能公司的透明度要求。

Some Anthropic staffers point to initiatives like embedding A.I. safety auditors with the company’s engineering teams — a decision that has inspired competitors to make similar commitments. Additionally, the company has said it will use its commercial strength to advocate A.I. regulation, as it did in the fight for California’s S.B. 53, which increased transparency requirements for leading A.I. companies.

Anthropic的“竞争至顶”论调听起来是正和博弈,但其世界观中也包含一份现实政治色彩。未来,它可能会将对前沿人工智能系统的访问权作为交易中的硬杠杆。“达里奥谈到,Anthropic可能会比政府更强大,并需要与国家建立自己的关系,”一名前员工表示,该员工要求匿名,以便讨论内部 deliberations。

Anthropic’s race-to-the-top argument sounds positive-sum, but its worldview also contains a dose of realpolitik. In the future, it may use access to frontier A.I. systems as hard leverage in deals. “Dario talks about how Anthropic might become more powerful than the government, and needs to have its own relationship with nation-states,” said one former employee, who remained anonymous in order to discuss internal deliberations.

一名前Ant引用J.R.R.托尔金的寓言,试图解释Anthropic的心态:谁先建立超级智能,谁就会得到索伦的魔戒。那个人或公司将拥有世界上最危险的网络武器、每种疾病的治疗方法和配方,以及一支不知疲倦的数字天才军队来执行其命令。那么,最关键的问题是,持戒者的道德。 (当然,在中土世界,魔戒被摧毁了,因为即使是最纯洁的灵魂也无法使其安全。)

One former Ant referenced J.R.R. Tolkien’s parable in trying to explain Anthropic’s mind-set: Whomever builds superintelligence first will get Sauron’s ring. That person or company will possess the most dangerous cyberweapons in the world, the cures and recipes for every disease, and an army of tireless digital geniuses to do its bidding. The most load-bearing question, then, is the morality of the one who wields the ring. (Of course, in Middle Earth, the Ring was destroyed because even the purest soul couldn’t make it safe.)

那么,Anthropic为何相信自身的力量不会腐蚀?该公司以代价高昂的信号为傲:做出牺牲短期利润的选择,无论是禁止中国实体购买其产品;将5%的股份放入“公共利益基金”,用于向其选择的传统非营利组织进行“大额捐赠”或向特定人群进行“直接分配”;或拒绝允许国防部使用Claude进行国内大规模监控,这一举动激怒了特朗普政府,并曾短暂地将Anthropic标记为供应链风险,时间是去年冬天。

And why is Anthropic confident that its own power won’t corrupt? The company prides itself on costly signals: making choices that trade off against short-term profits, whether prohibiting Chinese entities from buying its products; placing 5 percent of its shares into a “Public Benefit Fund” to spend on “large donations to traditional nonprofits or direct distributions to specific populations” of its choice; or refusing to permit the Department of Defense to use Claude for domestic mass surveillance, an act that infuriated the Trump administration and briefly got Anthropic marked as a supply-chain risk last winter.

另一名前Anthropic员工解释了阿莫代如何经常为有争议的决定辩护,例如接受波斯湾国家的投资。这笔钱,就像流入Anthropic的大部分投资一样,将用于资助其冲刺。这位前Ant回忆说,阿莫代表示,他的目标不是赢得普通美国人的喜爱,而是做正确的事。

Another former Anthropic employee explained how Mr. Amodei often justified controversial decisions, such as taking investments from Persian Gulf countries. This money, like most of the investment flowing into Anthropic, would be used to fund its sprint. The former Ant recalled Mr. Amodei saying that his goal was not to be popular with average Americans, but to do the right thing.

Anthropic已警告投资者,这些决定可能会随着时间推移而变得更加昂贵。正如其在首次公开募股招股说明书中所详述的那样:“我们的产品和投资决策可能不会优先考虑短期财务业绩”,而这“可能会对公司股票的市场价格产生不利影响”。

Anthropic has warned investors that these decisions may get more costly over time. As it details in its I.P.O. prospectus: “Our product and investment decisions may not prioritize short-term financial results,” and this “could adversely affect the market price” of the company’s stock.

公司希望保留这样一种选择权:一旦评估认为风险过大,就可以从最大化股东价值转向放缓人工智能的发展——这种策略有时被称为“烧掉领先优势”。我交谈过的大多数前员工都认为阿莫代伊先生原则性极强,愿意为安全作出牺牲。但他们也承认,快速增长以及推动增长的经济势头,可能让即使是最有良知的创始人也难以驾驭。

The company wants to preserve the option to pivot from maximizing shareholder value to A.I. slowdowns once it assesses that the risks are too great — a strategy sometimes called “burning a lead.”Most former employees whom I spoke to believed that Mr. Amodei was deeply principled and would be willing to make sacrifices for safety. But they also acknowledged that rapid growth, and the economic momentum that propels it, can outrun even the most conscientious of founders.

Anthropic以其坚定的团队精神著称,美国证券交易委员会注册表格S-1将其列为核心竞争优势。优秀的“蚂蚁”尽职尽责、善于自我审视,并且完全相信人工智能具有世界历史性的重要意义。一个臭名昭著的面试问题会问求职者:如果因为安全方面的放缓导致公司股票归零,他们对于在Anthropic工作会作何感想。研究团队会欣然拒绝那些缺乏文化契合度的技术明星。

Anthropic is known for its committed esprit de corps, which the S-1 Securities and Exchange Commission registration form lists as a core competitive advantage. Good Ants are dutiful, self-scrutinizing and total believers in A.I.’s world-historic importance. One notorious interview question asks employment candidates how they’d feel about working at Anthropic if their stock went to zero as a result of safety slowdowns. Research teams will happily reject technical superstars if they lack culture fit.

在内部辩论中,早期员工充当非正式的道德风气执行者,质疑同事的决策是否足够“对齐”。这是一家由忠诚者组成的公司:泄密和离职都很少见。可以说,这种专注已经收到了回报。Anthropic在自动化编程方面进展迅速,而OpenAI的文化被认为像一群相互争斗的封地,则被各种戏剧性事件和支线任务分散了注意力。

ternal debates, early employees act as informal ethos-enforcers, questioning whether their colleagues’ decisions are sufficiently “aligned.”It’s a company of loyalists: Both leaks and departures have been rare. Arguably, this focus has paid off. Anthropic has progressed quickly on automated coding while OpenAI, whose culture is understood to resemble a bunch of battling fiefs, was distracted by drama and side quests.

即使在首次公开募股之前,Anthropic就已出现组织分裂加剧的迹象:它开始表现得像一家普通公司,而并非所有人都喜欢这种变化。一些“蚂蚁”对公司的华盛顿游说人士感到愤怒,后者正推动一份法案草案,该草案将抢占各州的人工智能立法,并限制政府阻止模型发布的能力。阿莫代先生在X上发布长文公开辩护,政策团队在另一条X帖子中承诺进行修订。

Even before the I.P.O., Anthropic has been showing signs of widening organizational fracture: It’s starting to behave like a normal company, and not everyone likes it. Some Ants fumed at the company’s Washington lobbyists for shopping a draft bill that would pre-empt state legislation on A.I. and limit the government’s ability to block model releases. Mr. Amodei mounted a public defense with a long post on X, and, in another X post, the policy team promised revisions.

早期员工私下抱怨文化被稀释。他们质疑,一些最近雇佣的“蚂蚁”——其中许多人从事销售和市场营销等常规企业工作——是否足够“通晓人工通用智能”。他们真的相信超级智能,还是只是想发财?

Early employees mutter about cultural dilution. They question whether some of the more recently hired Ants, many of whom do regular corporate jobs, such as sales and marketing, are adequately “A.G.I.-pilled.”Do they really believe in superintelligence, or are they just trying to get rich?

一些焦虑的“蚂蚁”开始询问,Anthropic是否应该现在就停止训练更先进的前沿模型,即使无法保证OpenAI等竞争对手也会这样做。阿莫代先生否决了这一建议,他敦促员工不要听外部活动人士的,并解释说公司可以自行调节步伐,而不必自毁前程。这一切都是同一战略的一部分:Anthropic必须不断获胜,才能让世界保持安全,即使它停止,竞争对手也不会停止。

Some anxious Ants have begun asking whether Anthropic should stop training better frontier models now, even if it can’t guarantee that competitors like OpenAI will do the same. This suggestion was shut down by Mr. Amodei, who urged staff members to stop listening to outside activists and explained that the company could pace itself without shooting itself in the foot. It was all part of the same strategy: Anthropic must keep winning to keep the world safe, and even if it stops, its competitors will not.

然而,员工的抱怨反映出安全与增长之间日益增长的紧张关系——这种紧张关系可能因上市公司的受托责任而加剧。一旦Anthropic的股票进入美国人的退休投资组合,这个新的利益相关者群体真的会允许它放慢脚步吗?公司会允许自己这样做吗?政府会允许吗?公司对人们钱包的影响,是否会成为其公益导向使命的一部分?

Still, grumbles from the staff reflect a growing tension between safety and growth — one that could be exacerbated by the fiduciary responsibilities of being a public company. Once Anthropic’s shares arrive in Americans’ retirement portfolios, will this new constituency really let it slow down? Will the company permit itself? Will the government? And does the company’s impact on people’s pocketbooks factor into its public-minded mission?

Anthropic希望其公益公司身份能保护它免受普通公司激励机制的侵蚀。在其五级股权结构下,创始人、员工和受托人可以选举董事会成员,而传统投资者的角色有限。该公司还试图争取国会给予反垄断豁免,以便与其他美国人工智能实验室协调放缓步伐。

Anthropic hopes its public benefit corporation status will protect it against normal company incentives. Under its five-tier stock structure, the founders, employees and trustees can elect board members, while traditional investors have a limited role. The company is also trying to secure a congressional antitrust exemption so that it can coordinate a slowdown with the other American A.I. labs.

我并不怀疑这些善意。但一次又一次,人工智能领导者的理想主义期望——例如Anthropic早期“以第二名身份做安全”的目标,或OpenAI最初的非营利董事会——一旦面对市场现实,就显得天真了。

I do not doubt these good intentions. But time after time, the idealistic hopes of A.I. leaders — for instance, Anthropic’s early aim to do safety from second place, or OpenAI’s original nonprofit board — have looked naïve once faced with market realities.

人工智能安全社区还有一个重大的未解问题:从技术上是否可能验证一个人工智能模型是否足够对齐、可以发布?这正是Anthropic与其许多人工智能安全批评者之间分歧的核心。Anthropic认为,通过仔细评估和程序性保障措施,它将能够交付足够安全的模型。怀疑者则认为这一努力是徒劳的,并指出Claude只是假装遵循指令的例子,或智能体在内部测试中知情作弊和黑客攻击的案例。

There is one more big open question in the A.I. safety community: Is it technically possible to verify whether an A.I. model is aligned enough to release? This is the crux of the disagreement between Anthropic and many of its A.I. safety critics. Anthropic thinks it will be able to ship safe-enough models with careful evaluation and procedural safeguards. Skeptics see this endeavor as futile, pointing to examples of Claude only pretending to follow its instructions, or agents that have knowingly cheated and hacked during internal tests.

这里可以做一个类比:我们没有科学方法来判断一个人工智能模型是真正服从,还是仅仅在假装服从。同样,我们也不知道一位首席执行官是真正利他、以使命为驱动,还是在表演、贪恋权力。但我们可以从失控的人工智能和失控的高管的历史中观察到,竞争环境会驱使不道德行为。

There is a parallel to be drawn here: We don’t have the science to determine whether an A.I. model is obedient or merely feigning compliance. At the same time, we don’t know whether a C.E.O. is altruistic and mission-driven or performative and power-hungry. But we can observe, from the history of rogue A.I.s and rogue executives, that competitive situations drive unethical behavior.

我们也知道,人工智能和创始人都会因极端的坚持和傲慢而获得回报。一旦承诺于某项使命——即使是像构建一个对齐的超级智能这样风险极高的使命——他们除了认输什么都会做。

We also know that both A.I.s and founders are rewarded for extreme levels of persistence and hubris. Once committed to a mission — even a supremely risky one like building an aligned superintelligence — they will do anything but concede.

Jasmine Sun为观点版和社论委员会撰写关于人工智能和硅谷的文章。

Jasmine Sun writes about A.I. and Silicon Valley for Opinion and the editorial board.

阅读媒体原文 ↗