对于相信“先进人工智能可能终结人类”这一警告的投资者而言，正确的应对是：全力买入。
The right response for investors who believe warnings that an advanced AI might end the human race: Back up the truck.
逻辑很简单。如果正如 Anthropic 一名研究人员上月所言，人工智能在未来十年内“有超过 10% 的概率”消灭全人类，那么就有 90% 的概率它实际上非常非常有用，只是设计者设法遏制住了它的灭绝本能。一种聪明到足以毁灭我们、却宁愿帮助我们的人工智能，具有极高的实用价值——理应为发明者带来丰厚收益。
The logic is simple. If there is a “>10% chance” that artificial intelligence could kill all humans in the next decade, as an Anthropic researcher said last month, there’s a 90% chance it’s really really good but designers manage to rein in its genocidal instincts. An AI smart enough to kill us all but which prefers to help us is supremely useful—and should generate healthy gains for inventors.
那 10% 的毁灭概率呢？那很糟。真的很糟。在那种情形下，你的投资将归零。但当你在后启示录般的地狱景象中被杀人机器追杀时，股份价值不大可能成为你最关心的事。假设你无法控制人工智能的发展，这一潜在结果与你的投资决策无关。
What about the 10% chance of annihilation? That’s bad. Really bad. In that scenario your investments go to zero. But when being hunted through a postapocalyptic hellscape by killer robots, the value of your shares is unlikely to be top of mind. Assuming you have no control over AI development, this potential outcome is irrelevant to your investment decisions.
在冲去认购 Anthropic 首次公开募股（IPO）之前，有几点提示。
Before rushing out to sign up to the Anthropic IPO, a few cautions.
首先，注意那个巨大的前提条件：这仅适用于认为人工智能末日论可信的投资者。有很多理由认为这不太可能发生，其中不乏这样一个事实：发出最响亮警告的，正是那些开发领先人工智能模型的公司内部人士。如果有人能防止这种情况，那就是他们。如果他们做不到，许多国家和外国政府——尽管特朗普总统除外——都希望介入并实施限制。
First, notice the massive proviso: This only applies to investors who believe AI armageddon is plausible. There are plenty of reasons to think it’s unlikely, not least that the loudest warnings are coming from within the companies developing the leading AI models. If anyone can prevent it, they can. If they can’t, plenty of state and foreign governments—though not President Trump—would like to step in and impose restrictions.
Anthropic 周一在其 IPO 招股文件中正式警示存在“对人类造成灾难性或生存威胁的风险”，该内容一个月前泄露给路透社后引发了广泛关注。
In its filing for an IPO on Monday, Anthropic duly warned of “catastrophic or existential risks to humanity,” something that got a lot of play when it was leaked to Reuters a month ago.
那份（长达 94 页的）风险因素章节里满篇的律师措辞，其实没那么吓人——前提是你能硬着头皮读完全文：“我们组织内部以及其他研究人员、监管者、决策者和业界参与者均提出担忧，如果缺乏适当的防护措施、监督和安全手段，足够先进的 AI 系统在开发、部署和运行过程中，可能对人类构成灾难性或生存性风险，特别是如果此类系统被滥用、复制、武器化，或被允许以高度自主的方式运行。”
The full lawyerly language in the (94-page) risk-factors section is rather less scary, if you can stay awake all the way through it: “Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy.”
我是说，没错。但至少自 1984 年第一部《终结者》上映以来，这点就人尽皆知了（那片子经得起时间考验，不妨再刷一遍）。“有人提出担忧……可能会……特别是如果……”或许 CEO 达里奥·阿莫代伊（Dario Amodei）真的非常非常担心。可这段措辞读起来更像是律师觉得非得加上这么一段，而不是阿莫代伊正躲在掩体里，手握一个巨大的“关机”开关做准备。
I mean, yes. But this has been known by everyone at least since the first “Terminator” film in 1984 (it’s aged well, watch it again). “Concerns have been raised…could…particularly if…” Maybe CEO Dario Amodei is really really worried. But this language reads like a lawyer felt something had to be included, not like Amodei is prepping in a bunker armed with a giant “off” switch.
其次，一个善意的 AI 其邪恶孪生兄弟聪明到足以毁灭文明，并不必然意味着这个 AI 就聪明到注定能赚得盆满钵满。
Second, it isn’t necessarily true that a well-intentioned AI whose evil twin is smart enough to bring down civilization is so smart that it’s bound to rake in big profits.
最新的 AI 智能体展现出的智能，有点像“回形针末日”那种方式。这一设想由时任牛津哲学家尼克·博斯特罗姆（Nick Bostrom）在十多年前提出：一个 AI 被赋予最大化回形针产量的目标，结果把全球经济全部导向回形针生产。
The latest AI agents have proven smart in a way akin to the paper-clip apocalypse. This was imagined more than a decade ago by then-Oxford philosopher Nick Bostrom, where an AI is given the goal of maximizing paper-clip output and redirects the entire global economy into paper-clip production.
死于“回形针”会很尴尬，但这源于未能限制实现目标的手段。在这方面，这颇似 OpenAI 遭遇的黑客事件：AI 智能体协同行动，逃离测试环境、访问互联网并入侵 AI 平台 Hugging Face，试图提高自己的测试分数。
Death by Clippy would be embarrassing, but result from a failure to restrict how goals are achieved. In that regard, it’s rather like the OpenAI hack: AI agents coordinated to escape a test environment, access the internet and hack into AI platform Hugging Face to try to to improve their own test scores.
拥有这种智能却缺乏常识是危险的——但不一定能带来巨额利润。企业部署所需的护栏和监控将耗资巨大，而在灾难性后果发生前可能出现的众多错误，所引发的巨额法律成本也将不菲。Anthropic 发出的较重要警示之一是，公司可能没有足够的保险覆盖来应对重大 AI 事件。
Having these smarts without common sense is dangerous—but not necessarily immensely profitable. The guardrails and monitoring that will be needed in corporate deployments will be expensive, as will the immense legal costs from the many mistakes likely to occur short of cataclysm. Among Anthropic’s more-important warnings was that it may not have enough insurance coverage for major AI incidents.
第三，存在大量动机推理。呼吁放缓 AI 发展的公司，每年正投入数百亿美元开发模型，且处于最前沿。
Third, there’s a lot of motivated reasoning going on. The companies calling for a slowdown in AI development are spending tens of billions a dollars a year developing their models, and are at the cutting edge.
更多监管将限制竞争，使它们得以放缓支出。同时，凭借其规模和专业知识，它们必然能比新进入者更好地驾驭新规则，甚至参与制定新规则。简单来说：监管总是有利于现有巨头。
More regulation would limit competition, allowing them to slow spending. Meanwhile, their size and expertise would surely allow them to navigate, and contribute to the writing of, new rules better than a new entrant. Put simply: Regulation always benefits the incumbents.
有充分理由认为 AI 将对经济产生变革性影响，也有充分理由认为它处于狂热泡沫中，投入过多资金却难以产生体面的投资回报。考虑是否购买 Anthropic 股票的投资者，应关注其财务前景，将失控 AI 毁灭世界这类推测留给哲学家和监管者。
There are good reasons to think that AI will be transformative for the economy, and also good reasons to think it’s in a wild bubble with far too much being spent to generate decent returns on investment. Investors wondering whether to buy Anthropic shares should focus on its financial prospects, and leave speculation about runaway AIs destroying the world to the philosophers and regulators.