在充斥着警告超级智能机器将导致人类灭绝的新闻报道中，一则CNN的报道却描述了截然相反的情景——美军对脆弱聊天机器人的依赖差点让美国与中国陷入战争——这篇报道却几乎没有引起公众的注意。
Amid a barrage of news stories warning about superintelligent machines rendering humanity extinct, a CNN story describing the opposite scenario – one in which the US military`s reliance on brittle chatbots almost brought the US into war with China – went mostly unnoticed by the public.
过去三周最大的国际人工智能新闻是Anthropic工程师雅各布·考克森（Jacob Coxon）的辞职。据他透露，OpenAI和Anthropic正“直奔自我改进的超级智能，并在拿我们的生命进行赌博”。考克森对“终结者”情景的描述——即机器变得比人类聪明得多并决定消灭我们——吸引了公众、记者和政治家的注意，引发了一波波制造恐惧、轻信的报道以及政策行动。
The biggest international AI news of the past three weeks was Anthropic engineer Jacob Coxons resignation. According to him, OpenAI and Anthropic are “racing straight towards self-improving superintelligence and gambling with our lives”. Coxons description of a “terminator” scenario, a machine becoming much smarter than humanity and deciding to wipe us out, captured the public, journalistsand politicians attention, stirring waves of fear-inducing, credulous reporting, and policy actions.
加利福尼亚州州长加文·纽森（Gavin Newsom）下令建立一个“人工智能紧急关停开关”；纽约州州长凯西·霍楚尔（Kathy Hochul）也在类似地探讨拥有“人工智能紧急关停开关”的可能性；参议员伯尼·桑德斯（Bernie Sanders）和国会议员格雷格·卡萨尔（Greg Casar）则提出了旨在“禁止人工超级智能”的立法。
California governor Gavin Newsom issued a mandate to create an “AI kill-switch”; New York governor Kathy Hochul is similarly exploring the possibility of having “AI kill-switches”; and Senator Bernie Sanders and Congressman Greg Casar introduced legislation to “ban artificial superintelligence”.
然而，当我们中的许多人把注意力集中在那些被告知可能假想导致人类灭绝的虚构超级智能机器上时，另一则更为严肃的报道正在被人们所忽视。
While so many of us were focused on fictional superintelligent machines that we were told could hypothetically cause human extinction, however, there was a more serious story being reported.
一系列事件差点把我们带向世界末日，这并非由“失控的超级智能AI”引起，而是由于政府依赖于那些被制造商标榜为接近超级智能、但却极易出错的系统。
A series of events almost got us to armageddon, not caused by “rogue, superintelligent AI”, but by government reliance on error-prone systems that are marketed by their manufacturers as being close to superintelligent.
CNN于9月18日发表了一篇尚未得到其他主流媒体核实的报道，描述了美国是如何“差点与中国开战”的。当时的一份情报报告称一艘中国船只正在运输核武器部件，军事人员因此制定了拦截该中国船只的计划，出动了战机和士兵准备登船。这种中美之间的军事冲突可能会迅速升级，并演变成第三次世界大战。
A CNN report published on 18 September, which hasn`t been verified by other major outlets, describes how the US “almost started a war” with China, based on an intelligence report that claimed a Chinese ship was transporting components of nuclear weapons. Military personnel made plans to intercept the Chinese vessel, complete with aircraft and soldiers ready to board the ship. A military skirmish of this kind between China and the US could quickly escalate and spiral into world war three.
就在执行该计划前的几分钟，美国官员对情报报告进行了进一步调查，发现该报告是由聊天机器人生成的，且其中关于船只所载货物的信息存在错误。
It was only moments before executing on the plan that American officials further investigated the intelligence report and found that it had been generated with the use of a chatbot and contained erroneous information about the vessel`s contents.
与《终结者》中“失控人工智能”的情节不同，这一事件并未促使国会议员们纷纷呼吁加强监管，以避免潜在的人类灭绝风险。一家又一家媒体也没有向公众发出警告，指出我们几乎因军方依赖易出错的聊天机器人而引发了一场针对核大国的战争。
Unlike the terminator, “rogue AI”, scenario, this incident didnt galvanize lawmakers into announcing calls for regulation so as to avoid potential human extinction. Media outlet after media outlet didnt warn the public that we almost triggered a war against a nuclear superpower because of our military`s reliance on error-prone chatbots.
我们将驱动这些聊天机器人的大语言模型（LLMs）描述为“随机鹦鹉”，即那些被设计用来复述其训练数据中模式的模型。新闻报道将这些模型描述为“强大”且“失控”的系统，声称它们可能单独导致人类灭绝，这种说法强化了公众认为基于这些模型的工具足够可靠、足以用于战争等高风险领域的认知。
We describe the large language models (LLMs) powering these chatbots as stochastic parrots, that is, models that are designed to regurgitate the patterns of the data they are trained on. News coverage describing them as “powerful” “rogue” models that could render humanity extinct all on their own, bolsters perceptions of tools based on these models being infallible enough to merit usage in stakes as high as warfare.
即便是美国有线电视新闻网（CNN）较为批判性的报道，也将美军使用的聊天机器人描述为存在“深远风险”，因为它是一种“强大、新颖且相对缺乏深入理解的技术”。
Even CNN`s more critical coverage describes the chatbot used by the US military as presenting “profound risks” because it is “powerful, new and relatively poorly understood technology”.
事实上，当今的大语言模型及其驱动的聊天机器人并非缺乏深入理解，也并非在作为该用例有效工具这一意义上“强大”。它们是用于生成看似合理文本的系统，由庞大且杂乱收集的数据集构建而成，随后经过“微调”（即进一步训练），以使其对目标用户更具吸引力。
In fact, today`s LLMs and the chatbots they power are not poorly understood, nor are they powerful in the sense of being effective tools for this use case. They are systems for generating plausible looking text, built out of enormous, haphazardly collected datasets and then “fine-tuned” (that is, further trained) to be especially appealing to their intended users.
CNN未能查明此案中具体使用了哪款聊天机器人产品，但驱动该产品的底层大语言模型很可能经过微调，以输出具有情报报告风格特征的文本。多年来，此类模型的基本功能及其风险已得到充分理解和详细记录，包括我们2021年发表的论文《随机鹦鹉的危险：语言模型可以太大吗？》（On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?）。
CNN wasn`t able to learn which chatbot product was used in this case, but the LLM powering the product was likely fine-tuned to output text with the stylistic hallmarks of intelligence reports. Both the basic functionality and the risks of such a model have been well-understood and well-documented for years, including in our 2021 paper On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
关于系统行为神秘莫测的说法，源于销售这些系统的人刻意制造的迷雾，以及采购和使用这些系统的人对其输出结果的误解。
Claims that system behavior is mysterious arise from deliberate mystification on the part of those selling those systems, and a misapprehension of what their output is by those procuring and using it.
与OpenAI和Anthropic首席执行官那些不科学、充满幻想的言论，以及那些不加批判地复述其说辞的人不同，这些公司产品造成的有害且已有充分记录的影响，是因为人们认为这些产品具有超智能，而非它们真的具备超智能。
Unlike the unscientific and fantastical claims of OpenAI and Anthropic CEOs and those who uncritically repeat their talking points, the harmful, and well-documented, impacts of these companies` products have arisen because people that the products are superintelligent, not because they actually are.
从医院使用的“AI”文书系统错误地将患者归类为非法药物使用者，到政府依赖将学校误判为军事设施的“情报分析”系统后杀害儿童，我们正目睹因盲目信任不可靠的自动化软件而带来的可预见的危害。
From “AI” scribes used by hospitals erroneously classifying patients as illicit drug users, to governments killing children after relying on “intelligence analysis” systems which mistake schools for military facilities, we are seeing the predictable harms that come from misplaced faith in unreliable automated software.
与此同时，销售此类软件的公司开展的营销活动警告着即将到来的超智能，将我们的注意力从真正的风险转移到了它们对末日的幻想上。
Meanwhile, the marketing campaigns from the companies selling this software warn of impending superintelligence, directing our attention away from the real risk to their fantasies of doom.
是时候对以“AI”名义销售的系统的使用进行监管了，不是因为它们是神奇的机器，而是因为它们是容易出错的产品，不应被用于高风险场景。任何用于军事、医疗或其他生死攸关情境的自动化系统，都必须在其具体使用场景中进行彻底评估，并配套使用规范，确保问责权掌握在真正拥有决策权的人手中。
It is high time for us to regulate usage of systems sold as “AI”, not because they are magical machines, but because they are error-prone products that shouldn`t be used in high stakes scenarios. Any automation used in military, medical or other life-and-death situations must be thoroughly evaluated within its use case and paired with usage norms that keep accountability with people who have actual power to make decisions.
正如前联邦贸易委员会主席莉娜·汗多次提醒我们的那样，在人工智能领域，法律没有豁免权：现有的法律可供联邦机构和其他监管机构执行，以保护公众免受人工智能行业不安全、不科学做法的侵害——这些做法在将不成熟的产品推向市场的同时，却将其营销为“天空中的魔法智能”。
As former FTC chair Lina Khan has repeatedly reminded us, there is no exemption to the law when it comes to AI: there are existing laws that federal agencies and other regulators can enforce to protect the public against the AI industry`s unsafe and unscientific practices of pushing unsound products while marketing them as “magic intelligence in the sky”.
顺从公司高管并将他们的产品描述为具有自主意识的生命体，会让这些高管逃避责任，同时欺骗公众。我们呼吁记者和立法者追究公司的责任，而不是鹦鹉学舌般重复他们的营销说辞。
Going along with company executives and describing their products as beings with agency allows these executives to evade accountability while deceiving the public. We call on journalists and lawmakers to hold companies accountable rather than parrot their marketing talking points.
蒂姆尼特·格布鲁（Timnit Gebru）是分布式AI研究所（DAIR）的执行主任，也是即将出版的新书《深度遗忘：一个科技理想主义者的激进化》（Deep Unlearning: The Radicalization of a Tech Idealist）的作者，该书目前已开启预售，定于2月16日出版。
Timnit Gebru is executive director of Dair and author of the forthcoming book Deep Unlearning: The Radicalization of a Tech Idealist, which is available for preorders now and set to publish on 16 February.
艾米丽·M·本德（Emily M Bender）是华盛顿大学的语言学教授，也是《人工智能骗局》（The AI Con）的合著者。
Emily M Bender is professor of linguistics at the University of Washington and co-author of The AI Con.