字号·· | 护眼
大西洋月刊

人工智能恐怖主义就是恐怖主义AI Terrorism Is Just Terrorism

点「原文对照」整页切到原文,或双击某段只看那段的原文。

一幅插画:一个人的头被一顶毛线帽罩住,帽子上为嘴和眼睛留了洞,但嘴和眼睛的位置是一块故障闪烁的电脑屏幕。An illustration of a head covered by a beanie with holes for the mouth and eyes, but the mouth and eyes are a glitching computer screen.

新技术尚未从根本上改变这一问题——至少目前还没有。文/伊恩·博戈斯特(Ian Bogost) 反恐战争与互联网的兴起同步,当时新出现的令人担忧的数字攻击被称为“网络恐怖主义”。如今,一种新威胁出现了:AI恐怖主义。

The new technology hasn’t fundamentally transformed the problem—at least not yet. By Ian Bogost The War on Terror coincided with the rise of the internet, when freshly concerning digital attacks became known as cyberterrorism. Now a new threat has appeared: AI terrorism.

过去几年,政府、政府间组织和智库的报告警告称,AI极有可能助长恐怖活动,要求做好准备和应对。2021年,联合国反恐怖主义办公室联合发布了一份关于“为恐怖目的恶意使用人工智能”的报告。报告重点涉及恶意软件和入侵等网络攻击,以及通过自动驾驶汽车、人脸识别欺骗、深度伪造实施的物理攻击,还有——新冠疫情当时仍在肆虐——人工合成生物武器。

Over the past few years, reports from governments, intergovernmental organizations, and think tanks have warned that AI is likely to fuel terrorist activity, demanding preparation and response. In 2021, the United Nations Office of Counter-Terrorism co-published a report on the “malicious use of artificial intelligence for terrorist purposes.”Cyberattacks such as malware and intrusion featured heavily, as did physical attacks via autonomous vehicles, facial-recognition spoofing, deepfakes, and—the coronavirus pandemic was still raging—engineered biological weapons.

更早些时候,2019年和2020年,蒙特雷国际研究学院(Middlebury Institute of International Studies)发布了两份报告,探讨语言模型(当时仍鲜为人知)对宣传和激进化构成的风险。就在这周,哈佛肯尼迪学院的一组研究人员发布了一份工作论文,假设“首起重大AI赋能恐怖主义事件”是一种可信威胁,并推测其可能发生的方式。

Earlier, in 2019 and 2020, the Middlebury Institute of International Studies published two reports on the risks that language models, still mostly unheard-of at the time, would pose to propaganda and radicalization. And just this week, a group of Harvard Kennedy School researchers published a working paper that assumes that the “first major AI-enabled terrorism event” is a credible threat and speculates how it might come about.

“网络恐怖主义”本身就是一个令人困惑的概念,部分原因在于,黑客攻击或瘫痪计算机基础设施可能由国家行为者实施,且不涉及暴力——这些特征通常与恐怖主义不相关。恐怖主义之所以糟糕,原因很多,其中一个非常次要的原因是,这个术语没有固定定义。在法国大革命时期,它指的是雅各宾国家实施的暴力,在许多情况下是通过断头台执行。

Cyberterrorism was already a confusing term, partly because hacking or disabling computer infrastructure might be carried out by state actors and without violence—properties that aren’t normally associated with terrorism. Among the reasons terrorism is bad, a very minor one is that the term has no fixed definition. In revolutionary France, it referred to violence exercised by the Jacobin state, in many cases by guillotine.

到19世纪末,它转变为无政府主义潜在革命者的非国家暴力。20世纪,恐怖主义开始与独立运动重叠,使其更接近分离主义冲突或游击战。到20世纪末,这种意识形态动机成为恐怖主义的必要条件。恐怖主义等同于旨在产生心理效应的暴力。

By the late 19th century, it had shifted to the nonstate violence of anarchist would-be revolutionaries. Terrorism began to overlap with independence movements in the 20th century, drawing it closer to separatist conflict or guerrilla warfare. By the late 20th century, this ideological motivation for terrorism became mandatory. Terrorism amounted to violence designed to produce a psychological effect.

20世纪70年代的劫机和人质危机体现了这一修订后的目的。国内袭击也是如此,例如蒂莫西·麦克维的袭击,他于1995年炸毁了俄克拉荷马城的一座联邦大楼。今天,在9·11袭击发生四分之一世纪之后,恐怖主义往往带有政治判断,而非描述一种行为或某种行为者:几乎任何对手现在都可能被称为“恐怖主义者”。

The hijackings and hostage crises of the 1970s exemplify this revised purpose. So do domestic attacks, such as that of Timothy McVeigh, who bombed an Oklahoma City federal building in 1995. Today, a quarter century after the September 11 attacks, terrorism often carries a political judgment instead of describing an act or a type of actor: Nearly any adversary whatsoever might be called a “terrorist” now.

鉴于这个术语的含义漂移,从某种意义上说,AI本质上具有恐怖性:它威胁要夺走你的工作、偷走你的钱、毁掉你孩子的教育、消灭艺术和文化、夺走你的医疗保健、殖民你的城镇、破坏气候、让电子产品永远无法负担。你还被告知,AI是不可避免的——而且只有极其富有的硅谷公司,可能由反社会人格领导人经营,才拥有推动其发展的资源和人才(他们必须这样做,即使这意味着人类的毁灭)。

Given the slippage of the term, AI is, in a way, terroristic by nature: It threatens to take your job, steal your money, ruin your children’s education, eradicate art and culture, take away your health care, colonize your town, destroy the climate, make electronics forever unaffordable. You have also been told that AI is inevitable—and that only immensely wealthy Silicon Valley companies, possibly run by sociopathic leaders, have the resources and talent to advance it (which they must do, even if it means the destruction of humankind).

但对恐怖主义的旧有概念仍然存在。该技术还可能在引发人工智能所威胁的熟悉混乱之外,促成国家或非国家实施的暴力或破坏行为,例如使电网瘫痪或在工作场所释放生物武器。

But the older sense of terrorism still persists. The technology could also facilitate, in addition to the familiar chaos that AI threatens to unleash, terrorism in the sense of state- or nonstate-enacted violence or disruption, such as disabling your electric grid and unleashing a bioweapon at your workplace.

研究人工智能恐怖主义的人提出的担忧不同于人工智能行业内部的吹哨人或人工智能领袖所提出的担忧。超智能人工智能导致的灭绝事件并未位列风险榜首,理论上可能自主的流氓人工智能代理也未列入首位。相反,研究人员、情报官员和情景规划者似乎主要关注人工智能如何使不法分子以更少的努力实施更为复杂的活动。例如,拉斯维加斯的Cybertruck炸弹制造者曾向ChatGPT询问如何引爆炸药,这种用法表明恐怖分子可能既愚蠢又有进取心。

The people who study AI terrorism raise different concerns than those raised by whistleblowers inside the AI industry—or AI leaders themselves. Superintelligent-AI-driven extinction events do not top the list of risks, nor do the rogue AI agents that might, in theory, become autonomous. Instead, researchers, intelligence officials, and scenario planners seem principally interested in the ways that AI might allow bad actors to carry out more sophisticated activities with reduced effort. The Las Vegas Cybertruck bomber, for example, asked ChatGPT for advice about how to detonate an explosion, a use that suggests terrorists might be stupid in addition to being enterprising.

当今的前沿人工智能模型功能强大、成本低廉且广泛可得。它们能够(或帮助实现,或看似实现)许多未被明确设计的任务。犯罪分子、恐怖分子乃至政府往往利用现有的通信、交通和工业基础设施从事不法行为。人工智能提供了另一种此类工具。这个结论既合理,又有些平淡。

Today’s frontier AI models are powerful, cheap, and widely available. They can accomplish (or help accomplish, or appear to accomplish) many types of tasks without having been explicitly designed to do so. Criminals, terrorists, and even governments have tended to use existing communication, transportation, and industrial infrastructure to do bad things. AI offers another such tool. This conclusion is both reasonable and a bit humdrum.

因此,我读到的大多数关于人工智能恐怖主义的报道在担忧上都比较保守。它们强调,实施暴力行为或其他物理攻击很困难,人工智能无法神奇地解决这些问题。它们或许是在回应多年来对网络恐怖主义威胁的夸大,最著名的例子是“电子珍珠港”的概念——一种会使国家瘫痪的、对数字基础设施造成毁灭性打击的攻击。

Consequently, most of the reports on AI terrorism that I have read are conservative in their concerns. They stress that carrying out acts of violence or other physical attacks is difficult and that AI can’t magically solve those problems. They are, perhaps, responding to years of cyberterrorism-threat overstatement, the most famous example of which is the idea of an “electronic Pearl Harbor”—a devastating attack on digital infrastructure that incapacitates the nation.

这个隐喻至少自1999年起就开始流传,2012年,时任国防部长莱昂·帕内塔向国会警告说,可能发生一次“网络珍珠港”,其破坏力“可能与9·11恐怖袭击一样具有破坏性”。许多国际政策专家认为,网络珍珠港是一种混合隐喻。珍珠港攻击是一项精心策划的国家战争行为,而非流氓行动。更可能的是,较小的网络安全事件可能会级联成更大的事件,或导致不可预测的次级效应。

That metaphor, which has been in circulation since at least 1999, was made more concrete in 2012, when then–Defense Secretary Leon Panetta warned Congress about a “cyber Pearl Harbor” that “could be as destructive as the terrorist attack on 9/11.”Many international-policy experts have argued that cyber Pearl Harbor amounts to a mixed metaphor. The attack on Pearl Harbor was an intricately planned act of state warfare, not a rogue operation. More likely, smaller cybersecurity incidents could cascade into bigger ones, or cause unpredictable secondary effects.

多年来,针对物理基础设施的真实且具有破坏性的网络攻击一直是可能的。人工智能在网络恐怖主义中的作用,很可能与互联网相同:放大了生产和传播宣传、混乱以及政治操纵的能力。

Real, damaging cyberattacks on physical infrastructure have been possible for many years. AI’s role in cyberterrorism is likely to be the same as the internet’s: an amplified capacity to produce and disseminate propaganda, confusion, and political manipulation.

帕内塔以风险即使不合理也可能存在为理由,为关于网络珍珠港的警告辩护。哈佛肯尼迪学院研究人员在其论文中似乎也出于类似理由。合著者将其方法描述为“事前复盘”,即假设此类事件已经发生,然后倒推以了解其发生过程。

Panetta justified the warning about a cyber Pearl Harbor on the grounds that the risk was possible even if it wasn’t plausible. A similar justification appears to motivate the Harvard Kennedy School researchers in their paper. The co-authors describe their methodology as a “pre-mortem,” which involves assuming that such an event has taken place and working backwards to understand how it came about.

我问了该论文的合著者之一斯蒂芬·卡斯珀,人工智能究竟是创造了新的灾难性能力,还是仅仅让现有的恶意行为变得更容易、更快捷或更便宜。“更多是后者,”他告诉我。具体来说,拥有危险能力信息的AI模型,例如如何破坏基础设施和制造化学武器,可能会让恶意行为者将其用于邪恶目的。另一位合著者马克斯·卡马奇告诉我,新模型“在没有防护措施的情况下,可以显著协助新手制造已知的严重生物威胁。”

I asked Stephen Casper, one of the paper’s co-authors, whether artificial intelligence creates new catastrophic capabilities or simply makes existing malice easier, faster, or cheaper. “It’s more the latter,” he told me. Specifically, AI models that possess information about dangerous capabilities, such as how to disrupt infrastructure and construct a chemical weapon, could allow malicious actors to use them to nefarious ends. Max Kamachee, another co-author, told me that new models “can meaningfully assist a novice in creating a known severe biological threat absent safeguards.”

简单来说,怀有恶意的坏人可能会利用AI实施恶行。该论文深入探讨了更多细节,包括就如何控制或监管不同类型的模型以防止伤害提出了合理的建议。

Put more simply, bad people with ill intent could use AI to carry out bad acts. The paper goes into much greater detail, including making reasonable suggestions about how different types of models might be controlled or regulated to prevent harm.

哈佛论文提出了两个首次重大AI恐怖主义事件的候选方案:一个是造成数十亿美元经济损失的网络攻击;另一个是造成超过100人死亡的生物武器攻击。我问卡斯珀和卡马奇,恐怖主义是否是这两种情景的正确名称。该论文将这些试金石标准源自近期美国关于AI行业模型开发和管理的法律定义。但卡斯珀承认,这些不良结果可能并非出于意识形态动机。

The Harvard paper presents two candidates for the first major AI-terrorism event: One is a cyberattack that would cause several billion dollars’ worth of economic damage; the other, a biological-weapon attack that would kill more than 100 people. I asked Casper and Kamachee whether terrorism was the right name for the two scenarios. The paper derives these litmus tests from definitions in recent U.S. laws governing the development and management of models in the AI industry. But Casper acknowledged that those bad outcomes might not be ideologically motivated.

“我们称之为恐怖主义,”他告诉我,但他也表示,一个等效的名称将是“为恶意目的而进行的人类主导的AI滥用。”卡马奇将这种风险称为“灾难性恶意使用。”预测未来是愚蠢的,而猜测真正可怕的事件是否会发生则更糟糕。一方面,卡斯珀及其合著者警告AI可能促成的最坏情景(即使是间接促成)似乎是有道理的。另一方面,AI恐怖主义风险可能会沦为类似“网络珍珠港”的恐怖主义修辞。

“We call it terrorism,” he told me, but he also said that an equivalent name would be “the human-led misuse of AI for malicious ends.”Kamachee called the risk “catastrophic malicious use.”Predicting the future is foolish, and guessing about whether truly terrible events might or might not come to pass is even worse. On the one hand, Casper and his co-authors seem justified in warning about the worst scenarios that AI could facilitate, even indirectly. On the other hand, AI terrorism risks devolving into the fearmongering rhetoric that cyber Pearl Harbor did.

我追问卡斯珀,要他把人工智能所能实现的事与可能出现的结果联系起来。他指出,模型能力的演进速度惊人,而且一些人工智能公司正直接建设人工智能驱动的生物研究设施。例如,Anthropic已为克劳德启用的DNA和蛋白质研究建造了湿实验室。这些实验室可能被攻破,或者其中的模型权重可能外泄,落入不法分子之手。

I pressed Casper to connect the dots between what AI makes possible and which outcomes are likely. He cited the shockingly rapid evolution of capability in the models, along with the fact that some AI companies are directly building AI-driven-biological-research facilities. Anthropic, for example, has built wet labs for Claude-enabled DNA and protein research. Those labs could be breached, or the model weights discovered inside them could make their way out and into the wrong hands.

“这是一个大世界——人多,群体多,国家多,动机也多,”卡斯珀说。“当前人工智能系统被用于恐怖主义类活动的先例,你知道,是一个相当不错的迹象。”什么迹象?人工智能行业将迎来卡斯珀所说的“挑战者号或切尔诺贝利或九一一或安然时刻”。然而,这些历史性灾难彼此截然不同。挑战者号因结构失效而毁灭,该失效因组织管理不善而被忽视;切尔诺贝利灾难由设计缺陷造成,并因操作失误和官僚体制的压制而被放大;九一一事件是一起意识形态驱动的非国家行为体袭击,得益于专业知识的易得性,并因国家情报失误而得以实现;安然丑闻是会计欺诈,因监管失效和审计师的共谋而得以发生。

“It’s a big world—lots of people, lots of groups, lots of states, lots of motivations,” Casper said. “The precedented misuse of current AI systems for terrorist-type activities is, you know, a pretty decent sign.”A sign of what? That the AI industry is going to experience what Caspar called “a Challenger or Chernobyl or 9/11 or Enron moment.”And yet, those historical catastrophes are all quite different. Challenger was doomed by a structural failure that had been overlooked because of organizational mismanagement; the Chernobyl disaster was caused by a design flaw magnified by operator error and stifling bureaucracy; September 11 was an ideologically motivated nonstate attack facilitated by ease of access to expertise and realized thanks to national-intelligence blunders; the Enron scandal was accounting fraud allowed by regulatory failure and auditor complicity.

其中两个例子是工业事故,一个是故意不当行为导致的非预期结果,只有一个算得上是恐怖袭击。

Two of the examples were industrial accidents, one was an unintentional outcome of deliberate misconduct, and only one counts as a terrorist attack.

人工智能所带来的实际风险,无需恐怖主义来引发担忧,而思考恐怖主义可能会让我们困惑,或使我们远离采取正确预防行动。由于恐怖分子活跃于社会边缘,他们往往会采用当代生活所提供的任何技术和基础设施。他们选择工具、武器和目标,不仅取决于什么能造成伤害或吸引注意,还取决于什么他们能够弄到手。

The actual risks created by AI don’t require terrorism to inspire worry, and thinking about terrorism may confuse or distance us from taking the right preventive actions. Because they act at the fringes of society, terrorists have a tendency to adopt whatever technology and infrastructure that contemporary life makes available. They choose their tools, their weapons, and their targets based on not merely what will cause harm or attract attention but also what they can get hold of.

20世纪初的炸药和中世纪的汽车炸弹就是如此;能够给攻击者带来臭名昭著的报纸、广播和电视媒体也是如此;成熟但安全性不足的商业航空也是如此;银行漏洞和非账面金融网络,其中包括加密货币,也是如此。没有人将麦克维称为“租赁卡车恐怖分子”或“农业肥料恐怖分子”,但这两种资源对他所期望的目的——确实是恐怖主义——来说,都证明是方便且有效的。

This was true of explosives such as dynamite at the turn of the 20th century and car bombs by mid-century; the newspapers, radio, and television media that would produce notoriety for attackers; commercial aviation once it became mature but not sufficiently secure; and banking exploits and off-books financial networks, among them cryptocurrency. Nobody called McVeigh a rental-truck terrorist or an agricultural-fertilizer terrorist, but those two resources proved convenient and effective for his desired purpose—which was indeed terrorism.

在其历史上,恐怖主义反复挪用民用技术,尤其是未经验证或管理不善的技术。方便的是,此类基础设施也往往会让公众感到危险,从而加剧我们对脆弱性的恐惧。像利用铁路、汽车、飞机、密集城市化和互联网的恐怖主义一样,利用人工智能的恐怖主义是可能的,甚至很可能。为其可能发生的情况做准备是明智的,尽管是推测性的。从这个意义上说,人工智能恐怖主义并不新鲜,即使它仍然令人恐惧。

Throughout its history, terrorism has repeatedly appropriated civilian technologies, especially unproven or insufficiently managed ones. Conveniently, such infrastructure also tends to make the public feel danger, which accelerates our fear of vulnerability. Like terrorism that uses railways, cars, airplanes, dense urbanism, and the internet, terrorism that makes use of artificial intelligence is possible and even likely. Preparing for its contingencies is wise, if speculative. In that sense, AI terrorism is not new, even if it is still frightening.

阅读媒体原文 ↗