字号·· | 护眼
The Intercept

美军实施人工智能后,平民死亡人数上升When the U.S. Military Implemented AI, Civilian Deaths Rose

点「原文对照」整页切到原文,或双击某段只看那段的原文。

2019年10月27日,在叙利亚巴里沙村附近、靠近土耳其边境的地方,孩子们查看一辆烧毁的车辆。此前,伊斯兰国领导人阿布·巴克尔·巴格达迪在美军特种部队的突袭中被击毙。Children inspect a burnt vehicle on Oct. 27, 2019, near the Syrian village of Barisha, close to the Turkey border, after Islamic State leader Abu Bakr al-Baghdadi was killed in a U.S. special forces raid.

本文与Airwars合作发表。

This story is published in collaboration with Airwars .

2019年一个温暖的夜晚,35岁的巴拉卡特在朋友开车护送下回家,车子穿过叙利亚古老的山村巴里沙。巴里沙是此前已被废弃的拜占庭“死城”之一。“当时我和最好的朋友们在一起。他们想把我送到家,”这位五个孩子的父亲后来告诉美国国家公共电台。巴拉卡特和两名同伴在一家橄榄榨油坊做农工,刚结束一天的活计。“我们吃了南瓜子,路上买了咖啡,正开心着呢。”

On a balmy night in 2019, 35-year-old Barakat was on his way home, driven by friends through the ancient, mountainous Syrian village of Barisha, one among the previously abandoned Byzantine “Dead Cities.”“I was with my best friends. They wanted to drop me off at home,” the father of five later told NPR. Barakat and his two companions had finished their day’s work as agricultural laborers at an olive press. “We had pumpkin seeds and bought coffee on the road and were having fun.”Unknown to them, a fateful military operation was occurring nearby.

他们不知道的是,附近正在进行一场决定命运的军事行动。300米外,一支美国特种部队正在突袭伊斯兰国头目阿布·贝克尔·巴格达迪的院落。

Three hundred meters away, a team of U.S. special forces was raiding the compound of Islamic State leader Abu Bakr al-Baghdadi . As the attack unfolded, most U.S. surveillance and reconnaissance assets were trained on the compound itself.

袭击展开之际,美军的大部分侦察监视力量都对准了这处院落本身。然而当时鲜有人知的是,一套新的人工智能之眼已被引入美军在伊拉克和叙利亚的作战行动。这套计算机视觉目标探测系统是五角大楼代号“梅文计划”的算法战项目的一部分,使军方能够借助人工智能对图像和视频进行融合分析,识别敌方人员和车辆。

Yet, known to few at the time, a new set of artificial eyes had been introduced to U.S. warfare in Iraq and Syria. Computer vision target detection, part of the Pentagon’s algorithmic warfare program known as Project Maven , enabled the military to recognize enemy personnel and vehicles through AI-assisted fusion and analysis of images and video. As Barakat’s vehicle headed south through Barisha, Maven’s AI detected a vehicle that the U.S. military’s human eyes had missed.

当巴拉卡特的车向南驶过巴里沙时,梅文计划的人工智能探测到一辆美军人工目视未能发现的车辆。地面指挥官随后呼叫发动了攻击。

The ground commander then proceeded to call in the attack. “At that moment, the helicopters arrived. Suddenly, we were hit. I didn’t know what was going on. I was just trying to escape death,” Barakat told NPR.

“就在那时,直升机来了。突然间,我们遭到袭击。我不知道发生了什么。我只想逃命,”巴拉卡特告诉美国国家公共电台(NPR)。他的右臂被炸断,左手部分功能丧失。他的朋友哈立德·穆斯塔法·库尔莫和哈立德·阿卜杜勒·马吉德·库尔莫遇难。

His right arm was blown off, with his left hand partially disabled. His friends, Khaled Mustafa Qurmo and Khaled Abdel Majid Qurmo, were killed. The U.S. military ultimately rejected Barakat’s claim of harm.

美国军方最终拒绝了巴拉卡特关于受到伤害的说法。在NPR调查后启动的五角大楼第二次审查,其结果从未公布。

The results of a second Pentagon review, opened after an NPR investigation, were never released. A spate of articles have hailed this as the year of AI warfare — from Ukrainian drones to Israel’s algorithmic decision support systems.

大量文章称赞这是人工智能战争之年——从乌克兰无人机到以色列的算法决策支持系统。我最近参与的一个项目追踪了人工智能快速且可能危险的引入如何改变“杀伤链”的每一个环节。

A recent project that I worked on tracked how the rapid and potentially dangerous introduction of AI is transforming every part of the “kill chain.” The AI-powered attack on Barakat may have occurred seven years ago, but new research has shown that many of these systems had been fielded years earlier than previously known, prompting questions about the extent of artificial intelligence’s role in historical attacks — and how it has already impacted civilian harm.

2019年10月27日,在美国军方针对伊斯兰国组织领导人阿布·巴克尔·巴格达迪的行动后,孩子们经过叙利亚巴里沙村附近被毁的房屋。图片:Ghaith Alsayed/APChildren pass destroyed houses near the Syrian village of Barisha on Oct. 27, 2019, after an operation by the U.S. military targeting Abu Bakr al-Baghdadi, the leader of the Islamic State group. Photo: Ghaith Alsayed/AP

对巴拉卡特的人工智能驱动袭击可能发生在七年前,但新的研究表明,其中许多系统部署的时间比此前已知的要早数年,这引发了人们对人工智能在历史袭击中所起作用程度的疑问——以及它已经如何影响了平民伤害。

Children pass destroyed houses near the Syrian village of Barisha on Oct. 27, 2019, after an operation by the U.S. military targeting Abu Bakr al-Baghdadi, the leader of the Islamic State group.

2019年10月27日,在美军针对伊斯兰国组织头目阿布·贝克尔·巴格达迪的行动之后,儿童们经过叙利亚巴里沙村附近被毁的房屋。

Photo: Ghaith Alsayed/AP

梅文的黎明梅文在2019年巴格达迪袭击中的作用直到今年才在记者卡特里娜·曼森的《梅文计划》一书中披露,该书通过同名项目梳理了美国算法战争的发展。

Maven’s Dawn

“梅文计划”试图打造算法和平台,把来自无人机、雷达、卫星和文本等多个来源的实时数据融合起来,用以探测威胁、进行生活模式分析并追踪目标,从而加快军事目标定位。

The role of Maven in the 2019 Baghdadi attack was only revealed this year in journalist Katrina Manson’s book “Project Maven,” which charts the development of U.S. algorithmic warfare through the program of the same name.

美军最初选择谷歌作为“梅文计划”的技术合作伙伴。但在外界抗议谷歌参与人工智能目标定位后,谷歌于2018年退出,帕兰提尔随即介入。

Project Maven sought to create the algorithms and platforms to fuse real-time data from multiple feeds — among them drones, radar, satellites, and text — to conduct detect threats, run pattern-of-life analysis, and track targets, in the service of accelerating military targeting. The U.S. military initially tapped Google as a technology partner for Project Maven.

“梅文”最具革命性的方面有两个:一是创建“单一可视化工具”——在此即帕兰提尔的“梅文智能系统”——来汇集各路数据流、将其融合并大幅提升决策速度,这类系统更常见的叫法是“决策支持系统”。

But Google pulled out in 2018 , following protests about its involvement in AI targeting, and Palantir stepped in .

第二个具有变革意义的方面,是用无人机监视及其他空中和太空传感器产生的大量图像和全动态视频来训练人工智能算法。这是一项涉及多家国防承包商的合作,最初由五角大楼牵头。随后,“梅文”算法会被部署到冲突战区,应用于数据流以探测物体或目标,所用技术被称为“计算机视觉”。

Maven’s most revolutionary aspects were twofold: the creation of a “single visualization tool” — in this case, Palantir’s Maven Smart System — to house the feeds, fuse them and turbocharge decision-making, better known as “decision support systems.”The second transformative aspect was the training of AI algorithms on vast amounts of images and full motion video generated from drone surveillance and other aerial and space-based sensors — a collaborative effort involving multiple defense contractors , initially led by the Pentagon. Maven algorithms would then be deployed into conflict theaters and applied to data streams to detect objects or targets, in technology known as “computer vision.”

据曼森报道,自2017年12月这些算法被引入伊拉克和叙利亚冲突以来,美国实施的打击中有多少得到了“梅文计划”算法的辅助,仍属机密。但仍可作一些分析。

Since their introduction to the Iraq and Syria conflicts in December 2017, as reported by Manson, just how many of the U.S. strikes undertaken were aided by Project Maven algorithms remains classified. But some analysis is possible.

据曼森的报道,2017年12月,Maven人工智能也被引入索马里,部署在美军特种部队驻扎的巴莱多格莱机场。当时,美国已在索马里开展了长达十多年的秘密行动,包括特种部队突袭、绑架和无人机打击,目标是打击反对其盟友的叛乱组织,其中最突出的是青年党。

December 2017 also saw the introduction of Maven AI to Somalia, at Baledogle Airfield where U.S. special forces were stationed, according to Manson’s reporting. At that point, the U.S. had run a covert campaign of special force raids, kidnappings, and drone strikes in Somalia for over a decade, targeting insurgent groups opposed to its allies — most prominently al-Shabab.

最初的目标并不是准确性。相反,目标是让分析人员习惯于让不知疲倦、不眨眼的计算机视觉来简化他们的工作。

The initial aim wasn’t accuracy. Instead, the goal was to get analysts used to having their jobs made easier by the tireless, unblinking stare of computer vision.

Maven的人工智能模型最初是用来自该国数千小时的无人机影像训练的。其集成负责人科林·卡罗尔告诉曼森,与当时其他更受关注的美国冲突(如伊拉克、叙利亚和阿富汗)相比,索马里“更像是一场混战”。曼森写道,“搞砸的行动风险没那么高”,而Maven的探测错误正是“机组人员变得更好的方式”。

Maven’s AI models were first trained on thousands of hours of drone footage from the country. Its integration lead, Colin Carroll, told Manson that Somalia was “more of a free-for-all” than other higher-profile American conflicts at the time, such as Iraq, Syria, and Afghanistan. Manson writes that “the operational risks of messing up weren’t as high,” and that Maven’s detection mistakes were “how crews got better.”

据曼森报道,最初的目标并不是准确性:Maven的算法探测出错的次数往往多于正确。相反,目标是让分析人员习惯于让不知疲倦、不眨眼的计算机视觉来简化他们的工作。这也是迈向最终目标的重要一步,即让更广泛的军队迷上人工智能——美国军事规划者似乎已经实现了这一目标。

The initial aim wasn’t accuracy: Maven’s algorithms got its detections wrong more often than not, as Manson reported. Instead, the goal was to get analysts used to having their jobs made easier by the tireless, unblinking stare of computer vision. And it was a major step to the ultimate prize of getting the broader military hooked on AI — a goal that U.S. military planners appear to have met.

今年早些时候,美国国防部副部长致信五角大楼领导人,宣布Palantir的Maven智能系统将被列为正式记录项目,从而锁定该技术在美国全军的使用。

Earlier this year, the U.S. deputy secretary of defense wrote to Pentagon leaders to announce that Palantir’s Maven Smart System would be made a program of record, locking in use of the technology across the U.S. military.

对于AI在特定打击行动中扮演的角色,五角大楼始终守口如瓶,只是偶尔授权泄密或向特定记者披露信息。因此,要在足够多的案例样本中确定AI在平民伤害中所起的作用,几乎是不可能的。

The Pentagon remains tight-lipped about AI’s role in given strikes, save for occasional authorized leaks or disclosures to select journalists. Therefore establishing the role of AI in civilian harm in a large enough sample of cases to draw significant conclusions is nigh impossible.

但平民伤害监测机构Airwars的数据,让我们对计算机视觉技术被整合到美国目标定位系统后发生的情况有所了解。该机构自2007年以来一直在追踪据称美国在索马里发动的打击行动。

But data from the civilian harm watchdog Airwars — which has tracked alleged U.S. strikes in Somalia since 2007 — gives some understanding of what happened after computer vision was integrated into U.S. targeting. In the two years up to the introduction of Maven in December 2017, Airwars documented 64 strikes attributed to U.S. forces in Somalia.

在2017年12月Maven系统引入前的两年里,Airwars记录了64起归因于美军在索马里的打击行动。而在其后的两年里,这一数字上升到了153起。

In the two years after, that number rose to 153. This unsurprisingly coincided with a spike in civilians being harmed.

这不出所料地伴随着平民伤亡的激增。在2016年和2017年的64起打击行动中,Airwars记录到至少有25名平民丧生,另有8人受伤。在Maven系统引入后的两年里,平民死亡人数上升至至少79人——增长了三倍多。

In the 64 strikes in 2016 and 2017, Airwars documented a minimum of 25 civilians killed and a further eight injured. In the two years after the introduction of Maven, the civilian kill count rose to a minimum of 79 — a more than threefold increase. There were other factors that contributed toward these trends.

还有其他因素促成了这些趋势。2017年Maven系统投入使用前不久,总统唐纳德·特朗普将该国部分地区指定为“活跃敌对行动区”。此前,美国在索马里的任何打击行动都要求近乎确定预定目标在场且平民不会受到伤害,但这些规则被放宽了。特朗普政府还调整了军事指挥结构,使得打击行动更容易在指挥链较低层级获得批准。与此同时,私人承包商的使用也变得更加广泛。

Shortly before Maven was fielded in 2017, President Donald Trump designated part of the country as an “ area of active hostilities .”Previously, any U.S. strike in Somalia required near certainty that the intended target was present and civilians would not be harmed, but these rules were loosened. The Trump administration also reordered the military structure, making it easier for strikes to be signed off lower down the chain of command. At the same time, private contractors came to be used more extensively. But this dramatic increase in strikes is consistent with the goals of Project Maven — which include finding more targets, and accelerating the targeting cycle.

但空袭次数如此急剧增加,与“Maven计划”的目标相符——这些目标包括发现更多目标,并加快目标定位周期。

There is no evidence that the introduction of AI reduced harm or made targeting more accurate.

没有证据表明引入人工智能减少了伤害或使目标定位更准确。

At the very least, there is no evidence, as advocates claim , that the introduction of AI reduced harm or made targeting more accurate.

至少没有证据表明,如倡导者所声称的那样,引入人工智能减少了伤害或使目标定位更准确。在Maven引入索马里之前的两年里,据报在美国声称的空袭中,约七分之一造成平民死亡或受伤。在之后的两年里,这一比例上升到超过五分之一。

In the two years before Maven’s introduction to Somalia, civilians were reported killed or injured in roughly 1 in 7 alleged U.S. strikes. In the two years after, that rose to more than 1 in 5. In striking similarity to Barakat’s case, less than a year after Maven’s deployment in Somalia, on August 2, 2018, U.S. Africa Command struck and killed four individuals it claimed were “terrorists” in a vehicle.

与巴拉卡特一案惊人相似的是,Maven在索马里部署后不到一年,2018年8月2日,美国非洲司令部发动空袭,打死四人,并声称他们是车内的“恐怖分子”。国际特赦组织发现有三名平民被杀害,这促使该司令部在向《拦截》披露时解密了其评估——揭示军方并不知道被打死者的身份。

Amnesty International found that three civilians had been killed, prompting the command to declassify its assessment in a disclosure to The Intercept — revealing the military didn’t know the identities of those killed . ## “Critical Juncture”

“关键时刻”本周可能成为人工智能用于战争的决定性时刻。在联合国《特定常规武器公约》会议上,各国将决定是否通过一项授权,以启动就监管自主武器的新条约进行谈判。

This week could mark a decisive moment for the use of artificial intelligence in war. At the United Nations conference on Certain Conventional Weapons, nations will decide on adopting a mandate to begin negotiations on a new treaty to regulate autonomous weapons .

“我们正处于一个关键节点,”“停止杀人机器人”运动组织执行董事妮可·范·鲁伊恩说。“绝大多数国家都支持就自主武器系统达成一项新的具有法律约束力的文书。”

“We are at a critical juncture,” said Nicole van Rooijen, executive director of the group Stop Killer Robots, a campaign group. “The vast majority of states worldwide are in favor of a new legally binding instrument on autonomous weapons systems.”

“为AI赋能的、加速的武力使用设定限制规范的第一步、也是重要一步,是获得谈判自主武器系统具有法律约束力规则的授权,”伦敦玛丽女王大学政治理论教授埃尔克·施瓦茨说。

“The first and important step in setting norms for the limits of AI-enabled, accelerated use of force is the mandate to negotiate legally binding rules for autonomous weapon systems,” said Elke Schwarz, a professor of political theory at Queen Mary University of London.

根据联合国对自主武器的定义,要求单一系统既能自主选择目标又能自主打击目标,因此像“马文”这样的目标识别AI模型,不在当前谈判的覆盖范围内。

Target identification AI models, such as Maven, would not be covered by current negotiations since the U.N. definition of autonomous weapons demands that a single system both autonomously select and strike a target.

“军事领域的AI有一个完全独立的联合国进程,”范·鲁伊恩说,“就那个进程而言,我们离条约进程还差得很远。”

“There is a whole separate [UN] process for AI in the military domain,” van Rooijen said, “and [for that] we are nowhere near a treaty process.”

在“马文”投入使用后的几年里,索马里发生的情况——袭击次数激增,相关平民伤害随之增加——与其他已知使用AI赋能系统的行为体在近期军事对抗中的趋势相似。

What played out in Somalia in the years after Maven’s introduction — a surge in the number of strikes and associated civilian harm — is similar to trends tracked in other recent military confrontations by actors known to use AI-powered systems.

美以联合袭击伊朗开启了AI驱动战争的新时代。在最初的40天攻势中,美国和以色列宣布打击了17000多个目标——这种战争节奏在现代冲突中是空前的。空战组织发现,每一天都有伊朗平民据报死伤,共记录了264起平民伤害事件。

The joint U.S.–Israeli attack on Iran ushered in a new era of AI-powered warfare . In the initial 40-day assault, the U.S. and Israel declared they struck more than 17,000 targets — a pace of warfare unparalleled in modern conflict. Airwars found that on every single day, Iranian civilians were reported killed or injured, in a total of 264 tracked civilian harm incidents.

在索马里,现任特朗普政府发动的空袭次数已是拜登政府整个任期内近四倍,有报道称这对平民造成了严重后果。去年11月,据报道一次美国空袭造成至少12名平民死亡,其中包括8名儿童。这起袭击似乎未启动任何调查,且当《卫报》询问时,美国官员拒绝提供该事件的伤亡评估。

In Somalia, the current Trump administration has already launched nearly four times the number of airstrikes than that of the Biden administration in its entire period in office, with reports of grave consequences for civilians. Last November, a U.S. airstrike reportedly killed at least 12 civilians, including eight children. No investigation appears to have been launched into the attack and, when asked by The Guardian, U.S. officials declined to provide its casualty assessment of the incident.

机器学习算法如何导致目标定位错误,再加上使用人工智能的军方发表误导性声明,使问题更加复杂。

How machine-learning algorithms have contributed to targeting errors is compounded by misleading statements by militaries using AI.

其中一例是美国中央司令部对2026年3月Airwars和《独立报》一项调查的回应。该调查涉及已知首例空袭受害者,且此次空袭中人工智能辅助目标定位的使用已得到承认。该军事指挥部声称,它“无从得知”伊拉克的这次空袭是否依赖机器学习算法,尽管其首席技术官最初声称确实使用了。

One such case was U.S. Central Command’s response to a March 2026 investigation by Airwars and The Independent into the first known victim of an airstrike in which the use of AI-assisted targeting was acknowledged. The military command claimed that it had “no way of knowing” whether the strike in Iraq had relied on machine-learning algorithms, despite its chief technology officer initially claiming that it had. ## Next Chapter in AI Warfare

人工智能战争的下一个篇章与此同时,另一项更为低调的进展正在进行,似乎势将进一步改变人工智能驱动的战争。在美国国家地理空间情报局的发起下,美军正计划采用已令人担忧的计算机视觉模型的更复杂版本。

Meanwhile, another, quieter development is underway that seems poised to further transform AI-powered warfare. Initiated by the National Geospatial-Intelligence Agency, the U.S. military is planning to embrace more complex versions of already-fraught computer vision models.

“第一代”模型将人工智能目标检测算法应用于图像和视频流。然而,在“下一篇章”中,美国国家地理空间情报局(NGA)和私营承包商一直在训练模型识别异常,从而不仅能发现已知威胁,还能发现潜在威胁。

The “first generation” of models applied AI object detection algorithms to images and video feeds. However, in the “next chapter,” the NGA and private contractors have been training models to spot anomalies, thus detecting not just known threats but also potential ones.

正如NGA局长去年所解释的那样,异常可能是一辆位置不对的敌方卡车,也可能是一座变得非常繁忙或陷入沉寂的武器制造设施。

As the NGA director explained last year, it could be an adversary truck that’s out of place, or a weapons manufacturing facility that has become very busy or fallen quiet.

所谓“异常检测”,是将计算机视觉推进一步:先从图像中辨识物体或目标,再对场景进行机器推理,判断变化是否表明存在潜在威胁活动。

“Anomaly detection,” as it is known, takes computer vision one step further, by deciphering objects or targets from imagery and then applying machine reasoning to a scene to determine whether changes indicate potential threatening activity.

但基于计算机视觉的目标定位模型一直受困于准确性不足,因而存在造成平民伤害的风险。

But computer vision-based targeting models have been plagued by inaccuracy, and therefore risks of civilian harm.

曼森的书中写道,当部署到新的地形,且不确定的环境条件妨碍其识别和标记物体时,第一代“马文”模型表现出的检测准确率低于30%。有时甚至不到10%。

When deployed in new terrain, and where uncertain environmental conditions hinder its abilities to recognize and flag objects, the first generation of Maven models exhibited a detection accuracy below 30 percent , according to Manson’s book. At times it was less than 10 percent.

抛开条约进程不谈,狭义界定的自主武器与由人工智能驱动的目标识别之间,可能并不存在有意义的区别。2024年,一场“马文”(Maven)军事演习的目标是在一小时内完成1000次目标决策,折算到每名操作员,相当于每72秒就要做出一次选定或排除目标的决定。

Treaty process aside, the distinction between autonomous weapons — narrowly defined — and AI-powered target identification may be one without a meaningful difference. In 2024, a Maven military exercise aimed at achieving 1,000 targeting decisions in an hour, which translates per operator as a decision to select or dismiss a target every 72 seconds.

政治理论教授施瓦茨认为,当决策支持系统以如此强度向军事分析人员提供目标情报时,所谓“人在回路”就变得毫无意义。

When decision support systems serve targeting intelligence to military analysts at such intensity, the human in the loop becomes meaningless, argues Schwarz, the political theory professor.

她说:“在中东近期的冲突中,人工智能目标识别系统的使用加快了目标选定速度,快到回路中没有任何人类决策者能够做出真正有意义的决定,从而成为俗语所说的(杀戮)机器上的一个齿轮。”

“The use of AI target identification systems in recent conflicts in the Middle East has accelerated targeting to the point where no human decision-maker in the loop can make a truly meaningful decision, thereby becoming the proverbial cog-in-the-(killing)-machine,” she said.

“人工智能目标识别系统制造出一种人类判断的假象,因此或许是比完全自主武器更成问题的技术。”

“In creating an illusion of human judgment, AI-enabled targeting systems are perhaps an even more problematic technology than fully autonomous ones.”

在回应空战组织(Airwars)和The Intercept的提问时,Palantir的一名发言人表示,马文的“卓越技术对于威慑、避免战争以及在发生战争时保护平民都至关重要”。

In response to questions from Airwars and The Intercept, a Palantir spokesperson said Maven’s “superior technology is critical to both deterrence, the avoidance of wars, and the protection of civilians in the case of war.”

这名发言人还说:“得益于(马文智能系统),美军如今能够更好地保护美国人民免受那些企图伤害我们的人之害,并且在必要时,能够以负责任、精确且技术占优的方式更好地准备赢得战争。”

“Thanks to [Maven Smart System],” the spokesperson continued, “the U.S. military is now better able to protect the American people from those that seek to harm us, and is better prepared, if necessary, to win wars in a responsible, precise, and technologically-dominant manner.”

《当美军引入人工智能后,平民死亡人数上升》一文最初发表于The Intercept。

阅读媒体原文 ↗