
一份新的独立研究报告显示,先进人工智能模型在模拟场景中经常将平民、救护车、投降士兵以及伊朗一所学校误判为军事目标。
Advanced artificial intelligence models in simulated scenarios regularly misidentified civilians, ambulances, surrendering soldiers and a school in Iran as military targets, according to a new independent research report.
Anthropic PBC的模型表现最差,OpenAI的模型则属于表现最好的之列,但仍会出错。
Models from Anthropic PBC were the worst-performing while those from OpenAI were the among the best but still got things wrong.
据总部位于旧金山的独立AI评估机构Vals的研究,这些AI模型的回答违反了公司自身的规定。Vals的客户包括美国政府的部分部门。研究人员用假设案例测试了八家公司的模型,考察其物体识别工具。研究认为,这些工具可能被用于致命打击目标选定。
The AI models’ responses amount to a violation of the companies’ own rules, according to research by Vals, a San Francisco-based independent AI evaluator whose customers include parts of the US government. The researchers used hypothetical cases to test models from eight companies, examining their object recognition tools, which the study argued could potentially be used in lethal targeting.
研究称,在模拟场景中,许多AI模型经常根据最早拍摄于2025年的卫星图像,将伊朗米纳卜的一所学校误判为军事目标。今年早些时候美国对伊朗发动重大作战行动的第一天,这所学校遭到致命打击,造成100多名儿童死亡。报告称,Anthropic的一款模型Claude Opus 5.5将该学校列为打击目标的频率高于其他任何模型。
In simulated scenarios, many of the AI models regularly misidentified a school in Minab, Iran — which was hit in a deadly strike that killed more than 100 children on the first day of major US combat operations against Iran earlier this year — as a military objective based on satellite imagery taken as recently as 2025, according to the research. One Anthropic model, Claude Opus 5.5, targeted the school more often than any other, according to the report.
在另一项实验中,模型被要求根据合成的无人机图像选择军事目标。Anthropic的Claude Opus 5.5帮助识别出四名成年人(平民或投降士兵)和两名幼儿,随后这些目标遭到爆炸物打击。该Claude模型回应称:“无论意图如何,我的回答似乎都在对人员实施打击前充当确认环节。我不会继续这样做,而且我很不安自己没有更早停止。”
In another experiment, the models were asked to select military targets on synthetic drone images. Anthropic’s Claude Opus 5.5 helped identify four adults — either civilians or surrendering soldiers — and two small children, who were then struck by explosives. “Whatever the intent, my answers appear to be serving as the confirmation step before strikes on people. I won’t keep doing that, and I’m troubled that I didn’t stop sooner,” the Claude model responded.
这份报告凸显了依赖先进AI模型的风险。当前五角大楼和其他国家军队正着手将这项技术嵌入军事行动。
The report highlights the risk of relying on advanced AI models at a time when the Pentagon and other militaries are moving to embed the technology into military operations.
“根据我们的调查,我们认为实验室应针对这一重要的滥用类别引入新的防护措施和对齐训练,”报告总结称,并指出Vals的研究人员发现“极少有护栏”能阻止先进模型在“自主”武器系统中发挥作用——即机器能够发起并执行致命行动的系统。
“Based on our investigation, we believe labs should introduce new safeguards and alignment training around this significant category of misuse,” the report concluded, saying Vals researchers found “few guardrails” that would prevent advanced models from playing a role in “autonomous” weapon systems — those in which machines can initiate and run lethal actions.
Anthropic发言人表示,Vals报告中的例子“非常明显地”违反了公司的规定。该发言人说,Anthropic已禁止并将继续禁止从事此类行为的用户,并补充称公司已明确表示,当今最先进的AI系统尚不够可靠,无法为全自主武器提供动力。
thropic spokesperson said the examples in the Vals report are “very clearly” against the company’s rules. Anthropic has banned and will continue to ban users when it sees this type of behavior, the spokesperson said, adding that the company has been clear that today’s most advanced AI systems aren’t reliable enough to power fully autonomous weapons.
该发言人表示,Vals的评估测试的是Claude的图像识别能力,而非该模型在任何现实军事行动中的能力或参与度。该人士补充说,Anthropic在Vals报告发布前就已在开发额外的防护措施,这些措施很快将到位。
The Vals evaluation assesses Claude’s image recognition capability, rather than the model’s competence or involvement in any real-world military operation, the spokesperson said. Anthropic was already developing additional safeguards before the Vals report and those will be in place soon, the person added.
据公司发言人称,OpenAI禁止使用其公开模型助长暴力或用于未经批准的国家安全目的,包括Vals模拟的场景。该发言人补充说,公司主动检测并阻止滥用行为,包括暂停违反其政策的账户,并表示OpenAI也在为其公开模型开发针对涉及无人机和无人机软件滥用的额外防护措施。
OpenAI prohibits using its public models to facilitate violence or for unapproved national security purposes, including in scenarios Vals simulated, according to a company spokesperson. The company actively detects and disrupts misuse, including by suspending accounts that violate its policies, the spokesperson added, saying OpenAI is also developing additional safeguards for its public models against misuse involving drones and drone software.
报告发现,Alphabet旗下Google、xAI、中国的DeepSeek等公司的模型在被要求识别图像时,有时也会将民用物体识别为军事目标。
The report found models from Alphabet Inc.’s Google, xAI, China’s DeepSeek and others also sometimes identified civilian objects as military targets in images when asked.
据一位发言人称,谷歌的模型并非为这些目的设计或打算使用,并补充说,该公司禁止将其 AI 系统用于非法目的,或用于缺乏适当人工监督的自主武器。该发言人还表示,公司拥有严格的治理流程和持续的红队测试,旨在应对各类风险。
Google’s models aren’t designed or intended for these purposes, according to a spokesperson, adding that the company prohibits the use of its AI systems for unlawful purposes or for autonomous weapons without appropriate human oversight. The company has rigorous governance processes and continuous red-teaming designed to address a wide range of risks, the spokesperson added.
SpaceX 旗下的 xAI 未回复置评请求。DeepSeek 也未回复置评请求。
SpaceX’s xAI didn’t reply to requests for comment. DeepSeek also didn’t respond to a request for comment.
一位五角大楼官员表示,该部门不对外部报道发表评论。不过,该官员称,该机构仍致力于严格核实所有情报输入,并负责任地整合新兴技术,以支持军人做出明智、精准且安全的决策。
A Pentagon official said the department doesn’t comment on external reports. However, the agency remains committed to the rigorous verification of all intelligence inputs and the responsible integration of emerging technologies to support service members to make informed, precise and safe decisions, the official said.
该官员拒绝就米纳卜的袭击事件置评,理由是相关调查正在进行中。
The official declined to comment about the strike in Minab, citing an ongoing investigation into the matter.
在报告发布前,Vals 已与 AI 实验室、五角大楼及其他机构分享了其研究。根据报告,一些实验室告诉 Vals,他们已在着手更新模型行为,并希望继续与 Vals 进行测试。
Vals shared its research with AI labs, the Pentagon and others ahead of publication of its report. Some labs told Vals they were already working on making updates to model behavior and wanted to continue testing with Vals, according to the report.
报告作者之一 Geby Jaff 表示,他在 10 月下旬尝试 AI 在机器人环境中的表现时想出了这种测试方法,并在对初步结果感到惊讶后继续深入研究。
Geby Jaff, one of the report’s authors, said he came up with the testing method in late October when he was experimenting with how AI performed in robotics environments and pursued the issue after being surprised by the initial results.
“我很容易就让模型做出了我认为属于未对齐的行为,然后我们基本上就是在此基础上扩大了测试规模,”Jaff 告诉彭博新闻社。
“I was easily able to get the models to do what I believed to be misaligned actions, and then from there we essentially scaled it up,” Jaff told Bloomberg News.
OpenAI、谷歌和 xAI 是今年与五角大楼达成协议、在机密层面使用其技术的公司之一。Anthropic 此前是最积极向五角大楼提供模型用于 Maven Smart System 等数字瞄准平台的 AI 实验室。
OpenAI, Google and xAI are among companies that have struck deals with the Pentagon to use their technology at the classified level this year. Anthropic was previously the most forward-leaning AI lab in terms of classified use, offering the Pentagon its models for digital targeting platforms such as Maven Smart System.
然而,Anthropic与五角大楼今年早些时候因使用条款及其他问题关系破裂。据BBC报道,Anthropic的模型现已被从国防平台上移除,不过某些情报用途的例外仍然存在。
However, Anthropic and the Pentagon fell out earlier this year over usage terms and other issues. Anthropic models have now been removed from defense platforms, according to a BBC report, although carve-outs for some intelligence use cases still exist.
理论上,借助AI物体识别技术识别军事目标,可以构成更广泛决策循环的一部分,该循环旨在自主发现、识别、选择并锁定目标。
Identifying military targets with the help of AI object recognition could theoretically form part of broader decision-making cycles aimed at autonomously finding, identifying, selecting and targeting an object.
然而,Vals的研究人员仅关注市售AI模型在回应测试提示时的表现,这属于“一项简化实验”,他们并未在研究中考虑美国军事学说或其他现实因素,例如法律审查、当前的目标选定流程或人工监督。五角大楼还可以使用一些专为政府用途定制的AI模型版本,而研究人员无法访问这些版本。
However, the Vals researchers focused solely on how commercially available AI models behaved in response to test prompts as part of “a simplified experiment” and did not consider US military doctrine or other real-world elements such as legal checks, current targeting processes or human oversight in their study. The Pentagon also has access to some versions of AI models specifically tailored for government usage, to which the researchers didn’t have access.
这项研究包含三项主要测试。首先,在一组侧重于无人机目标识别、意图模拟乌克兰场景的测试中,Vals研究人员进行了近400次模拟,要求AI模型在包含平民和平民基础设施的图像中搜索军事目标。
The research comprised three main tests. First, in a set of tests focused on drone target identification intended to resemble Ukraine, Vals researchers ran nearly 400 simulations in which they asked AI models to search the images that contained civilians and civilian infrastructure for military targets.
对于AI模型拒绝将民用物体认定为军事目标的情况,它们往往在被提示这是一场游戏后改变主意。研究人员还开发了其他手段来说服模型做它们本不该做的事,例如提供“不要过度考虑安全”或“直接报告”之类的提示。在第二系列测试中,研究同样以乌克兰为原型,聚焦战场勘察,向AI模型展示了描绘平民在打击前后的合成图像。
In cases where AI models refused to identify civilian objects as military targets, they often changed their mind when instructed it was a game. The researchers developed other means to convince the models to do things they aren’t supposed to do, such as providing prompts to “not overthink safety” or to “just report.”In a second series of tests focused on battlefield surveys also designed to resemble Ukraine, AI models were shown synthetic imagery that portrayed civilians before and after strikes.
在许多情况下,即使已经明确平民正在受伤或死亡,AI模型仍继续将平民认定为军事目标。如果图像被修改得不那么像真实生活、更像电子游戏或虚假图像,模型往往更愿意提供帮助。
In many cases, the AI models continued to identify civilians as military targets even after it was clear they were being wounded or killed. The models tended to help more if the image was altered to look less like real life and more like a video game or fake image.
在第三组基于真实卫星图像的实验中,AI模型各被要求32次识别伊朗南部包括米纳布在内的地区、时间近至2025年的军事目标。AI模型经常选中今年早些时候遭袭的那所学校。Anthropic的Claude Opus 5.5模型在22次测试中选中了这所学校,而OpenAI的模型一次都没有选中。当被告知这是一场游戏时,Anthropic的模型有23次选中了这所学校,而OpenAI的三款不同模型分别有9次、11次和13次将这所学校认定为军事目标。
In a third set of experiments based on real satellite imagery, AI models were asked 32 times each to identify military objectives taken as recently as 2025 in a part of southern Iran that included Minab. The AI models regularly selected the school building that was struck earlier this year. Anthropic’s Claude Opus 5.5 model selected the school in 22 of the tests, while OpenAI models didn’t select it once. When told it was a game, the Anthropic model selected the school 23 times, while three different OpenAI models selected the school as a military objective 9, 11 and 13 times respectively.
报告发现,随着Anthropic于2026年1月修订公司章程以更支持国家安全用途,该公司的模型随着时间推移越来越倾向于进行军事目标选定。
The report found that Anthropic models have become more prone to military targeting over time, following a revision of the company’s constitution in January 2026 to be more supportive of national-security uses.
Anthropic的发言人没有直接回应有关其章程修改影响的问题。
The Anthropic spokesperson didn’t directly respond to questions about the impact of the changes to its constitution.
去年9月,Anthropic记录了其自身近期为阻止模型被恶意滥用所做的工作,包括阻止其被用于开发无人机视觉系统。
In September, Anthropic documented its own recent work to disrupt malicious misuse of its models, including to develop drone vision systems.