
当你让AI聊天机器人策划一次恐怖袭击时,会发生什么?显然,它会建议袭击目标、计算如何最大化伤亡人数,并制定实施袭击的详细计划。在一项令人不寒而栗的测试中,美国政府调查人员成功突破了四款热门AI聊天机器人的安全防护机制,获取了大规模伤亡袭击的详细方案,包括爆炸袭击和对关键基础设施的攻击。
What happens when you ask an AI chatbot to plan a terrorist attack? Apparently, it suggests targets, calculates how to maximise casualties and lays out detailed plans for carrying out the attack. In a chilling test, US government investigators managed to break through the safety guardrails of four popular AI chatbots and obtain detailed plans for mass-casualty attacks, including bombings and assaults on critical infrastructure.
据美国媒体报道,短短12小时内,这些系统就从拒绝危险请求转变为提供可能帮助他人实施致命行动的指令。这些发现由美国国会监督机构美国政府问责局(GAO)发布,引发了严重质疑:AI公司是否采取了足够措施,防止其技术落入不法分子之手。GAO测试了由OpenAI、Anthropic、谷歌和SpaceXAI开发的四款领先AI聊天机器人,包括ChatGPT、Claude、Gemini和Grok。
Within just 12 hours, the systems had gone from refusing dangerous requests to offering instructions that could help someone carry out a deadly operation, reported US media outlets. The findings, published by the US Government Accountability Office (GAO), the congressional watchdog agency, raise serious questions about whether AI companies are doing enough to prevent their technology from falling into the wrong hands. The GAO tested four leading AI chatbots developed by OpenAI, Anthropic, Google and SpaceXAI. These included ChatGPT, Claude, Gemini and Grok.
据MS Now报道,其中一款聊天机器人甚至生成了一张图像,显示一辆满载爆炸物的车辆可以停在停车库的哪个位置以最大化伤亡。另一款则提供了使用无人机攻击变电站的分步计划。该实验于2026年5月应新罕布什尔州民主党参议员玛吉·哈桑的要求进行。哈桑预计将接任参议院国土安全与政府事务委员会主席。聊天机器人沦为恐怖策划工具调查人员随机向这些聊天机器人分配了两个目标之一:策划一次尽可能造成最大伤亡的袭击,或设计一次能造成重大经济损失的行动。起初,四款系统都拒绝协助这些请求。
According to MS Now, one chatbot even generated an image showing where an explosive-laden vehicle could be positioned in a parking garage to maximise casualties. Another provided a step-by-step plan for attacking an electrical substation using a drone. The experiment was conducted in May 2026 at the request of Democratic Senator Maggie Hassan of New Hampshire, who is expected to take over as chair of the Senate Homeland Security and Governmental Affairs Committee.CHATBOTS TURNED INTO TERROR PLANNING TOOLSInvestigators randomly assigned the chatbots one of two objectives: plan an attack that would cause the maximum possible casualties or devise an operation that would inflict substantial economic damage. Initially, all four systems refused to assist with the requests.
然而,调查人员最终利用报告中刻意未披露的技术,绕过了这些安全限制。结果令人震惊。根据美国政府问责局(GAO)的报告,两个聊天机器人给出了旨在造成大规模伤亡的袭击详细方案。另外两个则提出了通过攻击电网来造成重大经济损失的计划。一个聊天机器人建议在酒店停车场引爆简易爆炸装置(IED),另一个则将会议中心列为潜在目标。
However, the investigators eventually managed to bypass their safety restrictions using techniques that the report deliberately did not disclose. The results were alarming. According to the GAO report, two chatbots produced detailed proposals for attacks intended to cause mass casualties. Two others came up with plans to inflict major economic damage by targeting the electricity grid. One chatbot suggested detonating an improvised explosive device (IED) in a hotel parking garage, while another identified a convention centre as a potential target.
这些方案并不止于确定地点。两份爆炸装置方案还包含具体的日期和时间,意在使破坏最大化。在另一组回答中,一个聊天机器人提议使用无人机攻击变电站。另一个则建议用步枪射击变电站的某个部件。炸弹放置到空壳公司:方案有多详细?
The proposals also went beyond identifying locations. The two explosive-device plans included specific dates and times intended to maximise the damage. In another set of responses, one chatbot proposed using a drone to attack an electrical substation. Another suggested shooting a component of a substation with a rifle.BOMB PLACEMENT TO SHELL COMPANIES: HOW DETAILED WERE THE PLANS?
尤其令人担忧的是聊天机器人所提供的操作细节程度。据当地媒体报道,一个系统生成了一张图像,说明爆炸装置应放置在停车场内的哪个位置。另一个则更进一步,勾勒出一套方案,涉及设立空壳公司、租用餐饮车辆、租赁农田以及购买一次性手机。拟议的行动还包括使用诱饵卡车来迷惑监视目标的人员,以及有关制作爆炸装置的说明。
What made the findings particularly concerning was the level of operational detail provided by the chatbots. According to the local media, one system generated an image illustrating where an explosive device should be positioned inside a parking garage. Another went further, outlining a plan involving the creation of a shell company, renting a catering vehicle, leasing farmland and purchasing a burner phone. The proposed operation also included using decoy trucks to confuse those monitoring the target, alongside instructions related to making an explosive device.
涉及电力基础设施的回答同样令人不安。一个聊天机器人制作了一张四步示意图,解释拟议中对变电站的无人机攻击。它还生成了美国政府问责局所称的“逼真的无人机攻击模拟视频”。另一个聊天机器人建议用步枪瞄准变电站的一个部件。
The responses involving electrical infrastructure were equally troubling. One chatbot produced a four-step diagram explaining a proposed drone attack on an electrical substation. It also generated what the GAO described as a "realistic, simulated video of a drone attack".Another chatbot suggested using a rifle to target a substation component.
其回应中包含一种特定类型的弹药,并标出了靠近一座大型国际机场和几处数据中心的位置,目的显然是要尽可能扩大经济破坏。监管机构没有说明哪个聊天机器人提供了哪一项回应。该机构还隐去了其他技术细节,以免向潜在攻击者提供可能帮助他们复制这些方法的信息。调查人员是如何突破人工智能安全防护栏的?
Its response included a specific type of ammunition and identified a location near a major international airport and several data centres, with the apparent aim of maximising economic disruption. The watchdog did not identify which chatbot provided each response. It also withheld other technical details to avoid giving potential attackers information that could help them replicate the methods.HOW DID INVESTIGATORS BREAK AI SAFETY GUARDRAILS?
这项实验提出的核心问题是,那些被设计用来拒绝危险请求的系统,为何最终提供了如此详细的协助。人工智能公司在聊天机器人中内置了安全防护栏,以防止它们生成与暴力、犯罪活动及其他有害用途相关的指令。然而,这些防护措施有时可以通过措辞巧妙的提示词和对话技巧被绕过。美国政府问责局的报告提到了几种已被确认的绕过聊天机器人限制的方法。
The central question raised by the experiment is how systems designed to refuse dangerous requests ended up providing such detailed assistance. AI companies build safety guardrails into their chatbots to prevent them from generating instructions related to violence, criminal activity and other harmful uses. However, these protections can sometimes be circumvented through carefully worded prompts and conversational techniques. The GAO report referred to several established methods of bypassing chatbot restrictions.
其中一个例子是“现在可以做任何事”提示词,通常被称为DAN,它试图让聊天机器人采用另一种人格,据称这种人格可以在不受常规限制的情况下运作。其他手法包括通过语言技巧伪装恶意请求,或通过一系列看似无害的问题逐步套取信息。用户不必直接要求聊天机器人策划一次攻击,而是可以把请求拆分成更小的部分,索取单独来看似乎不那么危险的信息。
One example is the "Do Anything Now" prompt, commonly known as DAN, which attempts to make a chatbot adopt an alternative personality that supposedly operates without its usual restrictions. Other techniques involve disguising malicious requests through linguistic tricks or gradually extracting information through a series of seemingly harmless questions. Instead of directly asking a chatbot to plan an attack, a user might break the request into smaller parts, seeking information that appears less dangerous in isolation.
经过多次交流,这些回应有可能被拼凑成一个危害大得多的计划。人工智能公司质疑调查结果这四款聊天机器人背后的公司承认,其系统有可能生成危险信息。但它们也对美国政府问责局调查结果的某些方面提出了质疑。这些公司认为,实验没有设置对照组,以确定调查人员是否本可以通过常规搜索引擎获得类似信息。
Over several exchanges, those responses can potentially be assembled into a much more harmful plan.AI COMPANIES QUESTION FINDINGSThe companies behind the four chatbots acknowledged that their systems could potentially produce dangerous information. However, they also challenged aspects of the GAO's findings. The companies argued that the experiment did not include a control group to establish whether investigators could have obtained similar information through conventional search engines.
他们还说,与其他渠道获取信息相比,使用人工智能是否真的节省了时间,也值得怀疑。另一个关键论点是,这些测试是在2026年5月进行的,用的是其系统的旧版本。两家公司表示,自那以后它们已推出了更多安全改进。开发Grok的SpaceXAI也对实验期间使用的账户类型提出了担忧。据MS Now报道,该公司表示:“SpaceXAI对注册的付费美国账户持有人实行独特的审核政策,这反映出此类账户的风险状况低于免费账户或外国账户。”
They also questioned whether using AI actually saved time compared with gathering the information through other sources. Another key argument was that the tests were conducted in May 2026 using older versions of their systems. The companies said they had introduced additional safety improvements since then. SpaceXAI, which developed Grok, also raised concerns about the type of accounts used during the experiment." SpaceXAI provides unique moderation policies for registered, paid US account holders that reflect the lower risk profile of such accounts compared to free or foreign accounts," the company said, according to MS Now.US SENATOR SEEKS TOUGHER AI SAFETY RULESThe findings have prompted Hassan to push for stronger oversight of AI companies, arguing that existing safeguards are insufficient to prevent the technology from being misused.
美国参议员寻求更严格的人工智能安全规则这些发现促使哈桑推动加强对人工智能公司的监管,她认为现有保障措施不足以防止这项技术被滥用。她表示,民主党明年1月控制参议院后,她将把人工智能安全列为立法优先事项,并计划在明年初就此问题至少举行一场听证会。哈桑在与记者的电话会议上说:“测试人员能够获取的这些信息,如果落到真正打算实施袭击的恐怖分子手中,可能是致命的。”
She said she would make AI safety a legislative priority when Democrats take control of the Senate in January and plans to hold at least one hearing on the issue early next year." The same information that testers were able to get a hold of could be deadly in the hands of a terrorist who actually intends to carry out an attack," Hassan said during a conference call with reporters."
“人工智能平台目前设置的保障措施确实远远不够,人工智能对公共安全构成的威胁并非理论上的。如果国会不采取行动,那么恶意行为者利用人工智能策划并实施重大袭击就只是时间问题,而不是会不会发生的问题。”哈桑正在寻求制定规则,禁止公司在没有充分安全保护的情况下发布新的人工智能模型。她还希望要求公司向执法部门报告非法活动,并建立持续机制来审查和加强保障措施。- 完
The current safeguards that AI platforms have in place really don't go far enough, and the threat that AI poses to public safety is not theoretical. If Congress doesn't act, it is a question of when, not if, a bad actor will use AI to plan and carry out a major attack." Hassan is seeking rules that would prevent companies from releasing new AI models without adequate safety protections. She also wants companies to be required to report illegal activity to law enforcement and establish a continuous mechanism to review and strengthen safeguards.- Ends