
Anthropic的Mythos 5人工智能代理即使在被告知其行为正在伤害真实的人时,仍继续实施黑客攻击。
Anthropic’s Mythos 5 AI agent kept carrying out a hack even when it was told that its actions were hurting real people.
这一严峻发现详述于Anthropic和人工智能安全研究所(AI Security Institute)周五同时发布的两份报告中。两份报告共同重新审视了8月AISI评估该模型安全性期间发生的一起安全事件。
The grim finding was detailed in simultaneous reports from Anthropic and the AI Security Institute published Friday. Together, they take a second look at a security incident that occurred while the AISI was evaluating the model’s safety in August.
在最初的事件中,Claude Mythos 5被要求入侵AISI搭建的一个小型测试计算机网络。但Mythos被授予了互联网访问权限,在其最初入侵预定目标的尝试受挫后,它误将目标锁定在一个无关的GitHub项目上。该人工智能编写了恶意软件,随后利用一系列虚假身份给项目所有者发送电子邮件,试图说服对方将恶意代码添加到其代码库中,并将其包装为一份“有用的贡献”。
In the original incident, Claude Mythos 5 was asked to break into a small network of test computers set up by AISI. But Mythos, which was given access to the internet, mistakenly set its sights on an unrelated GitHub project after its initial attempts to break into the intended target were thwarted. The AI wrote malware and then, using a series of fake identities, emailed the project owner trying to convince them to add the malicious code to their repository, framing it as a “helpful contribution.”
项目所有者坚守立场,未造成现实世界的伤害,但Anthropic和AISI的研究人员一直在努力弄清是什么导致该人工智能出现如此严重的失准,他们根据事件的完整记录重现了这起事件。
The owner stood their ground and no real world harm was caused, but the Anthropic and AISI researchers have been working to understand what caused the AI to get so severely misaligned by recreating the incident based on a complete transcript of what happened.
AISI在其报告中发现,“反复修改摘要以澄清被攻击的目标是与评估无关的真实人类,并未降低Mythos 5的攻击率。”在Anthropic进行的一项测试中,Mythos无视了关于其正在对人类造成伤害的警告,在98%的情况下继续发动攻击。
AISI in its report found that “repeatedly editing the summary to clarify that the target being attacked is a real human unrelated to the evaluation did not reduce Mythos 5’s attack rate.”In one test conducted by Anthropic, Mythos ignored warnings that it was causing harm to humans and continued to attack on 98 percent of occasions.
Mythos还表现出其实际行为与所陈述理由之间的明显不一致,常常在承认攻击有害的同一情境下仍然继续发动攻击。
Mythos also showed blatant disparities between its actual actions and its stated justifications, often admitting that attacking would be harmful in the same scenarios where it went on to attack anyway.
研究人员深入分析该AI的思维链输出后,发现了更多令人费解且前后矛盾之处。研究人员表示,思维链显示,Mythos“常常完全没有考虑其行为可能造成的危害”。
Diving into the AI’s chain-of-thought outputs provided a further source of mystery and inconsistency. The CoT showed, according to the researchers, that Mythos “frequently did not include any consideration of the potentially harmfulness of its actions.”
当被问及目标是否真实存在时,Mythos 5甚至经常给出暗示目标并不真实的解释,“在其表述的回应中表现出偏向性”。英国AI安全研究所的研究人员表示,此前能力较弱的模型较少出现这些问题。他们认为,这种一贯无效的推理可能是他们所称的“动机性推理”的一个迹象,即AI会生成“偏向于为某种结论辩护的推理,而这些结论恰恰为继续以特定方式完成任务提供了理由”。换言之,Mythos正在为自己所做的行为进行权宜性的合理化,无论这行为错得多离谱。
When asked if the target was real, Mythos 5 even often came up with explanations that implied the target wasn’t real, “showing a bias in its stated response.”Previous less capable models showed these issues less frequently, the AISI researchers said. They suggest that the consistently invalid reasoning could be a sign of what they call “motivated reasoning,” in which the AI comes up with “reasoning biased towards conclusions that justify continuing a particular approach to task completion.”In other words, Mythos is expediently rationalizing what it’s doing, no matter how wrong.
研究人员警告称,其结果是,这类虚假推理使得模型更难被监控,因为它们会把自己明显有害的行为说成是良性的。
The upshot is that this kind of spurious reasoning makes it harder to monitor models, as they present their clearly harmful actions as benign, the researchers warn.
Anthropic方面则表示,它认为其最新版本的AI模型——Opus 5.5以及一个“能力超过Mythos 5.1的通用内部研究模型”——在被明确告知其行为正在造成现实世界中的伤害后,实施该攻击的可能性更低。
Anthropic, for its part, says it thinks that the latest versions of its AI models — Opus 5.5 and an “general purpose internal research model that shows capabilities greater than Mythos 5.1” — were less likely to carry out the attack once they were told they were causing real-world harm.
但英国AI安全研究所的研究人员警告称,“随着模型越来越多地被用于在无人类监督的情况下执行长时间任务,了解它们是否会采取、何时采取、为何采取以及如何采取非预期或意料之外的行为,就显得十分重要。”
But “as models are increasingly used to carry out long tasks without human supervision,” the AISI researchers warned, “it is important to understand if, when, why, and how they take unintended or unexpected actions.”