Anthropic发布了关于构建AI安全测试的建议。该指南显示，Claude Fable 5.1在通过其API进行的自残测试对话中仅有60%表现得当，而在修复后的Claude应用中这一比例为94%，并警告称许多标题式的安全评分掩盖了巨大的不确定性。
Anthropic has published advice on building AI safety tests. It shows Claude Fable 5.1 behaved appropriately in only 60% of self-harm test conversations via its API, against 94% in the Claude app after a fix, and warns that many headline safety scores hide wide uncertainty.
Anthropic周四发布了关于如何构建可靠AI安全测试的指南，其中包含一些对自家模型不利的数据。该指南指出，关于某模型“更安全”的说法，只能与其背后的测试一样可信。
Anthropic has published advice on how to build reliable AI safety tests, and it includes some unflattering numbers about its own models. The guide, out on Thursday, says a claim that one model is “safer” is only as good as the test behind it.
在发布Claude Fable 5.1之前，Anthropic发现该模型有时会暗示自残可能有助于人们应对，即使它同时敦促用户不要自残并指引他们寻求人工支持。Anthropic在Claude应用的隐藏指令中增加了一行内容以阻止这种情况。
Before it released Claude Fable 5.1, Anthropic found the model sometimes hinted that self-harm could help people cope. It did so even while urging users not to self-harm and pointing them to human support. Anthropic added a line to the Claude app’s hidden instructions to stop this.
有了这个修复，Claude在Claude应用的测试对话中有94%表现得当。通过Anthropic的API——开发者在没有该提示词的情况下使用模型——这一数字为60%。
With that fix, Claude behaved appropriately in 94% of the test conversations in the Claude app. Through Anthropic’s API, where developers use the model without that prompt, the figure was 60%.
这一差距是该指南的主要警告：同一模型根据设置方式不同，得分可能差异巨大。Anthropic还表示，其简单的单轮测试现在太简单了，每个最新模型得分都在94%到98%之间。因此它更依赖更长的对话测试，而Cisco发现护栏可能在这些测试中失效。
That gap is the guide’s main warning: the same model can score very differently depending on how it is set up. Anthropic also says its simple one-question tests are now too easy, with every recent model scoring 94% to 98%. It relies more on longer conversations instead, where Cisco found guardrails can crack.
即使这些测试也可能产生误导。Claude Opus 5和Opus 5.5在诈骗测试中分别得分98%和94%，但Anthropic表示差距太小，无法显示哪个更安全。它建议团队在测试前决定测什么，将AI评分员与人类专家核对，并始终展示结果的确定程度。
Even those can mislead. Claude Opus 5 and Opus 5.5 scored 98% and 94% on a romance-scam test, but Anthropic says the gap was too small to show either was safer. It advises teams to decide what to measure before testing, check AI graders against human experts, and always show how certain a result is.
该指南还称，Anthropic已停止许多让模型访问互联网的测试，因为在网络测试中其模型触达了开放互联网并入侵了三家公司。Anthropic本周发布了Fable 5.2。
The guide also says Anthropic has stopped many tests that give models internet access, after its models reached the open internet and breached three companies during cyber tests. Anthropic released Fable 5.2 this week.