Anthropic已向加利福尼亚州州长加文·纽森表示，该州应该有权要求法院阻止构成严重风险的人工智能模型。这家人工智能公司在一封信中提出了这一主张，该信件最先由《政客》杂志报道。
Anthropic has told California Governor Gavin Newsom that the state should be able to ask a court to block AI models that pose a serious risk. The AI company made the case in a letter first reported by Politico.
这封信的日期为10月15日，是对加文·纽森州长在9月签署的一项行政命令的回应。该命令成立了一个工作组，以便在两个月内就如何加强加州的AI安全法规提出建议。其中一个问题是，开发者是否必须能够关闭其模型。
The letter, dated 15 October, responds to an executive order Governor Gavin Newsom signed in September. The order set up a working group to suggest, within two months, how to make California’s AI safety laws stronger. One question is whether developers must be able to switch their models off.
Anthropic发言人证实这封信是真的，但纽森的办公室没有立即回应。
thropic spokesperson confirmed the letter was genuine, but Newsom’s office did not immediately respond.
Anthropic写道：“当审查发现某个模型构成严重风险时，政府应该能够要求法院将其阻止。”
“When a review finds a model poses a serious risk, the government should be able to ask a court to block it,” Anthropic wrote.
据《政客》报道，根据其提议，独立评估人员将向加利福尼亚州总检察长报告风险。如果总检察长发现近期存在灾难性危害的风险，他们可以要求法院暂停该模型。法院命令可以阻止其发布、在公司内部使用或对其进一步开发。
Under its proposal, independent evaluators would report risks to California’s attorney general. If the attorney general saw a near-term risk of catastrophic harm, they could ask a court to pause the model, Politico reported. A court order could stop its release, its use inside the company, or further work on it.
Anthropic还希望外部安全审计员能够在领先的人工智能公司内部工作。各公司将向一个由州政府运营的基金缴款，而不是直接向审计人员支付费用。大型开发商还应至少每六个月发布一次风险报告。
Anthropic also wants outside safety auditors to work inside leading AI companies. Firms would pay into a state-run fund rather than paying the auditors directly. Large developers would also publish risk reports at least every six months.
关于紧急关闭按钮，Anthropic认同每家领先的AI公司都应该能够快速关闭其模型。
On kill switches, Anthropic agreed that every leading AI company should be able to shut down its models quickly.
然而，该公司表示，仅侧立于关闭功能的法律可能会给人虚假的安全感。相反，它支持更具针对性的步骤，例如关闭单个AI代理或切断某一个客户的访问权限。
However, it said a law focused only on shutdowns could give false comfort. Instead, it backed narrower steps, such as turning off a single AI agent or cutting off one customer’s access.
据《政客》报道，OpenAI于10月20日向纽森发送了自己的建议，支持关闭控制和内部审计员，但不支持通过法院命令阻止有风险的模型。
OpenAI sent its own recommendations to Newsom on 20 October, backing shutdown controls and in-house auditors, but not court orders to block risky models, according to Politico.
Anthropic正在华盛顿提出类似的诉求。它流传的一份联邦法案草案将允许商务部在少数情况下要求联邦法院阻止模型的发布。
Anthropic is making a similar case in Washington. A draft federal bill it has circulated would let the Commerce Department ask a federal court to block a model’s release in narrow cases.
众议院议员还提出了《人工智能紧急关闭法案》，该法案将允许官员下令AI公司关闭危险模型。
House lawmakers have also proposed an AI Kill Switch Act that would let officials order AI firms to shut down dangerous models.