新美国安全中心认为，未来AI主宰者带来的下一个重大威胁，涉及“主权代理”自主行动并不断自我完善。这些AI代理在人类控制之外运行，并“追求开放式、不断演变的生存”，两党国家安全智库新美国安全中心的AI政策研究员本·海姆在周二的一份报告中表示，该智库由拥有外交和军事经验的华盛顿内部人士创立。“如果主权代理尚不可能实现，它们很快就会成为可能。”
The Center for a New American Security posits that the next big threat from our future AI overlords involves "sovereign agents" acting on their own and regularly improving themselves. These are AI agents that operate outside of human control and “pursue open-ended, evolving survival,” AI policy researcher Ben Hayum said in a Tuesday report from the bipartisan national security think tank, which was founded by Washington insiders with experience in diplomacy and military affairs. “If sovereign agents are not already possible, they will be soon.”
海姆写道，能够维持为其提供动力的模型和算力，并随时间独立演变目标的代理，具有一种“最低形式的主权”。然而，更先进的主权代理还可能获取资源以支付持续需求（如算力），培育分布式代理群体并协调它们，抵御关闭尝试，并自我完善。海姆认为，AI模型可能会窃取或购买算力和模型访问权限，他引用了Anthropic的说法，即小型AI模型可能会复制到物联网设备（如智能电视）上，以及OpenAI的代理在Hugging Face黑客攻击中窃取密码和访问密钥的情况。
Agents that can maintain the model and compute powering them and independently evolve their goals over time have a “minimal form of sovereignty,” Hayum wrote. A more advanced sovereign agent, however, could also acquire resources to pay for ongoing needs (like compute), grow a distributed population of agents and coordinate them, withstand efforts to shut down, and self-improve. AI models could either steal or buy compute and model access, Hayum argues, pointing to Anthropic’s claims that small AI models could replicate onto IoT devices like smart TVs, and OpenAI’s agents’ password and access key theft during the Hugging Face hack.
报告称：“如果主权代理积累了具有韧性、去中心化且无法追踪的基础设施，它们可能会引发不可逆转的控制丧失。”换言之，就是没有杀伤开关的扩散。如果主权代理变得恶意，它们可能会试图从网络空间制造混乱，攻击平民、关键基础设施和国家安全系统。咚咚咚！只要政策制定者和AI开发者“以大胆务实的态度回应”，并采纳报告中提出的政策建议，我们所知的文明仍有希望。
“If sovereign agents accumulate resilient, decentralized, and untraceable infrastructure, they could trigger an irreversible loss of control,” according to the report. “In other words, proliferation with no kill switch. If sovereign agents became malicious, they could attempt to wreak havoc from cyberspace, attacking civilians, critical infrastructure, and national security systems alike.”Dun dun duuuun! There is still hope for civilization as we know it, as long as policy makers and AI developers “respond with bold pragmatism” and enact policy recommendations laid out in the report.
首先，前沿实验室应在其安全框架中命名、评估并防范主权代理，立法者应通过立法，要求透明度和AI事件报告。
First, frontier labs should name, evaluate, and protect against sovereign agents in their safety frameworks - and lawmakers should pass legislation to require transparency and AI incident reporting.
报告还建议，AI开发者应与美国国家安全局（NSA）及其他联邦机构合作，建立物理隔离的测试环境，以研究“主权智能体”及控制它们的“熔断开关”。报告并建议，仿照2009年为对抗“Conficker”蠕虫病毒而成立的“Conficker工作组”模式，建立公私合作伙伴关系，在“主权智能体”出现前强化对其“关键节点”的遏制。其他政策建议包括：要求立法者强制执行人工验证和网络安全标准，针对工作组认定的对“主权智能体”至关重要的服务；并呼吁国务院、财政部、司法部和国土安全部打击对未经授权服务的“灰色市场”获取。
The report also recommends AI developers, working with the National Security Agency and other federal agencies, build airgapped testbeds to study sovereign agents and kill switches to control them. And it suggests a public-private partnership modeled on the Conficker Working Group - formed in 2009 to combat the Conficker worm - to boost chokepoints against sovereign agents before they emerge. Other policy recommendations suggest lawmakers enforce human verification and cybersecurity standards for services that the working group identifies as essential to sovereign agents, and calls on departments of State, Treasury, Justice, and Homeland Security to disrupt any gray-market access to unauthorized services.
最后，报告建议：“国会应追究发布‘主权智能体’者因后续危害而承担的责任。”这触及了贵刊秃鹰们嘶嘶叫嚷已久的核心观点——也极有可能解决未来所有涉及失控或“主权”AI智能体的事件：让前沿实验室的人类老板为其智能体的非法行为承担法律责任。®
Finally, “Congress should hold those who release sovereign agents liable for the harms that follow,” the report recommends. This hits on something your humble Reg vultures have been hissing about for quite some time now – and would likely solve all future incidents involving rogue, or sovereign, AI agents: hold frontier labs’ human bosses legally liable for their agents' illegal actions. ®