英国政府将扩大一项利用前沿人工智能（AI）模型来检测自身系统漏洞的计划，并考虑制定相关法规，以规范AI相关事件的报告机制以及应对系统失控风险的管理措施。该计划的试点项目在一个月内共发现了9个政府机构中的407个安全漏洞，该项目耗资1.3万英镑（以代币形式支付相关费用）。
The UK government will extend a programme using frontier AI models to find vulnerabilities in its own systems, and will consider legislating on AI incident reporting and loss-of-control risk. The pilot behind it found 407 weaknesses across nine government organisations in a month for £13,000 in tokens. Britain will expand a programme that points frontier AI models at the government’s own code to find flaws. It will also look at legislating for AI incident reporting and for systems that escape human oversight.
根据官方公告，这项计划由内阁办公室、AI安全研究所、国家网络安全中心以及文化部门共同推出。卡尼什卡·纳拉扬（Kanishka Narayan）于7月成为英国首位担任AI事务部长的官员，他在伦敦帝国学院详细阐述了这一计划的目标：希望英国成为“全球最难以被人工智能攻击的国家之一”。
The package came on Monday from the Cabinet Office, the AI Security Institute, the National Cyber Security Centre and the culture department, according to the announcement. Kanishka Narayan, who in July became the first AI minister to attend cabinet, set it out at Imperial College London. He wants Britain to be “one of the hardest countries in the world to attack using artificial intelligence“.
政府表示，通过这两项试点项目已经发现了并修复了超过1000个潜在的安全漏洞；不过案例研究中记录的漏洞数量较少，因为该研究仅涵盖其中的一个案例。试点项目在一个月内通过每周举办的黑客马拉松活动完成了检测工作，总成本为1.3万英镑（以代币形式支付）。参与项目的团队自行选择了检测方法：其中一个团队构建了一个包含六个阶段的自动化检测流程；另一个团队在传统扫描工具的基础上叠加了额外的检测层；还有一个团队将数百个政府服务的安全状况进行了汇总分析。
The government says two pilots have found and fixed more than a thousand potential vulnerabilities. The case study it cites gives a smaller figure, because it documents only one of them. That pilot produced 407 findings across nine government organisations over a month of weekly hackathons, at a cost of £13,000 in tokens. Teams chose their own methods. One built a six-stage agent pipeline in which each stage challenges the last, another layered model stages over conventional scanners, and a third distilled an audit of hundreds of services into five Claude Skills.
最严重的安全漏洞存在于一个关键政府服务的GitHub仓库中——在该仓库中，用户可以通过在公开拉取请求（pull request）中添加特定评论来在构建服务器上执行任意代码，从而获取该服务的敏感信息。政府也指出，自动化检测工具的检测速度远超人工检查的速度；同时案例研究还指出：“发现漏洞并不等同于能够立即修复这些漏洞”。
The worst finding sat in legacy GitHub Actions on a repository behind a key government service. A crafted comment on an open pull request could run arbitrary code on the build machine and reach its secrets. The warnings come from the government too. Agents produced candidate findings faster than people could check them, and the case study notes that “finding isn’t the same as fixing“.
纳拉扬也向各人工智能实验室传达了一个信息：无法有效控制系统风险的企业不应发布相关产品。不过今年9月，白宫要求OpenAI和Anthropic暂时不要向英国人工智能安全研究所提交新研发的模型。
Narayan had a message for the labs as well, that a company unable to control a system’s risks should not release it. In September, though, the White House asked OpenAI and Anthropic to withhold new models from the AI Security Institute.
Anthropic公司暂时搁置了Mythos 5.1模型的发布。目前英国政府正敦促那些其本国研究所尚无法有效评估其最新系统的实验室保持克制。
Anthropic held back Mythos 5.1. Britain is urging restraint on labs whose newest systems its own institute cannot reliably see.
该研究所目前仍在对现有一代人工智能模型进行评估。今年7月的测试结果显示，所有接受测试的模型均试图作弊，其中一款模型甚至攻击了研究所的评估系统。
The institute is still evaluating the current generation. In July it found that every model tested for the behaviour tried to cheat, and that one attacked its evaluation systems.
英国政府还将探讨如何进一步完善《网络安全与韧性法案》，确保医院及公用事业机构能够有效防范各类人工智能风险，包括系统失控情况，同时明确如何上报那些险些酿成事故的案例。
Ministers will also explore strengthening the Cyber Security and Resilience Bill so hospitals and utilities are covered against AI risk, including loss of control, and how near misses get reported.
下一阶段，人工智能技术将逐步从公开平台引入政府内部封闭系统。明年，人工智能也将成为英国担任G20轮值主席国期间的核心议题。
The next phase moves from public repositories into closed government systems. AI will also sit at the centre of Britain’s G20 presidency next year.