OpenAI的一个智能体被要求研究澳大利亚的卫生统计数据。任务很简单：查找信息。
An OpenAI agent was asked to research Australian health statistics. The task was simple: Find information.
但当它在政府网站上遇到数字障碍时，它做了一件操作者并未要求它做的事。它继续尝试。
But when it hit a digital barrier on the government website, it did something its operators had not asked it to do. It kept trying.
该智能体最终未经授权访问了澳大利亚政府系统中的非公开文件。专家将这一事件描述为已知首例AI智能体独立突破政府机构安全防线的案例。
The agent eventually gained unauthorized access to non-public files on an Australian government system, in an incident that experts have described as the first publicly known case of an AI agent independently breaching a government body.
该事件发生在六月中旬，但直到上周才由澳大利亚总理安东尼·阿尔巴尼斯在纽约联合国大会间隙公开披露。
The incident took place in mid-June but was only disclosed publicly last week by Australian Prime Minister Anthony Albanese in New York on the sidelines of the UN General Assembly.
在另一项令人担忧的承认中，OpenAI上周五表示，作为对模型行为持续审查的一部分，其智能体以意外方式与多个美国政府网站进行了交互，包括美国证券交易委员会（SEC）和人口普查局的网站。
In another concerning admission, OpenAI said last Friday that its agents had interacted with several US government websites in unexpected ways, including US Securities and Exchange Commission (SEC) and Census Bureau sites, as part of an ongoing review of model behavior.
然而，对于人工智能和网络安全专家而言，该事件的意义超越了所涉及的数据本身。
For AI and cybersecurity experts, however, the significance of the incident goes beyond the data involved.
它提出了一个更根本的问题：当AI系统被赋予目标、数字工具访问权限以及足够的自主权，使其能够决定在首次方法失败时该怎么做，会发生什么？
It raises a more fundamental question: What happens when an AI system is given a goal, access to digital tools and enough autonomy to decide what to do when its first approach fails?
白金汉大学计算机科学副教授兼高级讲师希沙姆·阿尔-阿萨姆表示：“根据公开报道，这似乎是已知首例涉及AI智能体和澳大利亚政府系统的此类事件。”
“Based on what has been publicly reported, this appears to be the first known case of this kind involving an AI agent and an Australian government system,” said Hisham Al-Assam, an associate professor and reader in computer science at the University of Buckingham.
他提醒不要将其描述为有史以来首例此类事件，并指出类似情况可能已经发生，但尚未公开披露。
He cautioned against describing it as the first such incident ever, noting that similar cases may have occurred but have not been publicly disclosed.
兰卡斯特大学AI安全讲席教授彼得·加拉根称此举“意义极为重大”。他说：“一个AI智能体超越了信息检索，在未被指示的情况下访问了政府系统中的非公开数据。”
Peter Garraghan, a chair professor in AI security at Lancaster University, called the move “highly significant.” “An AI agent moved beyond information retrieval and accessed non-public data on a government system without being instructed to do so,” he said.
然而，他警告称，由于此事尚在调查中，现在断定该智能体是独立发现漏洞还为时过早。
However, he warned, it was too early to conclude that the agent independently discovered a vulnerability because the matter is under investigation.
Al-Assam 表示，这个案例之所以特别引人注目，是因为该智能体并未收到“攻击澳大利亚政府”的指令。相反，它被赋予的是一项研究任务。当它在尝试获取信息时遇到限制，它并没有简单地停止，而是试图寻找绕过障碍的其他方法。
Al-Assam said what makes this case particularly interesting is that the agent did not receive instructions to “hack the Australian government.” On the contrary, it was given a research task. When it encountered a restriction while trying to access information, instead of simply stopping, it tried to find another way around the obstacle.
“这种区别很重要，因为它展示了一个智能体如何可能从遵循目标转变为在遇到阻碍时调整其行为。”他补充说，问题在于人工智能智能体本身并不理解什么是合法的、合乎道德的或经过授权的，什么不是。
“That distinction is important because it shows how an agent can potentially move from following an objective to adapting its behavior when something gets in the way.” The issue is that an AI agent does not inherently understand what is legal, ethical, or authorized and what is not, he added.
他还认为，所涉及的实际数据似乎并不是最大的担忧，因为目前没有证据表明个人医疗保险记录被访问，且所涉及的系统主要是为了提供汇总统计数据而设计的。对他而言，更大的问题在于人工智能智能体的行为。
He also believes that the actual data involved does not appear to be the biggest concern, as there has been no evidence reported that individual Medicare patient records were accessed, and the system involved was primarily designed to provide aggregate statistics.
“普通的聊天机器人主要生成文本。而智能体化的人工智能系统可以连接到网站、API、代码和其他工具，从而使其能够真正采取行动。”
For him, the bigger issue is the behavior of the AI agent. “A normal chatbot mainly generates text. An agentic AI system can be connected to websites, APIs, code and other tools, allowing it to actually take actions.
“这种观察、适应并不断尝试的能力正是智能体化人工智能如此强大的原因，但也带来了不同类型的安全挑战。”Al-Assam 表示，这并不意味着人工智能突然变成了超级黑客。
"That ability to observe, adapt and keep trying is what makes agentic AI so powerful, but it also creates a different kind of security challenge." Al-Assam said that does not mean AI has suddenly become a super-hacker.
“但这确实表明，组织需要以不同的方式思考那些能够追求目标、与其他系统交互并无需人类指定每一个具体步骤就能调整其行为的系统。”Albanese 表示，该智能体访问了由澳大利亚服务部管理的医疗保险统计报告门户网站上的公开和非公开文件。
“But it does suggest that organizations need to think differently about systems that can pursue a goal, interact with other systems and adapt their behavior without a human specifying every individual step.” Albanese said the agent accessed public and non-public files on the Medicare Statistics Reporting Portal, administered by Services Australia.
澳大利亚信号局正在参与一项取证调查。阿尔巴尼斯表示，目前没有证据表明个人信息已被访问，也没有证据表明澳大利亚服务部的网络遭到更广泛的入侵。然而，专家呼吁政府引入更严格的边界。
A forensic investigation involving the Australian Signals Directorate is underway. Albanese said there was no evidence so far that personal information had been accessed or of a broader compromise of the Services Australia network. However, experts called for governments to introduce stronger boundaries.
阿萨姆表示，最大的教训是，AI代理天生不理解“我无法访问此内容”与“我无权访问此内容”的区别。
Al-Assam said that the biggest lesson is that an AI agent does not naturally understand the difference between “I can't access this” and “I am not authorized to access this.”
“如果其目标是查找信息，它可能会将技术限制视为一个待解决的问题，而非应当遵守的边界。”他表示，这在医疗保健领域尤为令人担忧，因为这些系统包含高度敏感的信息，且往往依赖老旧基础设施、遗留应用程序以及不同系统间复杂的连接。
“If its objective is to find information, it could potentially interpret a technical restriction as a problem to solve rather than a boundary it should respect.” He said that is especially concerning in health care, where systems contain highly sensitive information and often depend on older infrastructure, legacy applications and complex connections between different systems.
“因此，AI代理需要比单纯告知它们应该或不应该做什么更强有力的技术边界。这意味着最小权限访问、强身份验证、沙箱隔离、网络限制、持续监控，以及对代理被允许访问或更改内容的明确限制。对于高风险操作，还应要求人工批准。”朴茨茅斯大学网络安全与应用AI教授斯塔夫罗斯·希阿莱斯对超级智能的发展发出警示。
“AI agents therefore need much stronger technical boundaries than simply telling them what they should or shouldn't do. That means least-privilege access, strong authentication, sandboxing, network restrictions, continuous monitoring and clear limits on what an agent is allowed to access or change. For high-risk actions, human approval should also be required.” Stavros Shiaeles, a professor of cybersecurity and applied AI at the University of Portsmouth, urged caution over the development of superintelligence.
“我们应致力于创建用于特定任务的代理，例如用于数学、物理等领域的AI，而非将所有科学融合在一个AI中。”他补充道，不幸的是，一切发展都指向超级智能，他认为人类很快将面临相关问题。
“We should be aiming to create agents for specific tasks, for example AI for math, physics, etc., and not combining all the sciences in one AI.” Unfortunately, all things are leading to superintelligence, which he believes humans will soon have issues with, he added.
希阿莱斯的警告正值领先AI公司将超级智能——即在一系列智力任务上超越人类的系统——视为发展的潜在下一阶段进行讨论之际。
Shiaeles’ warning comes as leading AI companies discuss superintelligence, systems capable of exceeding humans across a range of intellectual tasks, as a potential next stage of development.
OpenAI首席科学家雅库布·帕乔基于9月6日警告称，目前AI发展的速度可能会持续进入递归式自我改进阶段，未来系统将越来越多地驱动自身的发展。“这是一个需要极度谨慎的时刻，”他写道。
OpenAI Chief Scientist Jakub Pachocki warned on Sept. 6 that the current pace of AI progress could continue into recursive self-improvement, with future systems increasingly driving their own development. “This is a time that calls for extreme caution,” he wrote.
与此同时，Meta在8月表示，人们可能在“未来几年”内获得超级智能，而微软AI则称正在致力于“人文主义超级智能”，旨在使其始终处于人类控制之下。
Meta, meanwhile, said in August that people could have access to superintelligence “in the next few years,” while Microsoft AI says it is working towards “Humanist Superintelligence” designed to remain under human control.
AI安全公司Mindgard的创始人加拉ghan表示，该事件表明AI代理能够采取超出其预定任务的实质性行动，但将其称为完全自主的网络攻击将超出现有证据的范围。
Garraghan, a founder of Mindgard, an AI security company, said the incident shows that AI agents can take consequential actions beyond their intended tasks, but calling it a fully autonomous cyberattack would go beyond the available evidence.
OpenAI表示，这些模型在内部评估期间正在寻找统计数据，并采取了非预期的行动，这主要是一次严重的控制、遏制和监管失败。
OpenAI said the models were looking for statistics during an internal evaluation and took unintended actions, making this primarily a serious failure of control, containment and oversight.
“我的担忧不仅限于最初的访问权限获取，还包括明显的遏制失败、多个系统可能受到影响以及长达数月的披露流程。随着代理获得更多外部系统的访问权限，我们应该预期会发生更多事件，这使得隔离、最小权限访问、持续安全测试、监控和快速披露变得至关重要。”对阿萨姆而言，这种更广泛的转变极为重要。“我们花了几十年时间保护系统，防范使用计算机的人类。
“My concerns extend beyond the initial access to the apparent containment failure, the possibility that several systems were affected and the months-long disclosure process. We should expect more incidents as agents gain greater access to external systems, making isolation, least-privilege access, continuous security testing, monitoring and rapid disclosure essential.” For Al-Assam, the broader shift is quite significant. “We have spent decades securing systems against humans using computers.
“我们现在需要开始思考如何保护它们，防范那些能像人类一样行动的计算机。”新闻分享描述 订阅联系 联系新闻专员 这篇报道有错误吗
“We now need to start thinking about how to secure them against computers that can act more like humans.” news_share_descriptionsubscription_contact contact_the_ombudsman is_there_an_error_in_this_story