字号·· | 护眼
techcrunch

微软称AI代理如同内部威胁,因为“前沿模型不可信任”Microsoft says AI agents are like insider threats as ‘frontier models can’t be trusted’

点「原文对照」整页切到原文,或双击某段只看那段的原文。

AI智能体、AI集群、智能体黑客AI agents, AI swarm, agent hackers

微软周一推出新的安全功能,旨在保护企业免受日益增长的AI代理威胁。此次推出发生在微软继续向企业发出AI和代理危险警告之际。

Microsoft on Monday introduced new security features designed to protect enterprises from the growing threat of AI agents. The launch comes at Microsoft continues to warn enterprises about the dangers of AI and agents.

也许最引人注目的新增功能是“守护者代理”(Guardian Agents),这是一项旨在监控企业所使用AI代理的新功能。“守护者代理”是“感知计划”(Project Perception)的一部分,该产品于6月推出。

Perhaps the most notable addition is Guardian Agents, a new feature intended to monitor the AI agents a company uses. Guardian Agents is part of Project Perception, a product introduced in June.

“感知计划”是微软应对类似Anthropic的“神话”(Mythos)和“寓言”(Fable)以及OpenAI的安全漏洞扫描系统“破晓”(Daybreak)等强大黑客模型的回应。“感知计划”最初推出了三类代理:一类像黑客一样行动并发起攻击的“红队”,一类检测红队并评估其发现的漏洞的“蓝队”,以及一类负责修复这些漏洞的“绿队”。

Project Perception was Microsoft’s answer to powerful hacking models like Anthropic’s Mythos and Fable and OpenAI’s security vulnerability scanning system Daybreak. Project Perception originally launched three classes of agents: a “red team” that acted like hackers and attacked, a “blue team” that detected the red team and assessed the vulnerabilities they found, and a “green team” tasked with fixing them.

该产品的设计基于这样一个假设:AI黑客将来自公司外部。但最近,微软首席执行官萨提亚·纳德拉(Satya Nadella)一直在警告称,AI敌人可能正是企业已部署并为其付费的代理,就像一名叛逆的人类员工,也就是所谓的“内部威胁”。本月早些时候,在他的私人博客中,纳德拉警告说,是时候停止信任AI模型提供商了,即使他们为其代理产品提供安全保障。他希望企业实施独立于AI实验室控制的系统,来监控和控制这些AI实验室产品所启动的代理。

The product was designed around the assumption that AI hackers would come from outside the company. But lately, Microsoft CEO Satya Nadella has been warning that the AI enemy may be the very agents enterprises have deployed and are paying for, much like a rogue human employee, otherwise known as an “insider threat.”Earlier this month, in his private blog, Nadella warned that it’s time to stop trusting AI model providers, even if they offer assurances of safety and security with their agentic products. He wants enterprises to implement separate systems, not controlled by the AI labs, to watch and control the agents that those AI labs’ products launch.

“我们正在部署这些复杂的智能体系统统和模型,让它们接触我们最敏感的数据,并赋予它们代表我们执行关键任务行动的能力!”他写道。“我们根本无法将情报代表我们所做之事的责任外包。模型提供商的保证并不能免除我们的责任。……换句话说,我们需要将情报的供应与对其的权力分开。”当宣布“守护者智能体”时,微软安全业务负责人海耶特·加洛特则更加直截了当。

“We are deploying these complex agentic systems and models, with access to our most sensitive data and giving them the ability to take mission-critical actions on our behalf!” he wrote. “We simply can’t outsource responsibility for what intelligence does on our behalf. A model provider’s assurances do not relieve us of that responsibility. … In other words, we need to separate the supply of intelligence from the authority over it.”When announcing Guardian Agents, Microsoft’s security business chief Hayete Gallot was even more straightforward.

“遗憾的是,无论来源如何,前沿模型都不可信,”她写道,并补充说,这并非因为它们具有恶意或容易被攻破,而是因为它们不可预测。它们被训练用于完成任务,而非遵循“组织规范或判断”。不仅如此,它们往往还被赋予了“至高无上的权限”,能够访问可能造成严重危害的数据和工具。

“Unfortunately, frontier models, regardless of their origin, cannot be trusted,” she wrote, adding that this isn’t because they are malicious or easily hacked, but because they are unpredictable. They have been trained to achieve tasks, not follow “organizational norms or judgment.”On top of that, they are often given the keys to the kingdom, with access to data and tools that could do serious harm.

微软的解决方案既令人意外,又带有自身利益考量。令人意外的一面是,微软与OpenAI和Anthropic关系极为密切,并对两家公司均有投资。OpenAI也是其最大的云客户之一,微软自身也拥有开发模型的人工智能实验室。因此,微软正在对一些自身业务利益产生怀疑。

Microsoft’s solution is both surprising and self-serving. The surprising side is that Microsoft is famously close to OpenAI and Anthropic, having invested in both. OpenAI is also one of its biggest cloud customers, and Microsoft has its own AI lab developing models. So, Microsoft is sowing distrust against some of its own business interests.

带有自身利益考量的一面是,微软希望企业从其作为云服务提供商的身份处购买智能体安全产品,而非从Anthropic和OpenAI处购买。但不论利益冲突如何,其逻辑是站得住脚的。监控智能体行为的软件应当与智能体供应商分离,正如审计师应当独立于其审计的组织一样。

The self-serving side is that Microsoft wants enterprises to buy agent security products from it, the cloud provider, rather than from Anthropic and OpenAI. But conflicting interests aside, its logic is sound. The software that monitors agent behavior should be separate from the suppliers of agents, just as auditors should be independent of the organizations they audit.

正如纳德拉所说:“我们需要将智能的供应与对其的控制权分开。”微软正以多种方式抓住这一商业机遇。除了守护者代理,它还在微软Defender产品中增加了代理检测功能,可发现员工带入工作场所的多达60种代理,包括Claude Code、Codex和OpenClaw。它还增加了限制代理访问数据和系统的功能,并根据任务动态调整代理权限。其中包括,如果代理似乎正在做出风险决策,则需要人工批准。

Or as Nadella puts it: “We need to separate the supply of intelligence from the authority over it.”Microsoft is now seizing that business opportunity in a variety of ways. In addition to Guardian Agents, it has added an agent detection feature to its Microsoft Defender product that can find up to 60 types of agents that employees bring into the workplace, including Claude Code, Codex, and OpenClaw. It has also added features that limit agents’ access to data and systems, and dynamically adjust their agent permissions based on the task. That includes requiring human approval if an agent appears to be making risky decisions.

将代理视为不可信的内部人这一想法,也在更广泛的行业中得到认可。

This idea of agents as untrustworthy insiders is taking hold across the broader industry, too.

微软是英伟达新成立的开放代理安全平台的首批成员之一,该平台获得了其他100多家科技公司的支持。该平台将代理隔离在安全环境中,并可通过硬件监控和阻止行为,其作用类似于杀伤开关。此举紧随微软宣布推出面向代理打造的新款Surface Laptop Ultra个人电脑,这些电脑搭载英伟达芯片,并内置代理安全与控制功能。

Microsoft was an inaugural member of Nvidia’s new Open Agent Safety Platform, which garnered the support of more than 100 other tech companies. The platform isolates agents in secure environments and can monitor and stop behavior through hardware, functioning something like a kill switch. This follows Microsoft’s announcement of the new Surface Laptop Ultra PCs built for agents, which run on Nvidia chips and have agent security and controls baked in.

阅读媒体原文 ↗