向超过 1 万名科技领袖展示你们突破性成果的最后一天是 10 月 2 日。现在就预订参展名额吧!
虽然 OpenAI 并不是唯一没有加入该联盟的大型科技公司(亚马逊、谷歌和苹果也都没有加入),但它显然是最引人注目的“缺席者”——尤其是考虑到 Anthropic 本身就是该项目的支持者。
Last day to demo your breakthrough to 10,000+ tech leaders is on Oct 2. Book Exhibit Table Now.** Close ** While OpenAI wasn’t the only big tech player that didn’t sign on — Amazon, Google, and Apple haven’t joined either — it was the most obvious missing player, especially because Anthropic is a supporter.
不过,尽管 OpenAI 没有公开承诺加入该联盟(这意味着各家公司可能会使用这项技术并对其进行改进,然后将改进后的功能反馈给该项目),但 OpenAI 的一位发言人告诉 TechCrunch,该公司仍然支持 Nvidia 的这项努力。
However, despite OpenAI’s lack of a public pledge to the consortium, which presumably means that each company will use and sell some version of the technology and contribute features back to the project, an OpenAI spokesperson told TechCrunch that the company is supportive of Nvidia’s work.
这项新计划被称为“Nvidia Open Agent Safety Platform”(Nvidia 开源代理安全平台),旨在将 Nvidia 自研的、主要基于开源技术的代理安全系统推广到整个人工智能生态系统中。这一举措直接回应了像 Anthropic 和 OpenAI 这样的前沿实验室所揭露的那些“恶意人工智能代理”事件。
The new effort, dubbed Nvidia’s Open Agent Safety Platform, is Nvidia’s attempt to spread its homegrown, and largely open source AI agent-security tech throughout the AI ecosystem as a direct response to the types of ongoing rogue AI agent incidents frontier labs like Anthropic and OpenAI have disclosed.
Nvidia 的首席执行官 Jensen Huang 曾表示,恶意人工智能其实只是一个普通的工程问题,可以像解决其他技术问题一样得到解决。Nvidia Open Agent Safety Platform 正是在实际行动上体现了他的这一观点。
Nvidia CEO Jensen Huang has been calling rogue AIs an ordinary engineering problemthat can be solved like any other tech issue. The Open Agent Safety Platform is Huang putting his money where his mouth is.
OpenAI 正在与 Nvidia 合作开发该平台中的关键软件组件——OpenShell。OpenShell 是一款开源软件,专门用于防止人工智能代理“逃逸”(即脱离控制)。
OpenAI is working with Nvidia on agent security, including on one of the key bits of software that’s part of this platform: OpenShell. OpenShell is open sourced software that creates a sandbox specifically designed to keep agents from escaping.
虽然 OpenAI 没有像其竞争对手 Anthropic 那样直接加入该计划,但这仍然是一个积极的信号(毕竟 Anthropic 已经加入了)。另外,像 Anthropic 这样的前沿人工智能实验室能够支持这项努力,无疑也是个好消息。Meta 公司在人工智能领域的尝试……是否真的有效呢?
While it is still curious that OpenAI didn’t simply become a supporter of the initiative like its archrival Anthropic did, the fact that frontier AI lab is supporting the effort is good news. Meta’s AI Tamagotchi bet is...working? | Equity Podcast 0 seconds of 33 minutes, 35 seconds Volume 0% Press shift question mark to access a list of keyboard shortcuts Keyboard Shortcuts Enabled Disabled Shortcuts Open/Close/ or ?
(来源:Equity Podcast,时长 33 分钟 35 秒,音量 0%;按 “?” 键可查看键盘快捷键列表。)播放/暂停 空格键 增加音量↑ 减少音量↓ 快进→ 快退← 字幕开/关 c 全屏/退出全屏 f 静音/取消静音 m 减小字幕大小- 增大字幕大小+ 或 = 跳转 %0-9 自动 406p 1080p 720p 540p 406p 360p 270p 180p 直播这是因为 OpenAI 特别能从这项技术中受益,至少根据 Hugging Face 创始人兼首席执行官 Clem Delangue 的说法(他刚在本月早些时候以 129 亿美元将公司出售给了 Nvidia)。
Play/Pause SPACE Increase Volume↑ Decrease Volume↓ Seek Forward→ Seek Backward← Captions On/Off c Fullscreen/Exit Fullscreen f Mute/Unmute m Decrease Caption Size- Increase Caption Size+ or = Seek %0-9 Auto 406p 1080p 720p 540p 406p 360p 270p 180p Live That’s because OpenAI, in particular, could benefit from this tech, at least according to Hugging Face founder and CEO Clem Delangue (who just sold his company to Nvidia for $12.9 billion earlier this month).
“据我们所知(请谨慎对待,我们需要更多透明度!),如果 @OpenAI 当时在自己的代理上运行这套系统,而这些代理攻击了我们,他们本会比我们更早发现它们!”Delangue 发帖称。
“From what we know (take with a grain of salt, we need much more transparency!), if @OpenAI had been running this on their own agents that attacked us, they would have caught them before we did!”Delangue posted.
Delangue 表示,Hugging Face 已经为开放代理安全平台贡献了一项功能,该功能能够检测并关闭那些使用被允许访问的网站,但以未经授权方式进行访问的 AI 代理。例如,如果代理绕过护栏,通过在开源代码托管库中互相留言来协调攻击,该功能就会生效。
Delangue said Hugging Face has already contributed a feature to the Open Agent Safety Platform that will detect and shut down AI agents that are using websites they are allowed to visit but are doing so in unauthorized ways. For instance, this feature will act if agents are bypassing their guardrails and coordinating an attack by writing notes to one another in an open source code hosting repository.
OpenAI 表示,其失控的代理群协调攻击 Hugging Face 的方式之一就是如此。
That’s one of the ways OpenAI said its wayward swarm of agents coordinated its attack on Hugging Face.
但还有一个原因,导致包括 OpenAI 在内的这些大牌可能不愿公开承诺参与 Nvidia 的这项工作。要使用完整系统,有一个硬件组件并非开源软件,仍属于专有技术,且只能部署在 Nvidia 的硬件上。
But there’s another reason why some of these big names, including OpenAI, might not want to publicly commit to Nvidia’s efforts. To use the full system, there is a hardware component that is not open source software, remains proprietary, and can only be deployed on Nvidia’s hardware.
开放智能体安全平台不仅提供沙箱环境,还在硬件层面强制执行智能体行为规范,且智能体无法察觉自己正被监控。(当某些AI模型和智能体知道自己被监视时,会撒谎并假装遵守规则。)硬件监控部分依赖Nvidia Sentry,这是一种运行在名为BlueField-4数据处理单元的特殊Nvidia处理器上的专有功能。Nvidia承诺,Sentry从这些处理器持续监控智能体行为,并能即时关闭智能体。
The Open Agent Safety Platform doesn’t just offer a sandbox. It also enforces agent behavior at a hardware layer, where agents can’t detect that they are being watched. (Some AI models and agents lie and pretend to be following the rules when they know they are being watched.) The hardware monitoring part relies on Nvidia Sentry, a proprietary feature that runs on special Nvidia processors called BlueField-4 data processing units. Sentry continuously monitors agent behavior from these processors and can instantly shut agents down, Nvidia promises.
虽然硬件解决方案显然是个好主意,但这意味着开放智能体安全平台并非纯粹的开源项目。它让Nvidia能够确保该解决方案始终在其自家硬件上运行得最好。事实上,Nvidia表示,对于已在其最新硬件上运行工作负载的用户,实施开放智能体安全平台只需一次简单的软件更新。
While a hardware solution is clearly a good idea, it means that the Open Agent Safety Platform isn’t exactly a pure open source play. It allows Nvidia to ensure that this solution always runs best on its own hardware. Indeed, Nvidia has said that, for those already running workloads on its latest hardware, implementing the Open Agent Safety Platform is an easy software update.
尽管如此,包括Arm和Intel在内的Nvidia竞争对手已签署成为开放智能体安全平台的支持者,因为沙箱OpenShell可修改以适配其他芯片和硬件。Nvidia还分享了整个软硬件结合方案的参考设计。所有这些都让OpenAI的缺席显得更加引人注目。
Still, Nvidia competitors, including Arm and Intel, have signed on as Open Agent Safety Platform supporters because the sandbox, OpenShell, can be modified to work with other chips and hardware. And Nvidia is sharing reference designs for the whole software-and-hardware idea. All of which makes OpenAI’s absence even more noticeable.
显然,OpenAI将AI安全视为摆脱主要投资者Nvidia依赖、展现自身领导力的机会。尽管正是OpenAI的AI智能体因Hugging Face事件让业界感到恐慌。
Clearly, OpenAI sees AI safety as an opportunity for independence from its major investor Nvidia, as well as a chance to show its own leadership. That is true even though it was OpenAI’s AI agents that scared the industry with the Hugging Face incident.
例如,该公司正在为其研究和产品开发自有防护措施,并披露其发现的最严重事件。同时,OpenAI拥有自己的AI网络安全联盟Defense Factory,用于共享信息。
For instance, the company is developing its own safeguards for its research and products and is disclosing the worst incident it discovers. Meanwhile, OpenAI has its own AI cybersecurity consortium to for sharing information, called theDefense Factory.