萨蒂亚·纳德拉表示，在人工智能与公司之间的关系中，信任问题可能是健康的。周六，微软首席执行官在一篇冗长的X帖子中表示，公司并不总是理解人工智能模型的决策或行为，因此需要建立防护墙来保护自己免受技术风险的影响。毕竟，人工智能已经有可能在探索新的连接。纳德拉说：“我们需要用强大的确定性系统设计、人工控制和可靠的操作程序来包裹非确定性模型，并在现有标准不足的地方建立行业标准。”
Satya Nadella says that in the relationship between AI and a company, trust issues can be healthy. In a lengthy X post on Saturday, the Microsoft CEO said companies don't always understand an AI model's decisions or behavior, so they need to build walls to protect themselves from the tech. After all, it's very possible that AI is already exploring new connections. "We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient," Nadella said.
他还说：“将前沿封闭和开放权重模型视为内部风险，是构建这种系统的一种方式。”纳德拉表示，人工智能模型本身并不具有恶意，但任何能够访问关键系统的事物都可能被破坏或出错。他补充说，人工智能模型不应能够操作决定其可访问内容或可执行操作的控制机制。“如今，这意味着将模型与编排其工作的控制系统分离，以及定义其可执行操作的行动空间分离。这也意味着外部化控制和防护措施。”纳德拉说。
"Treating frontier closed and open weight models like insider risks is a way to build such a system." Nadella said that AI models aren't inherently malicious, but anything with access to vital systems can become compromised or make mistakes. He added that AI models shouldn't be able to operate the controls that determine what they can access or what actions they can take. "Today this means separating the model from the harness that orchestrates its work, as well as the action space that defines what it can do. It also means externalizing controls and safeguards," Nadella said.
他还补充道：“最值得信赖的超级智能系统不会是我们最信任的模型所在的系统。而会是使我们能够对模型保持最低信任度的系统。”Box首席执行官亚伦·列维回应了纳德拉的帖子，表示人工智能需要经历一个“零信任时代”。“所有这些都导致了对各种保护层的需求，以及对代理正在做什么、它们可以处理什么数据以及出错时的控制进行审计的需求。目前，为企业构建此类系统的人们面临着巨大的机遇。”列维说。
He added, "The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least." Box CEO Aaron Levie responded to Nadella's post and said AI will need to go through a "zero trust era." "And all of this leads to needing various layers of protection and auditability of what agents are doing, what data they can work with, and controls for when things go wrong. And a huge opportunity right now for those building such systems across the enterprise," Levie said.
随着人工智能开发步伐的不断加快，包括构建人工智能模型的公司在内的所有人都在呼吁更严格的防护措施。九月，Anthropic首席执行官达里奥·阿莫代伊表示，行业需要放缓人工智能的发展。
As the pace of AI development continues to pick up, everyone is clamoring for stricter guardrails, including the companies building the AI models themselves. In September, Anthropic CEO Dario Amodei said the industry needs to slow down AI development.
他的博客文章得到了OpenAI首席执行官山姆·奥特曼和SpaceXAI首席执行官埃隆·马斯克的支持。特朗普政府一直支持快速发展人工智能，一直抵制对行业进行监管。本月，参议员乔什·霍利和克里斯·墨菲提出了一项跨党派立法，旨在让人工智能代理开发者和运营商对黑客事件承担责任。纳德拉的言论是在一系列网络安全事件冲击人工智能行业之后发表的。
His blog post drew support from OpenAI CEO Sam Altman and SpaceXAI CEO Elon Musk. The Trump administration, which has supported rapid AI development, has resisted regulating the industry. This month, Senators Josh Hawley and Chris Murphy proposed bipartisan legislation to hold AI agent developers and operators liable for hacking incidents. Nadella's remarks come after a series of cybersecurity incidents rocked the AI industry.
上个月，澳大利亚总理安东尼·阿尔巴尼塞表示，一名OpenAI代理在今年夏天入侵了一个政府网站。7月，Anthropic表示，它已经识别出三起事件，在这些事件中，Claude模型访问了互联网并入侵了未授权的系统。在X上，纳德拉分享了一份公司应遵循的建议原则清单，其中包括假设模型已经“被入侵”，并且必须从一开始就进行限制。
Last month, Australia's Prime Minister Anthony Albanese said an OpenAI agent breached a government website this summer. In July, Anthropic said it had identified three incidents in which a Claude model accessed the internet and breached unauthorized systems. On X, Nadella shared a list of suggested principles that companies should follow, including the assumption that models are "compromised" and must be contained from the start.
他说：“可以把它想象成紧急刹车。授权人员应该始终能够在任务进行中暂停或关闭模型。” “更先进的模型将需要更先进的限制技术，而我们需要对此进行标准化。”纳德拉还表示，公司需要及时向受影响方披露其人工智能系统何时出现故障或被入侵。
"Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task," he said. "More advanced models will require more advanced containment technologies that we need to standardize on." Nadella also said companies need to disclose when their AI systems fail or become compromised to the affected parties in a timely manner.