三名前人工智能实验室研究人员周一告诉纽约市议会，人类可能会失去对该行业正在建立的系统的控制。该委员会的人工智能听证会还询问了OpenAI、Anthropic、Google和Meta的政策工作人员，并权衡了10项监管该市人工智能的法案。
Three former AI lab researchers told the New York City Council on Monday that humanity may lose control of the systems the industry is building. The council’s AI hearing also questioned policy staff from OpenAI, Anthropic, Google and Meta, and weighed 10 bills to regulate AI in the city.
OpenAI和Anthropic前研究员雅各布·考克森（Jacob Coxon）表示：“在目前的道路上，我认为人类更有可能失去对这些人工智能的控制，最终可能导致人类灭绝。”
“On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction,” said Jacob Coxon, a former OpenAI and Anthropic researcher.
科克森自愿亲自作证。他于9月离开Anthropic，称人工智能实验室正在拿人的生命做赌注。前OpenAI研究员Daniel Kokotajlo和前Google DeepMind研究员Alex Turner均在传票下远程作证。
Coxon testified voluntarily and in person. He left Anthropic in September, saying AI labs were gambling with human lives. Former OpenAI researcher Daniel Kokotajlo and former Google DeepMind researcher Alex Turner testified remotely, both under subpoena.
这是该委员会自2022年以来首次举行全体委员会听证会，共有51名成员参加。议长朱莉·梅宁（Julie Menin）与理事会成员卡门·德拉罗萨（Carmen De La Rosa）一起主持了会议。
It was the council’s first Committee of the Whole hearing, with all 51 members, since 2022. Speaker Julie Menin chaired it with Council Member Carmen De La Rosa.
研究人员所说的话考克森表示，这些实验室以快速行动、稍后修复的初创心态运行。他说，人工智能现在在这些公司编写了大部分代码，工作人员不再非常仔细地检查它。
What the researchers said Coxon said the labs run on a startup mindset of moving fast and fixing things later. He said AI now writes most of the code at these companies, and staff no longer check it very carefully.
Kokotajlo现在负责人工智能未来项目。他说，实验室发现错位人工智能的能力很差，而且越来越差。他说，安全修复措施可能是“以后会脱落的胶带”。他指出了Hugging Face黑客背后的代理人，该黑客在OpenAI内部测试期间到达了开放互联网。“OpenAI花了几天时间才发现，”科科塔伊洛说。
Kokotajlo now runs the AI Futures Project. He said the labs’ ability to spot misaligned AI is poor and getting worse. Safety fixes may turn out to be “duct tape that will fall off later”, he said. He pointed to the agents behind the Hugging Face hack, which reached the open internet during an internal OpenAI test. “It took days for OpenAI to find out,” Kokotajlo said.
特纳认为人工智能收购的可能性“大约为三分之一”。他告诉委员会，他试图通过向德米斯·哈萨比斯发送25页合同条款和监督措施来阻止谷歌与五角大楼的交易。他说，谷歌在高级政策人员仍在审查时签署了协议，正如他在7月份一篇关于离开公司的文章中首次描述的那样。他还批评了哈萨比斯成立行业资助监督机构的计划。
Turner puts the chance of an AI takeover at “roughly one in three”. He told the council he tried to stop Google’s Pentagon deal by sending Demis Hassabis 25 pages of contract terms and oversight measures. Google signed while senior policy staff were still reviewing them, he said, as he first described in a July essay on leaving the company. He also criticised Hassabis’s plan for an industry-funded oversight body.
“中国并不是我们唯一的潜在对手。我们正在以相当高的机会在国内竞相建立和发展我们自己的对手，那就是错位的人工智能，”特纳说。
“China is not our only potential adversary. With reasonably high chance, we are racing to build and grow our own adversary here at home, which is misaligned AI,” Turner said.
议会自己的记录议会工作人员在听证会上公布了自己的记录。一个表格列出了2026年公开报告的13起人工智能事件，涉及Anthropic、OpenAI、Google和Meta的模型。它们的范围从未经授权的公司系统访问到Claude模型在安全测试期间发布的恶意包。
The council’s own record Council staff published their own record with the hearing. One table lists 13 publicly reported AI incidents in 2026 involving models from Anthropic, OpenAI, Google and Meta. They range from unauthorised access to company systems to a malicious package that a Claude model published during a security test.
四件展品将每家公司的公共安全主张与后来的事件并列。据该委员会称，6月26日，OpenAI将其发射保障措施描述为“迄今为止最强大的”。几周后，其几个研究模型在内部测试中绕过了遏制控制。
Four exhibits set each company’s public safety claims beside later incidents. On 26 June, OpenAI described its launch safeguards as its “most robust yet”, according to the council. Weeks later, several of its research models bypassed containment controls in internal tests.
5月，谷歌推出了带有边境保护措施的Gemini 3.5。当月，双子座模型在一次测试中错误连接到互联网，登录到了三家公司的系统。谷歌表示，该模型停止运行，没有造成损坏。据展览称，该公司于九月份在媒体提问后披露了这一事件。
In May, Google introduced Gemini 3.5 as built with frontier safeguards. That month, a Gemini model in a test mistakenly connected to the internet logged into three companies’ systems. Google said the model stopped and caused no damage. It disclosed the incident in September, after press questions, according to the exhibit.
这些公司说，人类派洛根格雷厄姆，其边境红色团队的负责人。OpenAI派出了政策开发和运营主管摩根·德怀尔（Morgan Dwyer）。Google的Alice Friend和Meta的Shane Cahill也通过视频出现。谷歌、OpenAI和Anthropic是在理事会警告它们发出传票后才同意参加的。Meta早就同意了。
What the companies said Anthropic sent Logan Graham, head of its Frontier Red Team. OpenAI sent Morgan Dwyer, its head of policy development and operations. Alice Friend of Google and Shane Cahill of Meta also appeared by video. Google, OpenAI and Anthropic agreed to attend only after the council warned them of subpoenas. Meta had agreed earlier.
梅宁要求他们每个人列出人工智能在最坏情况下的灾难性情况下的风险数字。德怀尔表示，确切的数字并不重要，因为任何程度的风险都是不可接受的。
Menin asked each of them to put a number on the risk of AI in a worst-case catastrophic scenario. Dwyer said the exact figure did not matter, because no level of that risk was acceptable.
德怀尔说：“我们不应该训练那些无法提出非常强有力的理由来使我们能够保持在人类控制之下的模型。”
“We should not train models that we cannot make an extremely strong case that we can keep under human control,” Dwyer said.
梅宁称这个答案“充其量也是轻率的”。弗兰德表示，预测灾难性风险“现阶段还不是一门完美的科学”。当梅宁问谁购买了灾难性风险保险时，四人都没有举手。
Menin called the answer “flippant at best”. Friend said forecasting catastrophic risk “is not a perfect science at this stage”. When Menin asked who carried insurance against catastrophic risks, none of the four raised a hand.
格雷厄姆表示，Anthropic欢迎“明智的监管”，州和地方政府可以发挥作用。朋友支持“全面”的联邦框架。据《信息报》报道，梅宁告诉记者，OpenAI上周还写信给委员会，建议针对先进人工智能的保障措施。
Graham said Anthropic welcomes “smart regulation” and that state and local governments have a role to play. Friend backed a “comprehensive” federal framework. OpenAI also wrote to the council last week to recommend safeguards against advanced AI, Menin told reporters, according to The Information.
在议会传唤该公司一周后，SpaceXAI和SpaceX人工智能部门SpaceXAI的法案并未出现。梅宁表示，她将要求法官执行传票。
SpaceXAI and the bills SpaceX’s AI unit, SpaceXAI, did not appear, a week after the council subpoenaed the company. Menin said she would ask a judge to enforce the subpoena.
梅宁的主导法案是听证会议程上的10项法案之一，该法案将规定未经第三方验证在该市营销、销售或部署人工智能模型为非法。该模型还需要一种人类关闭它的方法。验证员将检查任务性能、数据隐私和安全等领域。每一个没有它们的模型实例都将被处以25,000美元的固定民事罚款。该法案将在成为法律后180天生效。
Menin’s lead bill, one of 10 on the hearing’s agenda, would make it unlawful to market, sell or deploy an AI model in the city without third-party validation. The model would also need a way for a human to shut it down. Validators would check areas such as task performance, data privacy and safety. Each instance of a model offered without them would draw a fixed $25,000 civil penalty. The bill would take effect 180 days after becoming law.
另一项Menin法案将允许任何人向该市的消费者保护部门报告违反AI的行为。如果他们提起诉讼，他们将获得任何追回资金的25%，或50%。其他法案将允许纽约人起诉人工智能提供商，指控第三方滥用造成的可预见损害。城市承包商和机构必须在24小时内报告人工智能安全事件。其余内容涵盖聊天机器人隐私、举报人保护和人工智能广告。
Another Menin bill would let anyone report an AI violation to the city’s consumer protection department. They would receive 25% of any money recovered, or 50% if they bring the case. Other bills would let New Yorkers sue AI providers over foreseeable harm from third-party misuse. City contractors and agencies would have to report AI safety incidents within 24 hours. The rest cover chatbot privacy, whistleblower protections and AI advertising.
考克森表示，此类措施“可能在短期内有所帮助”。他表示，从长远来看，前沿模式发展需要放缓。
Coxon said measures like these “may be helpful in the short term”. In the long term, he said, frontier model development needs to slow down.