字号·· | 护眼
法国24新闻台

“一场未来主义的辩论现已非常真实”:为什么世界正艰难监管杀人机器人'A futuristic debate now very real': Why the world is struggling to regulate killer robots

点「原文对照」整页切到原文,或双击某段只看那段的原文。

来自128个国家的代表周一在日内瓦会聚一堂,决定是否推进谈判,为使用无需人类介入即可杀戮的人工智能武器制定具有法律约束力的规则。《法兰西24台》与人工智能研究员文森特·布拉尼安(Vincent Boulanin)进行了对话,以了解此次会议的关键利害所在。

Representatives from 128 nations are meeting in Geneva on Monday to decide whether to move ahead with negotiating legally binding rules around the use of AI-powered weapons that can kill without human involvement. FRANCE 24 spoke with artificial intelligence researcher Vincent Boulanin to find out just what is at stake.

“计算机永远无法承担责任,”IBM在1979年的一份内部培训手册中写道,“因此,计算机绝不能做出管理决策。”自那时起,世界已经走过了极其漫长的道路。

“A computer can never be held accountable,” IBM wrote in a 1979 internal training manual. “Therefore, a computer must never make a management decision.”

来自128个国家的政府特使周一在日内瓦会聚一堂,决定是否就为无需人类输入即可杀戮的自主武器系统制定具有法律约束力的规则展开会谈。

The world has come an awful long way since.

这个问题并非纸上谈兵。在乌克兰上空,莫斯科和基辅都已部署了致命的无人机,这些无人机配备有人工智能制导系统,即使与人类操作员失去联系,也能识别并打击预先编程的目标。

Government envoys from 128 nations met in Geneva Monday to decide whether to start talks around setting down legally binding rules for autonomous weapon systems that can kill without human input. The question is not an academic one.

这些进展——如果这个词适用的话——源于驱动所有战争的不断升级的残酷逻辑:如果一方制造信号干扰器,在攻击无人机击中目标前将其逼离航线,另一方就会制造不再需要任何信号的无人机。

In the skies over Ukraine, Moscow and Kyiv have both unleashed lethal unmanned drones equipped with AI-powered guidance systems that allow them to identify and strike pre-programmed targets even if they lose contact with their human operator.

据信,俄罗斯和乌克兰都在测试更进一步的攻击无人机——不仅能独立锁定由人类操作员为其选择的目标,还能自行选择目标。

These advances – if the word applies – have been born of the relentless logic of escalation that drives all wars: if one side builds signal jammers to drive attack drones off course before they hit their target, the other will build drones that have no further use of a signal.

《纽约市报》的报道表明,第一架夺去人类生命的完全自主无人机已经发射。乌克兰法医调查人员认为,今年9月,俄罗斯无人机在乌克兰扎波罗热市袭击一家加油站并造成3人死亡,其中半数无人机完全由人工智能系统引导。

Both Russia and Ukraine are believed to be testing out attack drones that go a step further – not just independently locking onto the targets chosen for them by human operators, but choosing the targets themselves. New York Times reporting suggests that the first fully autonomous drone to take human life has already been launched.

本周的会谈正在联合国《特定常规武器公约》(CCW)审议大会这一不太寻常的名称下举行,会谈预计将会十分紧张。虽然多数成员国原则上支持对自主武器系统进行具有约束力的监管,但包括美国和俄罗斯在内的大主要军火生产国一直反对达成此类条约。根据该公约,所有决定必须通过协商一致的方式达成。

Ukrainian forensic investigators believe half a dozen Russian drones that killed three people when they hit a petrol station in Ukraine’s city of Zaporizhzhia in September were entirely guided by AI systems. This week’s talks, which are taking place under the unlikely name of the Review Conference for the UN’s Convention on Certain Conventional Weapons (CCW), are likely to be tense. While a majority of member states support binding regulation around autonomous weapon systems in principle, major arms-producing countries including the US and Russia have consistently opposed such a pact.

法国24台采访了瑞典斯德哥尔摩国际和平研究所高级研究员兼治理与人工智能项目主任文森特·博拉宁,以了解本周会议所涉利害关系。

Under the convention, all decisions must be reached by consensus.

法国24台:在你看来,自主武器系统有哪些特点,证明了对其进行进一步监管(甚至禁止)的正当性或必要性——超出了国际人道主义法现有涵盖的范围?

FRANCE 24 spoke with Vincent Boulanin, a senior researcher and director of the Governance and Artificial Intelligence Programme at the Swedish International Peace Research Institute, to find out just what is at stake at this week’s conference.

关于自主武器的讨论在联合国《特定常规武器公约》框架下已经进行了十多年,这是有关国家出于几个原因认为重要的一个话题。

FRANCE 24: What is it about autonomous weapon systems that you believe justifies or necessitates further regulation – or even prohibition – beyond what is already covered under international humanitarian law?

主要原因涉及它能否在符合现有国际人道主义法的情况下使用。核心关切在于,它迫使我们去思考究竟是谁在围绕武力使用做出决定。

Autonomous weapons have been discussed in the context of the UN Convention on Certain Conventional Weapons for more than 10 years, and it's a topic that the states concerned have considered important for a few reasons. The main reason relates to whether it can be used in compliance with existing international humanitarian law.

在自主武器的背景下,其关键特征是决定以及识别、选择、确定优先级和打击目标的方式,基本上是由武器自身完成的——通过一个算法过程,系统自动检测目标,使用算法确定优先级,然后攻击目标。

The core concern is that it forces us to kind of reflect on who is making decisions around the use of force.

这基本上是核心辩论所在。各国最终同意这一事实,即根据国际法,人类或指挥官最终才是负责做出决定的人,但问题是,自主功能将如何影响他们确保其使用合法性的能力。

In the context of autonomous weapons, the key feature is that the decisions, the way targets are being identified, selected, prioritised, and engaged is basically done through the weapon itself – through an algorithmic process where the systems automatically detect the target and use an algorithm to prioritise and then attack the target. And that’s basically a core debate.

这就是为什么各国十年来一直在讨论这方面的法律技术细节,涉及它们在多大程度上可以依赖自主武器,以及在多大程度上需要限制这些系统的使用方式。

States eventually agree on the fact that, ultimately, humans, or the commanders, are the ones responsible for making decisions under international law, but the question is, how do the autonomous features affect their ability to see that their usage will be lawful.

法国24电视台:在俄罗斯对乌克兰发动全面入侵,以及部分甚至可能完全自主的无人机在乌克兰进行测试甚至可能部署的扩散之后,这些讨论似乎变得更加紧迫。自2022年以来我们在乌克兰看到的情况如何影响了这些辩论?

So that's why states have been discussing for 10 years now the legalistic technicalities of this in terms of to what extent they can rely on autonomous weapons, and to what extent do they need to constrain how these systems may or may not be used. FRANCE 24: Those discussions seem to have taken on a fresh urgency following Russia’s full-scale invasion of Ukraine and the proliferation of partially and potentially even fully autonomous unmanned drones that are being tested and possibly even deployed in Ukraine.

取决于你如何定义自主武器,这些系统已经存在相当长一段时间了,各国为了防御目的开发自主武器系统已有几十年历史。

How have these debates been affected by what we’ve seen in Ukraine since 2022?

其理念是,在某些时间至关重要的语境下,例如你确实没有足够的时间对来袭导弹做出反应,那么你就需要依靠自动化流程来识别和交战目标。人们普遍认识到,这类使用场景通常没问题,因为它们被限制在非常具体的使用场景和防御目的中,你可以知道这些系统将如何运行,以及你将交战何种类型目标——在大多数情况下,那将是来袭的弹丸或大型军事物体。

Depending on how you define autonomous weapons, these systems have been around for quite a long time where states have been developing autonomous weapon systems for defensive purposes for decades. And the idea was that in some contexts where time is critical, and you don't really have enough time to react to, for instance, an incoming missile, then you rely on automated processes to identify and engage targets.

在乌克兰战争的语境下,令人不安的是,我们现在可以看到更多进攻性使用场景,即无人机和导弹的组合,自主性使这些系统能够在由于通信被切断而无法在技术上实现远程控制的地区自动识别或交战目标——即“最后一英里”目标锁定。

And it was broadly recognised that these types of use cases were generally fine because they were constrained to very specific use cases and defensive purposes where you could know how the systems would operate and what type of target you will engage – and in most cases, that would be incoming projectiles or large military objects.

令人惊讶的是,乌克兰战场上的乌克兰和俄罗斯无人机在很大程度上仍然是由人工远程驾驶和控制的。但由于这种通信阻断,它们正越来越依赖自动化目标锁定功能来完成任务。并且有报道称,它们已被用于攻击人类目标。

And what is unsettling in the context of the war in Ukraine is that we can see now more offensive use cases, combinations of drones and missiles where autonomy is enabling these systems to automatically identify or engage targets in areas where remote control is not technically feasible because communication is denied – “last-mile” targeting. Surprisingly, drones in Ukraine from both the Ukrainian side or Russian side are still to a large extent remotely piloted and controlled. But due to this communication denial, they're increasingly relying on automated targeting capabilities to finish the mission.

这就是新颖之处——突然之间,长期以来被视为极具未来感的东西正日益在今天上演,我们看到越来越多的案例表明,武器正在以自主模式被用来完成攻击。

And there have been reports that they have been used against human targets.

改变的是,曾经被认为属于未来主义的争论,现在已经变得非常具体和真实。

So that’s the novel element – where suddenly something that for a long time would be seen as futuristic is increasingly taking place today, where we see increasing use cases where the weapon is being used in autonomous mode to finish an attack.

法国24电视台:对于国家使自己受到具有约束力的法规限制、从而从我们在乌克兰看到的这种相互升级的逻辑中退缩的方式,你看到了哪些先例?

What has changed is that what used to be considered a futuristic debate is now something that is very tangible and real.

有一些案例表明,国家最终承认其部署的武器从法律或伦理角度来看是有问题的。

FRANCE 24: What precedents do you see for ways in which states have submitted themselves to binding regulations that have pulled them back from this logic of mutual escalation that we’re seeing in Ukraine?

一个著名的案例原本被视为自主武器系统对话开始时的蓝图,那就是关于致盲激光的议定书[编者按:《1995年关于激光致盲武器的议定书》禁止在战争中使用可对人的视力造成永久性损伤的激光武器——这种武器此前未曾、迄今也未曾在战场上使用过]。这是各国首次同意禁止一项尚未得到妥善部署的技术,并且它们最终能够达成共识,即使用会产生永久影响的致盲激光是有问题的。

There's been a few cases where states eventually agreed that the weapon that they had deployed was considered problematic from a legal or ethical standpoint. One famous case that was meant to be the blueprint when the conversation on autonomous weapon systems started was the protocol on blinding lasers [Editor's note: The 1995 Protocol on Blinding Laser Weapons prohibits the use in war of laser-based weapons that could permanently damage people's vision – a kind of weapon that had not, and still hasn't, been used on the battlefield].

这被认为是一个成功,而作为自主武器争论的发起者并从一开始就推动禁令的非政府组织,一直希望复制这一模式。

That was one of the first times that states agreed to ban a technology that had not yet been properly deployed, and they could eventually agree on the fact that it would be problematic to use blinding lasers that have a permanent effect.

遗憾的是,这种情况并没有发生,很大程度上是因为自动武器和人工智能的使用场景远不够具体,这使得辩论看起来更加复杂。

That was considered to be a success, and the NGOs that were at the origin of the debate on autonomous weapons and were pushing for a ban from the beginning were hoping to replicate the model.

我们谈论的不仅仅是一个具有非常特定效果的非常具体的使用场景,就像你在反人员地雷的情况中所看到的那样。

Unfortunately that did not happen, in large part because the case of autonomous weapons and AI is much less specific in terms of use case, and that makes the debate look more complicated.

自动武器可以呈现出如此多的外形和形式,并被用于如此多种不同类型的冲突场景——非对称战争,比如你在地道中追捕恐怖集团,与美国和中国在海上交战的高强度冲突等等。

We're not just talking about a very specific use case with a very specific set of effects, like you might have in the case of anti-personnel landmines.

但回到你的问题,我认为地雷的案例是一个有用的例子,因为我们都知道它们曾被使用,并且各国最终一致认为这些武器正在导致不必要的痛苦和多余的伤害,至少其中一些国家赞成禁止这些武器是有道理的。

Autonomous weapons can take so many shapes and forms and be used in so many different types of conflict scenarios – asymmetric warfare, where you’re chasing terrorist groups in tunnels versus high-intensity conflicts involving the US and China fighting at sea, and so on.

所以这是可能的,但归根结底,我认为这取决于许多因素。从历史上看,重大进展与人道主义损害的实际证据有关,即有实际记录的案例表明该技术正对平民造成不必要的伤害。

But to go back to your question, I think the case of landmines is a useful example, because we all know they have been used, and eventually states agreed that these were weapons that were leading to unnecessary suffering and superfluous injury and at least some of them agreed that it would make sense to ban those. So it's possible, but at the end of the day I think it depends on a number of factors.

但更重要的是,我认为这也是政治背景与各国达成共识的意愿之间的方程式。我今天看到的挑战是,目前的政治条件让各国很难就一项新法规达成一致。至少是一项具有法律约束力的法规。

Historically, major progress was linked to actual evidence of humanitarian harm, where there are actually documented cases that technology is leading to unnecessary harm to civilians. But more importantly, I think it's also the equation of the political context and the appetite of states to work towards a consensus. And the challenge that I see today is that the political conditions are very difficult for states to agree on a new regulation.

法国24台:在进入本周于日内瓦举行的讨论之际,美国、俄罗斯和以色列等主要武器制造国似乎确实存在阻力。你如何解释这种阻力?

At least a legally binding regulation. FRANCE 24: Going into this week’s discussions in Geneva, it does seem like there is resistance from major weapons-manufacturing states such as the US, Russia and Israel.

首先,我认为重要的一点是要强调,自公约启动以来,俄罗斯、美国以及在某种程度上的以色列,从一开始就一直反对关于具有法律约束力的条约或具有法律约束力的禁止自动武器的想法。

How do you account for this resistance?

不同之处在于,公约启动之时,至少美国仍然愿意并积极主动地就实质内容进行讨论,并愿意就一项成果文件达成一致。该文件可以就自主武器如何在国际法框架内使用等问题提供新的指导,也可以进一步说明在发展和使用此类能力时,什么才是负责任的行为。

First of all, I think it's important to underline that Russia, the US and to some extent Israel have been opposed to the idea of a legally binding treaty or a legally binding prohibition of autonomous weapons since the very beginning, since the convention started. What was different is that at the time when the convention started, the US at least was still willing and proactive in having discussions on the substance and was open to possibly agreeing on an outcome document that could provide new guidance as to, for instance, how the use of autonomous weapons can be used in compliance with international law, and could add further details as to what would be responsible behaviour when it comes to the development and use of such capabilities.

但特朗普政府上台后,情况发生了变化。现在,美国的立场强硬得多,在某种程度上反对任何超越现有国际法的实质性进展。

But that shifted with the Trump administration. Now, the US has a much harder line and is now in a way opposing any substantial progress that would go beyond existing international law.

这种反对部分也与一种突出观点有关:美国正与中国进行一场高风险的竞争,他们需要为未来美中之间可能爆发重大冲突做好准备,因此不应限制自己的作战能力。

And part of that opposition is also linked to the prominence of the idea that the US is engaged in a high-stakes competition with China, and they need to plan for the possibility that there could be a major conflict between the US and China in the future, and therefore, they should not be limiting their ability to fight.

这就是反对相关监管的战略理由。他们并不会非常公开明确地阐述这一点,但我认为这往往就是他们抵制任何监管的根本原因——他们不信任潜在对手会遵守这些新规,并且希望在如何在作战中利用新技术方面尽可能保留回旋余地。

That's the strategic rationale for opposing the regulations. That’s not something that they articulate in a very open and explicit way, but I think that's often the bottom line of why they are resistant to the idea of any regulation – because they don't trust their potential adversaries to abide by this new regulation, and they want to keep as much room to manoeuvre as possible as to how they may be leveraging new technologies in warfighting.

我认为以色列也是同样的逻辑——他们希望保留在对哈马斯战争中利用这项技术的能力,不愿看到自己的进攻能力受到限制。本次采访经过编辑,以控制篇幅并提高清晰度。

And I think that's the same rationale for Israel – they want to keep the ability to leverage that technology in their war against Hamas, and they are not willing to see restrictions on their offensive capabilities. This interview has been edited for length and clarity.

阅读媒体原文 ↗