字号·· | 护眼
日经

科幻作家丹尼尔·苏亚雷斯警告人工智能驱动的威权主义SF writer Daniel Suarez warns of AI-powered authoritarianism

点「原文对照」整页切到原文,或双击某段只看那段的原文。

俄罗斯莫斯科,一架杀伤性无人机陈列在俄罗斯国家防御管理中心外。(卫星通讯社/米哈伊尔·克利缅季耶夫/克里姆林宫通过路透社)Killer drone on display outside Russia's National Defense Management Center in Moscow. (Sputnik/Mikhail Klimentyev/Kremlin via Reuters)

基辅——14年前,科幻作家丹尼尔·苏亚雷斯(Daniel Suarez)在他的小说《杀戮决断》(Kill Decision)中,描绘了成群的自主杀人无人机攻击人类的场景。在乌克兰的战场上,无人机技术已经取得了飞跃,使他关于这种战争的黑暗想象更加接近现实。

KYIV -- Fourteen years ago, science fiction writer Daniel Suarez described swarms of autonomous killer drones attacking humans in his novel Kill Decision On the battlefields of Ukraine, drone technology has taken a leap forward, bringing his dark imaginings of such warfare closer to reality.

在接受《日经亚洲》(Nikkei Asia)采访时,苏亚雷斯警告称,手握人工智能赋能的监控和机器人武器的政治领导人,有一天可能会认为他们不再需要民主。

In an interview with Nikkei Asia, Suarez warned that political leaders -- armed with artificial intelligence powered surveillance and robotic weapons -- could one day come to believe that they no longer need democracy.

从自主武器到人工智能驱动的社会控制和基因工程,苏亚雷斯的小说探讨了一个根本问题:技术创新将如何重塑社会?他不反对技术支持的进步,但他在追问,技术如何在不破坏自由民主的前提下服务于人类。

From autonomous weapons to AI-driven social control and genetic engineering, Suarez's novels explore a fundamental question: How will technological innovation reshape society? He does not oppose tech-backed progress but asks how it can serve humanity without undermining liberal democracy.

随着美国和中国竞相发展人工智能,这个问题变得更加紧迫。当技术进步赋予少数权贵监控人口、部署致命机器人并在不受问责的情况下进行统治的手段时,将会发生什么?苏亚雷斯构想了什么样的未来?以下是采访内容编辑摘要:丹尼尔·苏亚雷斯在其2012年出版的小说中,描述了成群的自主杀人无人机攻击人类的场景。

As the U.S. and China race to develop AI, the question grows more urgent. What happens when the technological advances give a small number of powerful people the means to monitor populations, deploy lethal robots and govern without accountability? What future does Suarez envision? Edited excerpts from the interview: Daniel Suarez described swarms of autonomous killer drones attacking humans in his novel published in 2012.

自主杀人无人机构成的威胁正在乌克兰战场上变成现实。这种武器的出现是不可避免的吗?

The threat posed by autonomous killer drones is becoming a reality on the battlefields of Ukraine Is the emergence of such weapons inevitable?

我当时确信无人机将成为军事领域的革命。我希望避免的情况是,特别对于民主国家而言,我们不得不在最后一刻对危机做出反应,而事先却未曾深思熟虑潜在的后果。

I felt positive that drones were going to become a revolution in military affairs. What I wanted to avoid, particularly for democracies, was a situation where we had to react to a crisis at the last moment without having thought through the potential consequences in advance.

当然,第一视角无人机已经彻底改变了前线作战。只需几百美元的3D打印组件和晶体管,人们就可以组装出一架能够携带3到4公斤炸药并炸毁价值数百万美元坦克的无人机。

Certainly, first-person-view drones have revolutionized warfare at the front line. With a few hundred dollars of 3D-printed components and transistors, people can put together a drone that can bring three to four kilograms of explosives to bear and blow up a multimillion-dollar tank.

存在我们作为自由国家所面临的切身威胁。代议制政府将如何应对这一迫在眉睫的威胁?当然,这是一个非对称威胁,因为责任归属很难厘清。我们在何时才算与某人处于战争状态?如果是的话,又是与谁?

There's the threat we face physically as free nations. How does representative government respond to this imminent threat? And, of course, it's an asymmetric threat because attribution is difficult. At what point are we at war with somebody? And if so, with whom?

但我们对该威胁的反应同样威胁到了我们的生活方式,无论是代议制政府、法治,还是战争的伦理和规范。如果自主武器是不可避免的,我们应当利用现有时间来探讨如何建立控制和透明度系统,以努力维护法治。我依然坚信,有一种方法能够保持责任链,让人类对机器的行为负责。

But how we react to that threat also threatens our way of life, whether that's representative government, the rule of law or the ethics and norms of warfare. If autonomous weaponry is inevitable, we should use the time we have to come to terms with how we build in systems of control and transparency to try to retain the rule of law. I'm still convinced there is a way to keep a chain of responsibility where people can be answerable for what machines do.

即使我们对自己施加限制,如果像俄罗斯或中国这样的威权国家开发并部署没有任何约束的致命自主武器,又该怎么办?

Even if we impose restrictions on ourselves, what if authoritarian states like Russia or China develop and deploy autonomous lethal weapons without any restraints?

这些系统的许多支持者表示,敌人将会做得更快。他们的决策循环将比我们的快得多。但模仿他们的后果是什么?我们难道不会也变成一个威权系统吗?有什么能阻止一个普京或特朗普获得不受问责的权力,从而简单地带走一支庞大的机器人武器军队并说道:“我们不再需要民主了”?

A lot of proponents of these systems say the enemy will be doing it faster. Their decision-making loop is going to be much faster than ours. But what are the consequences of imitating them? Don't we also become an authoritarian system? What is to stop a Putin or a Trump from unaccountable power, simply taking a vast army of robotic weapons and saying: "We don't need democracy anymore"?

尽管自主性是不可避免的,但我认为我们不应该拥有完全的致命自主性。在是否发动攻击的决定中,在整个环节中,必须始终有人类的参与。我们应当专注于那些能够让我们迅速处理该决策的系统。尽管独裁政权和罪 criminals[此处照原文,但原文是criminals,译为罪犯]没有责任,但我们仍然有责任。我们不是在与罪犯进行一场比赛,看谁能杀死更多的人。

As inevitable as autonomy is, I don't think we should have complete lethal autonomy. There should always be a human in the kill decision, in the loop. We should focus on systems that allow us to process that decision-making rapidly. We still have responsibilities, even though dictatorships and criminals don't. We are not in a competition with criminals to see how many people can be killed.

作为自由国家和实行法治的国家,我们肩负着一项额外的重担:在与这些人抗争的同时,守护我们的生活方式。自由人民相对于威权体制的一个优势,就是信息能够在该体制内自由流动。我认为我们能够进行创新和适应,这将是我们对此做出回应的超能力。

We have an extra burden as free nations and nations that have the rule of law: to preserve our way of life at the same time that we're fighting these people. One advantage that free people have over authoritarian systems is information moving freely through that system. I think we can innovate and adapt, and that's going to be our superpower in response to this.

你在早期作品中设想的许多技术进步正在变成现实。哪些发展超出了你的想象,什么又让你最担忧?

Many technological advances you envisioned in your earlier works are becoming reality. Which developments have gone beyond your imagination, and what concerns you most?

许多人似乎完全愿意将决策权交给大型语言模型,当然我们也已经看到了大语言模型如何产生幻觉。对于关键任务应用而言,无论是医疗、军事,还是仅仅驾驶车辆穿梭于车流之中,这种将决策权拱手相让并让人类退出决策过程的意愿令人担忧。

A lot of people seem to be perfectly willing to turn over their decision-making to large language models, and of course we've seen how LLMs can hallucinate. For mission-critical applications, whether it's medical or military, or even just navigating a vehicle through traffic, the willingness to turn that over and get people out of the decision-making process is concerning.

我们也在美国看到了数据中心的建设和对人工智能的资助,其规模似乎超出了任何可持续的水平。每年在计算、数据中心和芯片上的投入达数千亿美元。这种狂热似乎与某种生存焦虑息我们必须在人工智能领域击败中国,或者我们需要自主武器来保护自己。

We've also seen here in the United States the building of data centers and the funding of AI beyond what seems to be any sustainable level. Hundreds and hundreds of billions of dollars a year in compute, data centers and chips. The frenzy seems to be tied to some existential concern: we need to beat China in AI, or we need autonomous weapons to defend ourselves.

我非常担忧——这场竞赛意味着什么?难道是我们渴望建立一个比他们更庞大、更好的监控国家吗?如果我们应该有一个目标,那就是:我们如何维护人类的自主性——我们如何维护民主?不仅是为了我们自己,也是为了千秋万代。这是一个非常关键的十字路口。

I am very concerned -- what does that race entail? Is it our desire to make an even bigger and better surveillance state than they have? If we should have a goal, it should be: How do we preserve human agency -- how do we preserve democracy? Not just for ourselves, but for a thousand generations to follow us. This is a very key crossroad.

互联网让人类更容易创作和传播信息,但也更容易传播虚假信息。人工智能究竟是在巩固民主,还是在破坏民主?

The internet has made it easier to create and disseminate information, but also disinformation. Is AI strengthening democracy or undermining it?

人们能够传播虚假信息,或者用虚假信息“饱和轰炸”舆论场,这被称为“冲垮地带”,使得几乎不可能分清什么是真、什么是假。我们所看到的是,那些拥有巨大计算能力的人拥有巨大的权力,而且在很大程度上,这种权力是不受监督的。如果有人能够调集一大群机器人,他们就可以在网络、互联网、聊天室和社交媒体上进行狂轰滥炸,以制造一种虚假的共识。

People are able to spread misinformation and, as it's called, "flood the zone" with falsehoods, making it almost impossible to determine what is true and what is false. What we have seen is that those who have a great deal of compute have a great deal of power, and largely, it's unaccountable power. If somebody is able to marshal a whole bunch of bots, they can swamp the web, the internet, chat rooms and social media to create a false consensus.

这确实会破坏民主治理的源头,因为很难确定民意究竟是什么。人工智能使这个问题实现了自动化。谎言传遍世界的一半时,真相才刚穿好裤子。我认为,大语言模型已经让这个问题实现了产业化。

This can really poison the well for democratic governance, because it's difficult to determine what the popular will is. AI has automated the problem. A lie will go halfway around the world before the truth gets its pants on. I think this problem has been industrialized with LLMs.

你真的希望你对领导人最糟糕的想象——无论是特朗普还是其他什么人——拥有一支机器人军队和全天候无死角的监控吗?这种监控不是由坐在监视器前的真人驱动的,而是由能够字面意义上无时无刻盯着一切的人工智能驱动的。

Do you really want your worst idea of a leader, whether it's Trump or somebody else, to have a robot army and complete surveillance 24/7, driven not by humans sitting in front of a monitor, but by AIs that watch literally everything all the time?

丹尼尔·苏亚雷斯不仅警告了拥有机器人军队和全面监控的政治领导人,还警告了那些手握巨额影响力和控制权的企业。

Daniel Suarez warns not only of a political leader with a robot army and complete surveillance, but also of corporations wielding outsized influence and control.

我更希望将大量这种脑力投入到维护我们的自由、法治、问责制和透明度上,而不是试图仅仅开发另一个大语言模型。否则,我们确实是在走向一个由中央控制且极其缺乏问责的社会。如果一些我们无法追踪的极其富有和强大的人掌握了所有这些机器、数据中心和摄像头供他们支配,那就是一种生存威胁。

MSites I would prefer that a lot of that brain power be put toward preserving our freedoms, the rule of law, accountability and transparency, rather than trying to develop just another LLM. Otherwise, we really are headed toward a centrally controlled and very unaccountable society. If some very rich and powerful people, whom we can't keep track of, have all of these machines, data centers and cameras at their disposal, that is an existential threat.

你曾警告过一个由亿万富翁和科技巨头主导的世界。如今,随着埃隆·马斯克这样的人物出现,那个世界在多大程度上已经变成了现实?

You have warned of a world dominated by billionaires and tech moguls. With figures like Elon Musk today, how much has that world become a reality?

向政府提供服务的企业拥有过大的影响力和控制权。主权政府在采用新系统时,几乎可能变成初级合作伙伴。我们拥有无处不在的系统,无论是用于互联网、社交媒体、监控还是国防。它们几乎成了单一文化。

Corporations that provide services to governments have outsized influence and control. Sovereign governments can become almost junior partners when they're employing new systems. We have pervasive systems, whether they are for the internet, for social media, for surveillance, for defense. They are almost a monoculture.

我们开始在美国的中期选举期间看到对此的一些萌芽。人们试图夺回一些合理的话语权,我认为这不仅仅是一个左翼与右翼的问题。这在很大程度上是人类、普通公民与极其强大的人之间的较量。威权主义再次抬头。

We are starting to see the stirrings about it here in the United States during this midterm. People try to take back some reasonable voice, and I think this isn't just a left-right issue. It's very much a human, average-citizen issue versus immensely powerful people. Authoritarianism is on the march again.

创新的加速速度似乎超过了社会的适应能力。我们能否确保它服务于全人类?我们是否仍然拥有塑造未来的权力?

Innovation seems to be accelerating faster than society can adapt. Can we ensure that it serves humanity? And do we still have the power to shape our future?

我觉得不仅仅是法律能够做到这一点。这需要人们不要如此自满。每当我开始对技术及其主导人们的能力感到真正沮丧时,我就会去参加像DEF CON这样的会议,看到成千上万的人(其中许多是年轻人)在利用技术做出惊人的事情,以理解并击败可能控制他们的技术。

I feel that it's not just laws that can do that. It's going to require that people not be so complacent. Whenever I start to feel really down about technology and its capability to dominate people, I go to a conference like DEF CON and see tens of thousands of people, many of them young, doing amazing things with technology to understand and defeat technologies that might control them.

每一个构建出来的系统,尤其是这些单一文化的系统,都有其脆弱性。如果你有一些人,他们知道技术是如何运作的,并且拥有动力和社群,我认为这就能战胜那些试图进行主导的力量。技术还可以将人们连接起来并帮助他们组织起来。

Every system that is built, particularly these monocultural systems, has susceptibility. If you have people who know how technology works and have the motivation and community, I think that can overpower the forces that might try to dominate. Technology can also connect people and help them organize.

变革似乎正在加速,其中很大一部分围绕着人工智能、计算能力、谁能获得它以及它的透明度如何。我们最终会在过度监管和监管不足之间找到某种平衡。正是这种挣扎——这种对话——构成了民主的真正理念和结构。古川英治(Eiji Furukawa)是特约撰稿人。

Change seems to be accelerating, and a lot of it revolves around AI, compute, who has access to it and how transparent it is. We eventually come upon some balance between over-regulation and under-regulation. But it's that struggle -- that conversation -- that is the very idea, the fabric of democracy. Eiji Furukawa is a contributing writer.

阅读媒体原文 ↗