字号·· | 护眼
连线

AI越来越擅长对付网络犯罪分子AI Is Getting Really Good at Messing With Cybercriminals

点「原文对照」整页切到原文,或双击某段只看那段的原文。

欢迎来到《Kernel Panic!》,这是由Lily Hay Newman和Matt Burgess每周发布的通讯,内容聚焦于隐私与数字安全的新世界。如需每周收到此通讯,请在此处注册。

Even as AI supercharges digital scamming and cybercrime, it’s also being recruited in the fight to protect potential victims.

尽管AI正在加剧数字诈骗和网络犯罪,但它也正被招募用于保护潜在受害者的斗争之中。全球各国政府一直在努力应对网络犯罪的全球危机,却因试图打击跨国罪犯而受限。这激发了替代性努力,例如利用自动化手段大规模追踪诈骗者——或许是通过向诈骗者发送垃圾信息来耗尽他们的资源。如今,AI平台正将这类实验转化为对抗网络犯罪的真实工具。

Governments around the world have consistently struggled to address the global crisis of online crime, grappling with the limitations of attempting to crack down on criminals beyond their borders. This has inspired alternative efforts, like using automation to attempt to go after scammers en masse—perhaps by spamming the spammers to squander their resources. Now, AI platforms are turning these types of experiments into real tools in the fight against cybercrime.

过去两年,澳大利亚公司Apate(以希腊欺骗女神命名)一直在构建一个系统,该系统将电话诈骗者转接至与AI机器人的通话。这些机器人经过训练,能够尽可能长时间地维持对话,而当然不会真的上当受骗。其目标是让诈骗者产生足够的希望,使其继续通话,同时收集他们所实施诈骗的情报。

For the last two years, the Australian company Apate, named after the Greek goddess of deception, has been building a system that diverts phone scammers onto calls with AI bots that are trained to keep conversations going as long as possible without, of course, ever actually falling for the scam. The goal is to give scammers enough hope that they stay on the phone, while also collecting intelligence about the scams they’re running.

“我们真正喜欢认为的是,我们正在为诈骗者打造完美的受害者,”Apate的创始人兼首席执行官Dali Kaafar对我们表示。“诈骗者与机器人或客服通话的一分钟,就意味着你可能正在拯救数百,甚至数千人免受该同一诈骗者的侵害,”他补充道,并指出诈骗者能够利用自动拨号工具进行大规模作案。

“What we really like to think is that we’re building the perfect victims for scammers,” Dali Kaafar, the founder and CEO of Apate, tells us. “A minute that a scammer is talking to a bot or an agent is a minute where you’re probably saving hundreds, if not thousands of possible people being reached out to by that exact same scammer,” he says, referencing the ability for scammers to use automated dialing tools.

Kaafar 表示,Apate 的平台被银行使用并得到电信公司的支持,目前拥有约 35 万个机器人。这些机器人不仅能接听电话,还能渗透到网络上的诈骗聊天群组,并通过回复短信来实现挫败诈骗者的目的,同时从他们那里收集情报。他表示,公司已实时收集超过 25 万条关于诈骗者的信息,内容涵盖诈骗网址、洗钱账户和银行信息。

Kaafar says Apate’s platform, which is used by banks and supported by telecom companies, has around 350,000 bots. They not only pick up calls, but also infiltrate scam chat groups online and respond to text messages with similar goals of frustrating scammers while collecting intelligence from them. He says the company has collected more than 250,000 pieces of information about fraudsters in real-time, from scam URLs to money mule accounts and bank details.

Kaafar 说,这些机器人拥有不同的性格、语言能力和背景资料,因此诈骗者希望不会察觉他们正在与人工智能对话。就像真实的人一样,“有时[机器人]会有 WhatsApp,有时则没有。有时他们会接听电话,有时他们只是真的挂断诈骗者的电话,说‘我稍后再回电给你’。”

The bots have different personalities, language skills, and profiles, Kaafar says, so scammers hopefully don’t detect that they’re conversing with an AI. Like real people, “sometimes [the bots] do have WhatsApp, sometimes they don’t. Sometimes they pick up the phone, sometimes they just actually hang up on the scammer saying, ‘I'll come back to you later,’” he says.

但它们是否具有说服力?Kernel Panic 使用了一个演示版本的工具进行测试,用户在其中扮演诈骗者的角色,与 Apate 的一名 AI “受害者”进行对话。这些角色被设计为表达出适度的怀疑态度,但又保留足够的空间让诈骗者继续尝试赢得他们的信任,这种互动动态我们能够立刻感受到,并且觉得非常令人沮丧。

But are they convincing? Kernel Panic tested the tool using a demo version where users play the role of the scammer and talk to one of Apate’s AI “victims.”The personas are designed to express a healthy amount of skepticism but leave enough openings for scammers to continue to try to win them over, a dynamic that we could immediately feel and found intensely frustrating.

骗子很可能熟悉这种怀疑的互动方式,而卡法尔表示,Apate的通话通常会持续超过两个小时。在我们的测试中,Apate系统表现出积极的参与和响应能力,对话的节奏感觉很自然,即使我们两人轮流扮演朋友“露西”和她的财务顾问“米奇”来与AI互动。尽管我们付出了六分钟的认真努力,但仍未成功让Apate AI系统投资于我们的加密货币机会。

Scammers are likely familiar with this type of skeptical interaction, and Kaafar says that Apate calls regularly go on for more than two hours. In our testing, the Apate system was engaged and responsive, and the timing of the conversation felt natural, even with both of us tag-teaming the AI as a friend, “Lucy,” and her financial adviser “Mickey.”We did not succeed at getting the Apate AI system to invest in our cryptocurrency opportunity even after six minutes of diligent effort. (The returns are truly incredible, though, we swear.)

(虽然回报确实令人难以置信,但我们是说真的。)目前,即使Apate拥有庞大的机器人群在案件中奔波,也只是时间问题——你很快就会收到下一次诈骗电话或短信。网络犯罪分子每年发动数十亿条信息和电话,其中最复杂的运营甚至在规模化的工业设施中开展诈骗活动。虽然专业的反诈骗者和渗透诈骗行动的举措多年来一直在进行关键的威慑工作,但执法部门和研究努力尚未能够遏制数字诈骗的整体扩张。

For now, even with Apate’s massive swarm of bots on the case, it’s only a matter of time before you get your next scam call or text. Cybercriminals initiate billions of messages and calls each year, with the most sophisticated operations running physical, industrial scale scamming sites. And while professional scambaiters and initiatives to infiltrate scam operations have done critical deterrent work for years, law enforcement and research efforts haven’t been able to stop the expansion of digital scamming overall.

但人工智能可能为反诈骗工作开启一个新时代,而不仅仅是骗子自身。警方、社交媒体公司、银行及其他行业之间的情报共享亟需加强,而AI驱动的监控和数据分析可能在这方面提供帮助。此外,越来越多的创新努力正在尝试通过利用网络犯罪分子的“心理弱点”来进行干扰。在某些情况下,这些心理战行动包括直接挑衅勒索软件攻击背后的幕后黑手。随着对干扰骗子和网络犯罪分子的关注度不断扩大,利用生成式AI让他们忙于应对、浪费资源、避免发动其他攻击的策略正变得越来越有效。

But AI could create a new era for anti-scam efforts, not just scammers themselves. Better intelligence sharing between cops, social media companies, banks, and other industries is desperately needed and could potentially be aided by AI-driven monitoring and data analysis. Increasingly, too, innovative efforts have been looking to disrupt cybercriminals by exploiting their “psychological vulnerabilities.”In some cases, these psyops have included directly trolling those behind ransomware attacks. As the focus on disrupting scammers and cybercriminals expands, the use of generative AI as a way to keep them preoccupied, wasting resources, and not launching other attacks appears to be increasingly effective.

以蜜罐为例。多年来,公司和安全研究人员一直部署虚假的虚拟机来吸引黑客、浪费他们的时间、收集信息并了解他们的秘密黑客技术。苏黎世联邦理工学院计算机科学系的博士研究员马克·维罗表示,开源蜜罐提供商越来越多地将大语言模型(LLMs)融入其系统,以使其看起来更加真实。

Take honeypots, for instance. For years, companies and security researchers have deployed false virtual machines to attract hackers, waste their time, gather information, and understand their secret hacking techniques. Mark Vero, a doctoral researcher at the department of computer science at ETH Zurich, says open source honeypot providers have increasingly been incorporating LLMs into their systems to make them appear more realistic.

在最近的研究中,维罗及其同事发现,由LLM驱动的蜜罐能够让AI攻击者对其系统进行“显著更长时间”的攻击,而行为更可预测的蜜罐则不然。“这些具备代理能力的攻击者对LLM模拟的蜜罐更加相信,并且他们将其标记为实际蜜罐的比率也要低得多,”他说。“如果这些系统构建得足够好,那么我认为这对防御者来说是非常有利的。”如果您发现我们下周应该包含的内容,请告诉我们:kernelpanic@wired.com。愿您平安。

In recent research, Vero and colleagues found an LLM-powered honey pot would keep AI agents attacking their system “significantly longer” than a honey pot with more predictable behavior. “The agentic attackers are much more convinced by the LLM simulated honeypots and they also mark them as actual honeypots at a much lower rate,” he says. “If these systems are built well enough, then I think it’s quite advantageous for defenders.”Spotted something we should include next week? Let us know at kernelpanic@wired.com. And stay safe out there.