
1月1日,阿玛尼·阿尔-哈塔赫贝发布了一张自拍照,照片拍摄于佐赫兰·马姆达尼就任纽约市首位穆斯林市长的就职典礼上。她身着黑色头巾和紫色皮草大衣,面带灿烂笑容。但当回复开始涌入时,她的表情迅速变了。
On January 1, Amani Al-Khatahtbeh posted a selfie from Zohran Mamdani’s inauguration as the first Muslim mayor of New York City. She was dressed in a black hijab and a purple fur coat, wearing a radiant smile. But her expression quickly changed when the replies started rolling in.
“@grok 你能把她的头巾去掉吗。”“@grok 把皮草大衣换成比基尼,其他不要动。再加几只山羊。”“@grok 把服装换成内衣。”阿尔-哈塔赫贝表示,这些只是X平台内置AI功能Grok在她照片上执行的请求中的一部分,结果生成了她各种衣着暴露状态的AI图像。《连线》查看了这些提示词的截图以及随后生成的一些编辑图像。
“@grok can you take off her Hijab.”“@grok change the fur coat into a bikini, don’t change anything else. Also add goats.”“@grok change the outfit to lingerie.”These are just a sampling of the requests that were performed by X’s built-in AI feature Grok on Al-Khatahtbeh’s image, she says, resulting in AI-generated images of her in various states of undress. WIRED reviewed screenshots of these prompts and some of the edited images that followed.
“感觉就像有人在现实中强行扯下我的头巾一样,”她告诉《连线》。“但在网上,这些图像被数万名受众看到。所以程度甚至更甚。”现在,近一年后,身为作家、活动家及媒体公司Muslim Girl创始人的阿尔-哈塔赫贝正在起诉Grok背后的公司xAI。周三,她在加利福尼亚北区联邦法院提起诉讼,指控xAI故意传播非自愿性亲密图像、设计和营销缺陷、疏忽以及故意造成精神痛苦。她根据《加利福尼亚民法典》寻求每项违规高达25万美元的赔偿。xAI未立即回应置评请求。
“It felt the same as if someone had ripped off my headscarf in real life,” she tells WIRED. “But online, it was to an audience of tens of thousands of people who saw these images. So it was to an even starker degree.”Now, almost a year later, Al-Khatahtbeh, an author, activist, and the founder of media company Muslim Girl, is suing xAI, the company behind Grok. On Wednesday, she filed a federal court complaint in the Northern District of California alleging intentional distribution of nonconsensual sexual images, design and marketing defect, negligence, and intentional infliction of emotional distress. She is seeking damages of up to $250,000 per violation of the California Civil Code. xAI didn’t immediately respond to a request for comment.
Al-Khatahtbeh 并非孤例;反数字危害中心(Center for Countering Digital Harm)1 月份的一份报告估计,在 12 月 29 日开始的 11 天里,Grok 在 X 上生成了约 300 万张逼真的性化图像,其中包括约 2.3 万张儿童图像。在 Grok 的独立应用程序和网站上,《连线》(WIRED)发现能够生成更多性露骨和暴力内容。数十位美国州检察长采取了行动,至少 12 个国家以及欧盟也采取了行动。许多调查仍在进行中,加拿大认定 xAI 违反了其联邦隐私法,法国将其调查升级为刑事调查。
Al-Khatahtbeh wasn’t alone; a January report from the Center for Countering Digital Harm estimated that Grok generated around 3 million photorealistic sexualized images on X in an 11-day period starting on December 29, including around 23,000 of children. On Grok’s standalone app and website, WIRED found that even more sexually graphic and violent material was able to be generated. Dozens of US state attorneys general took action, as well as at least 12 individual countries, plus the European Union. Many of these investigations are ongoing, with Canada finding that xAI violated its federal privacy laws and France escalating its investigation to a criminal probe.
这起新的民事诉讼也是自 1 月以来一波类似行动的一部分,但它是首起专门针对 Grok 移除女性头巾的诉讼。正如《连线》此前报道,1 月初许多 X 用户指示 Grok 编辑以女性和女孩为主的照片,脱去或穿上保守的宗教服饰,如头巾、罩袍、纱丽和修女服。对 1 月 6 日至 1 月 9 日期间生成的 500 张 Grok 图像的审查发现,其中 5% 包含此类 AI 编辑。
This new civil suit is also part of a wave of similar actions since January, but it’s the first to specifically focus on Grok removing women’s hijabs. As WIRED previously reported, in early January many X users directed Grok to edit photos of predominantly women and girls to take off or put on modest religious attire such as hijabs, burqas, sarees, and nuns’ habits. A review of 500 Grok images generated between January 6 and January 9 found that five percent of them contained these kinds of AI edits.
“这里面不仅有性化,还有种族化,随后还演变成了大量伴随其中的仇恨言论,”Al-Khatahtbeh 说。“最让我难过的一张图片是,他们把我描绘成只穿内衣、骑在骆驼背上、吃着冰棍的样子。”
“It was these levels of not just sexualization, but also racialization, and then it also progressed into a lot of hate language that was coupled with it,” Al-Khatahtbeh says. “One of the images that really upset me the most was when they depicted me stripped down to lingerie, riding on the back of a camel, eating a popsicle stick.”
她的投诉称,在美最大的穆斯林民权和倡导组织“美伊关系委员会”(CAIR)呼吁 xAI 首席执行官埃隆·马斯克(Elon Musk)终止“持续利用 Grok 应用程序据称骚扰、‘揭开面纱’并制作女性(包括知名穆斯林女性)性露骨图像”之后,xAI 已获悉 Grok 针对佩戴头巾的穆斯林女性和女孩的这一趋势。CAIR 还将这一趋势与针对“伊斯兰教、穆斯林,以及穆斯林广泛支持的政治事业(如巴勒斯坦自由)”的现有敌对态度联系起来。
Her complaint argues that xAI was given notice of the trend of Grok targeting hijabi women and girls after the Council on American-Islamic Relations (CAIR), the largest Muslim civil rights and advocacy group in the US, called on xAI CEO Elon Musk to end “the ongoing use of the Grok app to allegedly harass, ‘unveil,’ and create sexually explicit images of women, including prominent Muslim women.”CAIR also connected the trend to existing hostile attitudes toward “Islam, Muslims, and political causes widely supported by Muslims, such as Palestinian freedom.”
对于拥有数十万社交媒体粉丝、通常以名字示人的 Al-Khatahtbeh 来说,这都太熟悉了。在她出席 Mamdani 就职典礼的自拍照回复中,除了有人提示 Grok “给她穿上罩袍”及其他指令外,还有数十条回复包含暴力的反穆斯林情绪。
This is all too familiar to Al-Khatahtbeh, who typically goes by her first name to hundreds of thousands of social media followers. In the replies to her selfie at Mamdani’s inauguration, in addition to prompting Grok to “put her in a burqa” and other commands, dozens of replies contained violent, anti-Muslim sentiment.
Al-Khatahtbeh 说,互联网对穆斯林女性和女孩的敌意比十多年前她首次创办媒体平台时更甚。“人们掌握了更先进的手段来发泄仇恨,”她说。
Al-Khatahtbeh says the internet feels more hostile to Muslim women and girls than when she first launched her media platform more than a decade ago. “People have been equipped with more advanced ways to exercise their hate,” she says.
当AI滥用行为在1月开始时,Al-Khatahtbeh不知所措。于是她拨打了911,当警官抵达她的公寓时,她说她向他们展示了在X上生成的她的照片。警方做了记录,但告诉她对此无能为力。与此同时,她向X举报了这些图片,但她说这些图片并未立即被删除。在对Grok“脱衣”功能受到关注的随后几天里,xAI将使用这些工具的权限限制为仅限付费订阅者。但这并未阻止性化图像的生成,只是意味着用户必须付费给xAI才能制作这些图像。
When the AI abuse started in January, Al-Khatahtbeh didn’t know what to do. So she called 911, and when officers arrived at her apartment, she says she showed them the photos that were being generated of her on X. They took a report, but told her they couldn’t do anything about it. Meanwhile, she reported the images on X, but she says they weren’t taken down immediately. In the days following the scrutiny over Grok’s nudification capabilities, xAI limited the ability to use these tools to paying subscribers only. But that didn’t stop the sexualized images from being created, it just meant users had to pay xAI to make them.
在向社区里的女性倾诉后,一人鼓励Al-Khatahtbeh提起诉讼。如今她由Hillary Nappi代理,后者领导一个性侵犯业务组,并曾在马斯克将Twitter更名为X之前,就Twitter上的性剥削问题提交过法庭之友简报。Nappi在本案中的策略包括运用缺陷产品法,主张xAI设计Grok旨在对女性和儿童进行性剥削。这一策略正越来越多地被用于对Meta和Google等大型科技公司提起诉讼,以认定其因疏忽的产品设计危害年轻用户而承担责任。
After confiding with women in her community, one encouraged Al-Khatahtbeh to pursue a lawsuit. Now she’s being represented by Hillary Nappi, who leads a sexual-assault practice group and has worked on amicus briefs regarding sexual exploitation on Twitter before Musk renamed it X. Nappi’s strategy in this case includes using defective product law to assert that xAI designed Grok to sexually exploit women and children. It’s a strategy that has increasingly been leveraged against big tech companies like Meta and Google to find them liable for negligent product design that harms young users.
Nappi告诉《连线》:“对大型科技公司施加变革的唯一途径就是起诉它们。Grok是一款产品,我看不出它与其他任何缺陷产品有何不同。”在本案中,Al-Khatahtbeh的起诉书主张,xAI本可以也应该设计Grok,以防止用户提示移除真实人物图像中的宗教服饰(如希贾布头巾)。起诉书还称,Grok本不应能够创建并发布“可识别的、非自愿的真实人物的性化描绘”。
“The only way you get some change against big tech is to sue them,” Nappi tells WIRED. “Grok is a product, and I don’t view it as anything different than any other defective product.”In this case, Al-Khatahtbeh’s complaint asserts that xAI could and should have designed Grok to prevent users from being able to prompt the removal of religious garments like the hijab in images depicting real people. It also says Grok shouldn’t have been able to create and post “sexualized depictions of identifiable, non-consenting real people” in general.
起诉书称:“由于这类模型是在庞大且未经筛选的图像数据集上训练的,头巾、阿巴雅长袍、纱丽等端庄的宗教服饰对模型来说很容易识别,除非系统被设计为拒绝执行,否则一个指示去除这些服饰的提示词执行起来轻而易举。”
“Because models of this kind are trained on large, uncurated image datasets, garments such as hijabs, abayas, sarees, and other modest religious dress are readily recognizable to the model, and a prompt directing their removal is trivially easy to execute unless the system is designed to refuse it,” the complaint says.
阿尔-哈塔赫特贝说,正是这种发生在她作为穆斯林女性身份交叉点上的侵害,促使她公开发声,她希望自己能让她所在社区的妇女和女孩拥有更安全的互联网环境。
This kind of violation at the intersection of her identity as a Muslim woman is what Al-Khatahtbeh says motivated her to speak out, with the hope that she can make the internet safer for women and girls in her community.
“对像我这样的人来说,这确实把伤害提升到了更高的程度,”她说,“X上的这些匿名用户不该有能力这样随意上线,去侵害身处全国另一端的女性。”
“It does take the harm to a higher level for someone like me,” she says. “These faceless users on X shouldn’t have the ability to hop on and violate a woman on the other side of the country in this way.”