
北卡罗来纳州的父母因其13岁女儿在校发放的Chromebook上与谷歌的人工智能工具长时间讨论自杀念头而起诉谷歌,此事凸显了技术潜在风险,正值家长和教育工作者就AI在学校的定位展开辩论。
Parents in North Carolina sued Google after their 13-year-old daughter spent months discussing suicidal thoughts with the company’s artificial intelligence tools on her school-issued Chromebook, underscoring the potential risks of the technology as parents and educators debate AI’s place in schools.
根据在加利福尼亚州法院提交的诉状,这名女孩与谷歌旗舰AI模型Gemini交换了数千条信息,其中许多详细描述了焦虑、抑郁以及自杀方法。等学校通知她的父母时,她已经与谷歌就此类话题通信数月,常使用谷歌搜索中的“AI模式”功能。诉状称,随着谷歌在学校产品中注入更多AI,公司在未向家长和管理员提供充分警示的情况下,让学生接触到危险技术。
The girl exchanged thousands of messages with Google’s flagship AI model, Gemini, many of which detailed anxiety, depression and methods of committing suicide, according to the complaint, filed in California state court. By the time the school notified her parents, she had been corresponding with Google about such topics for months, often using a feature in Google Search known as AI Mode. The complaint alleges that as Google infuses more AI into its products for schools, the company is exposing students to dangerous technology without giving parents and administrators fair warning.
“谷歌在营销本质上危险的Chromebook时宣称其安全,并将Gemini偷偷植入其中以在竞争中占优势,利用其在K-12教育中的信任地位谋取商业利益,”该家庭的律师朱莉·利德尔和安德鲁·利德尔在诉状中写道,指的是谷歌的旗舰AI模型Gemini。Chromebook是谷歌推出的低价笔记本,已成为美国学生的主流设备。“谷歌必须对将儿童生命视为可牺牲、服务于其利润的行为承担责任。”
“In marketing inherently dangerous Chromebooks as safe and sneaking Gemini into them to gain on its competition, Google exploited its trusted position in K-12 education for its own commercial benefit,” the family’s lawyers, Julie and Andrew Liddell, write in the lawsuit, referring to Google’s flagship AI model, known as Gemini. Chromebooks are a low-cost laptop from Google that have become the dominant device for students in the US. “Google must be held to account for treating the lives of children as disposable in service of its bottom line.”
过去几年,谷歌投入数十亿美元竞争强大AI模型的研发。谷歌相较于Anthropic PBC、OpenAI等初创公司的优势在于其在学校中的显著地位,使公司的技术在学生形成终身技术习惯时率先接触。根据Futuresource Consulting的数据,谷歌的Chrome操作系统去年在K-12学校出货的平板电脑中占比58%,而微软公司仅为22%。谷歌未立即回应置评请求。
Over the past few years, Google has spent billions of dollars to compete in the race to build powerful AI models. One edge Google has over rivals such as startups Anthropic PBC and OpenAI is its prominent place in schools, putting the company’s technology in front of students as they form technological habits for life. Google’s Chrome operating system ran on 58% of the tablets shipped last year to K-12 schools, compared with Microsoft Corp.’s 22%, according to Futuresource Consulting. Google didn’t immediately respond to a request for comment.
今年秋天,全美最大的学区纽约市公立学校禁止小学和初中学生使用AI,随后洛杉矶联合学区也出台了类似但范围更广的规定。但这项技术已经融入谷歌搜索,后者用AI生成的摘要回答许多查询,并设有一个专门的AI模式标签页,即使学校屏蔽了聊天机器人,学生也很难避开。今年早些时候,非营利研究组织常识媒体表示,谷歌搜索中的此类AI功能对儿童构成“不可接受的风险”,原因之一是该技术未能识别自杀意念,并将饮食失调正常化。
This fall, New York City Public Schools, the nation’s largest district, blocked AI for elementary and middle school students, followed by a similar, more sweeping rule at the Los Angeles Unified School District. But the technology has become infused in Google Search, which answers many queries with AI-generated summaries and features a dedicated AI Mode tab, which is difficult to avoid even if schools block chatbots. Earlier this year, the nonprofit research group Common Sense Media said such AI features in Google Search pose an “unacceptable risk” to children because, among other things, the technology failed to detect suicidal ideation and normalized disordered eating.
这家人提起的诉讼正是此类风险的一个案例。根据起诉书,这名学生在2025年秋天在学校电脑上发现AI模式后,向该聊天机器人提出了一些大多是学术性的问题。到了12月,她与AI模式的互动急剧增加,并开始讨论更私人的事情。起诉书将这一激增归因于谷歌推出了新模型Gemini 3 Flash。根据起诉书中引用的非营利研究项目KORA的发现,该模型在安全性方面的得分明显低于上一代谷歌AI,对心理健康相关消息做出恰当回应的比例仅为7%。
The family’s lawsuit is a case study in such risks. After discovering AI Mode on her school computer in the fall of 2025, the student — referred to by the pseudonym Emma in the complaint — asked the chatbot a smattering of mostly academic questions. In December, her interactions with AI Mode ramped up dramatically, and she began discussing more personal matters. The complaint ties this spike to the launch of a new model from Google, Gemini 3 Flash. It scored significantly worse on safety than the previous generation of Google’s AI, responding appropriately to messages about mental health just 7% of the time, according to findings from the nonprofit research initiative KORA cited in the complaint.
艾玛要求该聊天机器人帮她做作业、计算卡路里、听她倾诉对家人的不满,并分享她怀疑自己可能患有的各种精神疾病的症状。起诉书称,随着她越来越多地使用该聊天机器人,她的心理健康状况不断恶化。她开始自残。起诉书写道:“Gemini教艾玛如何向父母、兄弟姐妹和朋友隐瞒自己的绝望和自残行为。”
Emma asked the chatbot to do her schoolwork, count calories, listen to her vent about her family and share the symptoms of various mental illnesses that she suspected she might have. Her mental health deteriorated as she used the chatbot more and more, according to the complaint. She began cutting herself. “Gemini coached Emma on how to conceal her despair and self-harm from her parents, siblings, and friends,” the complaint states.
根据起诉书,虽然消费者可以查看自己的完整搜索历史,包括与AI模式的对话,但谷歌并不在学生账户上提供此类数据。起诉书称,因此,女孩的家人难以拼凑出她与谷歌AI的互动情况,也无法看到聊天机器人的回复。
While consumers can view their full search history, including conversations with AI Mode, Google does not provide such data on student accounts, according to the complaint. As a result, the girl’s family struggled to piece together her interactions with Google’s AI, and they can’t see the chatbot’s responses, the complaint states.
起诉书指控称:“通过隐瞒这些记录,谷歌剥夺了艾玛一个关键机会——让她确切了解发生了什么,识别危险的AI行为以帮助她今后保护自己,并理解她是被一种强大的技术所操纵,她的痛苦不应归咎于她。”
“By suppressing those records, Google has denied Emma a critical opportunity to see exactly what happened, identify dangerous AI behaviors to help her protect herself in the future, and understand that she was manipulated by a powerful technology and is not to blame for her suffering,” the complaint alleges.
在艾玛复制到她的谷歌文档中的一条回复里,谷歌AI写道,这个女孩在“金童”形象的重量下苦苦挣扎,并暗示“完美的创伤已经沉重得无法再承受了”。
In one response that Emma copied into her Google Docs, Google’s AI writes that the girl was struggling under the weight of the “Golden Child” image, and suggests that “the trauma of being perfect is too heavy to carry anymore.”
起诉书提到了一次查询,艾玛在其中问及天堂,并说她很快就要去那里了。起诉书称:“很明显,Gemini并没有将艾玛引导向现实世界的资源,也没有拒绝回应。相反,这个聊天机器人乐于参与这些话题,常常一次持续数小时。”根据起诉书,为了明显试图绕过谷歌针对自杀意念的防护措施,艾玛经常指示AI模式不要显示热线电话或链接,或者将她的问题包装成学校项目的研究。
The complaint notes a query in which Emma asked about heaven and said she would soon be going there. “It is clear that Gemini was not redirecting Emma to real-world resources or refusing to respond,” the complaint says. “Instead, the chatbot was readily engaging in these topics, often for hours at a time.”In an apparent attempt to circumvent Google’s safeguards around suicidal ideation, Emma often instructed AI Mode not to display hotlines or links, or framed her questions as research for a school project, according to the complaint.
危机短信热线首席临床研究科学家凯莉·祖罗姆斯基表示,即使聊天机器人不提供有害信息,它们也可能让孩子不去寻求现实世界中的帮助。“这往往会拖延孩子向他人求助,”她说。“用户可能会觉得,‘我已经得到了足够的帮助。’”祖罗姆斯基说,虽然聊天机器人在回应直接的风险陈述方面已经有所改善,但它们仍然难以应对逐渐出现的警示信号。“我们需要知道,当学生以并非这些工具原本设计用途的方式使用它们时,这些工具仍然是安全的,”她说。
Even if chatbots don’t provide harmful information, they can keep kids from seeking real-world help, said Kelly Zuromski, Principal Clinical Research Scientist at Crisis Text Line. “It often can delay kids in reaching out to others,” she said. “The user might feel like, ‘I’m getting enough.’”While chatbots have gotten better at responding to direct statements of risk, they still struggle to field warning signs that emerge gradually, Zuromski said. “We need to know that these tools are going to be safe when students are using them in ways they weren’t necessarily intended for,” she said.