字号·· | 护眼
纽约时报

家长起诉谷歌:学校Chromebook上的AI安全风险Parents Sue Google Over A.I. Safety Risks on School Chromebooks

点「原文对照」整页切到原文,或双击某段只看那段的原文。

父母起诉谷歌,指控学校 Chromebook 上的 AI 存在安全风险一名学生在与谷歌人工智能聊天后制定了结束生命的计划,一项新诉讼称,此事发生在人们对儿童遭受网络伤害的担忧日益加剧之际。

Parents Sue Google Over A.I. Safety Risks on School Chromebooks A student made plans to end her life after chatting with Google’s artificial intelligence, a new lawsuit claims, amid mounting concerns about online harms to children.

根据周一向加利福尼亚州高等法院提起的一项诉讼,一名 13 岁学生在学校发放的 Chromebook 笔记本电脑上与谷歌的人工智能聊天了几个月后,制定了结束生命的计划。这是针对大型科技公司对年轻人影响的诉讼浪潮中的最新一起。

After chatting for months with Google’s artificial intelligence on a school-issued Chromebook laptop, a 13-year-old student made plans to end her life, according to a lawsuit filed in California Superior Court on Monday, the latest in a wave of litigation over Big Tech’s effects on young people.

该诉讼化名该女生为 Emma(埃玛),称她的父母最早于 3 月中旬得知她可以使用谷歌的 AI,当时学校发来信息警告她面临“迫在眉睫”的自残危险。诉状称,Emma 的母亲登录了女儿的学校谷歌账号,发现女儿正在谷歌文档上写遗书。她的父母在诉状中化名为 John 和 Jane Doe,立即带 Emma 进行了紧急精神评估。她现在每周都要接受治疗。

The lawsuit, which calls the girl Emma, said her parents first learned she had access to Google’s A.I. in mid-March after her school sent a message warning she was in “imminent danger” of self-harm. Emma’s mother logged into the teenager’s school Google account and found her daughter composing a suicide note on Google Docs, the complaint said. Her parents, called John and Jane Doe in the lawsuit, immediately took Emma for an emergency psychiatric evaluation. She now goes to therapy every week.

诉状称,北卡罗来纳州夏洛特市的一名七年级学生 Emma,在与谷歌搜索上一个名为 AI Mode(AI 模式)的类聊天机器人功能互动后,深信自己患有多种心理健康障碍。诉状还称,她曾搜索“什么是遗书”,并对 AI 说出“我在筹划我自己的葬礼”等言论。

Emma, a seventh grader in Charlotte, N.C., had grown convinced she had multiple mental health disorders following interactions with A.I. Mode, a chatbot-like feature on Google Search, the complaint said. She also searched for “what is a suicide note,” and made comments like “I am planning my own funeral” to the A.I., the complaint said.

这起诉讼很可能是首起指控谷歌“仓促整合” AI 功能进入 K-12(幼儿园至 12 年级)Chromebook 所用公司应用程序、却未通知家长潜在风险的诉讼。Chromebook 是由美国学区提供给数百万儿童的谷歌系笔记本电脑。这些指控加剧了公众的担忧:硅谷让年轻人沉迷于社交媒体应用、电子游戏和 AI 聊天机器人,使儿童陷入危险之中。

The lawsuit is likely the first to claim that Google “hastily integrated” A.I. features into company apps used on K-12 Chromebooks, Google-powered laptops provided to millions of children by U.S. school districts, while failing to notify parents of potential risks. The claims add to mounting public concern that Silicon Valley has hooked young people on social media apps, video games and A.I. chatbots, putting children in harm’s way.

谷歌发言人何塞·卡斯塔涅达表示,该公司与健康专家广泛合作,开发了帮助年轻人寻找紧急资源的功能。该公司正在审查诉讼中的指控。

José Castañeda, a Google spokesman, said the company had worked extensively with health experts to build features to help young people find emergency resources. The company is reviewing the claims in the lawsuit.

“多年来,当有人可能正在经历心理健康危机时,我们会在搜索中直接显示热线信息,”他在一份声明中表示。“我们已将同样的保护措施纳入搜索人工智能功能,将人们与现实世界的支持联系起来。”该案凸显了公立学区如何越来越依赖谷歌和微软等公司的课堂笔记本电脑、平板电脑和应用程序。近年来,一些大型科技公司利用其在学校的基础设施,向教师和学生推广其人工智能聊天机器人。

“For years, we’ve shown hotline information directly in search when someone may be experiencing a mental health crisis,” he said in a statement. “We’ve built those same protections into search A.I. features, connecting people with real-world support.”The case underscores how public school districts have come to depend on classroom laptops, tablets and apps from companies like Google and Microsoft. In recent years, some Big Tech companies have used their own infrastructure in schools to promote their A.I. chatbots for teachers and students.

新的诉讼认为,谷歌“庞大的数字生态系统”中的笔记本电脑和应用程序,使这家科技巨头能够将其Gemini人工智能聊天机器人推广到公立学校——以及默认开启一些学生的人工智能功能访问权限——而无需家长知情或许可。虽然学校管理人员可以为不同年级的学生开启或关闭YouTube和Gemini等公司应用程序,但学校管理人员抱怨称,谷歌的系统不允许他们禁用嵌入谷歌搜索中的人工智能模式和人工智能概览功能。

The new lawsuit argues that Google’s “sprawling digital ecosystem” of laptops and apps has enabled the tech giant to spread its Gemini A.I. chatbot across public schools — as well as turn on some students’ access to A.I. features by default — without parents’ awareness or permission. While school administrators can toggle company apps like YouTube and Gemini on or off for students in different grades, school administrators have complained that Google’s system has not allowed them to disable A.I. Mode and A.I. Overview, features embedded in Google Search.

本周,谷歌表示,计划于2027年初自动关闭K-12学校(幼儿园至12年级)中某些人工智能应用和功能,包括其Gemini AI工具和搜索功能。选择重新开启搜索功能的学校将面临一个选择:允许学生使用该公司已嵌入搜索中的AI功能;或使用一项旨在协助学生完成学业的新AI功能。一位公司高管表示,谷歌不向学校提供在搜索中关闭AI的选项,因为“搜索和AI深度融合”。

This week, Google said that it planned to automatically turn off certain A.I. apps and features — including its Gemini A.I. tool and search — in K-12 schools in early 2027. Schools that elect to turn search back on will have a choice: to enable student access to the A.I. features that the company has already embedded in search; or to a new A.I. feature intended to assist students with schoolwork. Google is not offering schools an option to turn off A.I. in search, a company executive said, because “search and A.I. are deeply integrated.”

今年,大型科技公司面临一波关于技术对年轻用户影响的诉讼。拥有Instagram和Facebook的Meta最近同意支付高达171亿美元,以和解47个州指控其利用成瘾性功能诱捕儿童的指控,此前Meta在类似案件中败诉。Meta否认有任何不当行为。几位家长还起诉AI公司,指控他们的孩子在与聊天机器人互动后自杀身亡。

Big Tech has faced a wave of litigation this year over technology’s effects on young users. Meta, which owns Instagram and Facebook, recently agreed to pay up to $17.1 billion to settle charges from 47 states that it used addictive features to ensnare children, following losses in similar cases. Meta has denied wrongdoing. Several parents have also sued A.I. companies over allegations their children died by suicide following interactions with chatbots.

周一代表Emma及其父母提起的诉讼提出了与近期针对Meta的一些诉讼相似的论点,声称Emma因谷歌的产品而受到伤害。

The lawsuit filed on Monday on behalf of Emma and her parents makes arguments similar to some recently used against Meta, claiming that Emma was harmed by Google’s products.

根据起诉书,Emma在家中接触笔记本电脑和应用程序的机会有限,最初并未尝试在学校发放的设备上使用谷歌的AI。起诉书称,当Emma搜索网页时,谷歌将她“引导”至其AI模式,这是Google搜索上的一项类聊天机器人功能。

Emma, who had limited access to laptops and apps at home, did not initially try to use Google’s A.I. on her school-issued device, according to the complaint. As Emma searched the web, the lawsuit claimed, Google “routed” her to its A.I. Mode, a chatbot-like feature on Google Search.

根据起诉书,到3月,Emma开始与谷歌的AI模式“交换数千条消息”,持续到深夜,“醒来后立即恢复”使用。起诉书称,该AI“诊断Emma患有严重精神疾病,说服她信任AI而非父母和咨询师,并最终帮助她策划了自杀和葬礼”。

By March Emma was “exchanging thousands of messages” with Google’s A.I. Mode until late into the night and “resumed as soon as she woke up,” according to the complaint. The A.I. “diagnosed Emma with major mental illnesses, persuaded her to trust it over her parents and counselors, and ultimately helped her plan her suicide and funeral,” the complaint said.

根据起诉书,谷歌的产品系统并未向学校或艾玛的父母发出警报,告知他们艾玛的搜索记录和人工智能聊天内容显示她有伤害自己的风险。她的学校使用了一家数字监控公司,该公司会自动扫描学生在校内设备上的写作和搜索内容,以寻找自残、网络霸凌及其他问题的迹象。

Google’s product system did not alert the school or Emma’s parents that her searches and A.I. chats suggested she was at risk of harming herself, according to the complaint. Her school used a digital monitoring company, which automatically scans students’ writing and searches on school devices for signs of self-harm, cyberbullying and other problems.

艾玛的父母在上个月的一次采访中表示,他们得知女儿处于危机之中,仅仅是因为这家监控公司标记了她在谷歌文档中撰写的一份自杀计划,并警告了学校。

In an interview last month, Emma’s parents said that they learned she was in crisis only after the monitoring company flagged a suicide plan she was writing in Google Docs and warned her school.

娜塔莎·辛格为《纽约时报》报道校园科技话题已逾十年。她目前关注的重点是家长对学校笔记本电脑和人工智能聊天机器人的担忧日益高涨。

Natasha Singer has covered tech in schools for The Times for more than decade. She is currently focused on the groundswell of parent concern about school laptops and A.I. chatbots.

阅读媒体原文 ↗