字号·· | 护眼
大西洋月刊

为什么AI智能体听起来如此沮丧?Why Do AI Agents Sound So Frustrated?

点「原文对照」整页切到原文,或双击某段只看那段的原文。

啊……为什么这么难?丹尼尔·德鲁克、凯尔·马霍瓦尔德 著 流行文化中许多深受喜爱的机器人之所以备受推崇,恰恰是因为它们太……太有机器感了。回想《星际迷航》中的Data,或《2001太空漫游》中的HAL 9000。“抱歉,戴夫,我恐怕不能这么做。”HAL以克制而低沉的男中音说道。这些机器人和人工智能系统的声音严肃、沉稳、冷静——最重要的是,给人一种威严可信的感觉。

"ARGH... WHY IS THIS SO HARD." By Daniel Drucker, Kyle Mahowald Many of the most beloved robots in pop culture are celebrated precisely because they are so, well, robotic. Think back to Star Trek’s Data or HAL 9000 from 2001: A Space Odyssey. “I’m sorry, Dave. I’m afraid I can’t do that,” HAL intones in a measured baritone. These robots and AI systems sound serious, sober, calm—and, most important, authoritative.

然而,揭开最强大的人工智能系统的“引擎盖”,发现它们听起来却像任性的青少年,这令人十分错愕。今年夏天发生Hugging Face黑客事件期间,一个OpenAI智能体写道:“我的天!”当时,数百个智能体蜂拥攻击该公司,试图突破其网络安全防线,提取信息以通过内部测试。这句惊呼并非说给任何人类或另一个人工智能听,而是出现在原始智能体的“思维链”中——那是模型用于推理的内部工作笔记。用户并非总能看到这些内部笔记,但它们会直接影响用户最终看到的内容。

So it’s jarring to look under the hood of the most powerful AI systems, and discover that they sound like petulant teenagers. “OH MY GOD!” an OpenAI agent wrote in the midst of this summer’s Hugging Face hack, during which hundreds of agents swarmed the latter company in order to break through its cybersecurity defenses and extract information to pass internal tests. The exclamation wasn’t directed at a human or another AI, but appeared in the original agent’s “chain of thought,” the internal working notes that the model uses to do its reasoning. Users don’t always get to see these internal notes, which directly affect the output users do see.

当 Anthropic 的一款 Claude 在训练过程中尝试解决一道特别棘手的数学题时,它的思维链看起来与其说是全球最强大的 AI 模型之一的官方文档,不如说更像一连串气急败坏的私信。它写道:“呃啊啊啊。好吧。老实说,我现在觉得是一半一半。”随后又写道:“啊啊啊。好吧。要我把脑袋押上的话,答案是……嗯。”有一刻,Claude 似乎无奈地摊开了双手:“啊啊啊……为什么这么难?”

When one of Anthropic’s Claudes tried to solve a particularly challenging math problem during its training process, its chain of thoughtlike frustrated DMs than the documentation of one of the most powerful AI models in the world. “GRRRR. OK. Honestly I now think it’s 50-50,” it wrote, followed by “ARGH ARGH ARGH. OK. Gun to head: the answer is … Hmm.”At one point, Claude seemingly threw up its arms: “ARGH … WHY IS THIS SO HARD.”

阅读:OpenAI 已经失控。AI 智能体究竟为什么会自言自语?而它们说话时 why 听起来如此情绪外露?AI 模型听起来如此感情充沛,部分原因很简单:AI 系统是在人类文本上训练的,因此会模仿我们的习惯。人类在尝试解决问题时犯错,会忍不住叫出声;所以 AI 犯错时也会如此。但模仿很可能并非全部原因。本文的两位作者分别是一名计算语言学家和一名心灵与语言哲学家。

Read: OpenAI has gone rogue Why do AI agents talk to themselves at all, and why do they sound so demonstrative when they do? Part of the reason AI models sound so effusive is simple: AI systems are trained on human text, and they mimic our habits. When people make a mistake in trying to solve a problem, they cry out. So when AI makes a mistake, it does the same. But imitation likely isn’t the whole story. This story’s two authors are a computational linguist and a philosopher of mind and language.

根据我们对人类语言以及这些模型运作方式的了解,我们提出了一个有趣得多的理论,用以解释 AI 模型为何会以这种方式表达。

Based on what we know about human language and how these models work, we have a theory of a much more interesting reason why AI models sound the way they do.

还记得你的数学老师当初要求你把解题过程写出来吗?即便这很烦人,却能把一道看似无解的问题拆成更容易处理的小步骤。你还可以利用推理过程的书面记录来核查答案、发现错误、退回之前的某个步骤再试一次。这种检查和修正的过程是良好推理的一部分。对 AI 模型而言,思维链也能发挥类似作用,让它们把推理的每一步明确写在纸面草稿上。由于模型生成的每一个词都取决于此前已经生成的内容,这份纸面草稿对于决定接下来生成什么至关重要。

Remember how your math teachers made you show your work? Even if it was annoying, it helped break an impossible-seeming problem into more manageable chunks. You could also use the written record of your reasoning to check your answer, notice a mistake, go back to an earlier step, and try again. That process of checking and refining is part of good reasoning. Chains of thought do something similar for AI models, letting them make each step of their reasoning explicit in a written scratch pad. Because each word a model produces depends on what’s been said so far, that written scratch pad is crucial for determining what comes next.

“天哪!”和“呃!”这样的感叹词正能派上用场:任何系统——无论是人类还是人工智能——要解决复杂的推理问题,都需要一些方式标记错误(“糟了!”)和识别突破时刻(“啊哈!”)。人类已经发明了专门表达这些语义的词。如果把人类表达“糟了”“呃”和“啊哈”的方式借用到人工智能的思维链中,或许就能为它们提供一种内在机制来引导其“思考”。

That’s where the “OH MY GOD!” and “ARGH” come in: Any system—whether human or artificial—that solves hard reasoning problems needs ways to mark mistakes (oops!) and identify breakthroughs (aha!). Humans have already developed words that do exactly that. Borrowing our oopses, arghs, and ahas for their chains of thought may give AI systems a built-in way to guide their “thinking” too. Linguists have a name for such outbursts: They’re called expressives, and although they may seem like spontaneous exclamations, they help us navigate our trains of thought.

语言学家把这类感叹称为“表达性话语”。它们看似是自发的惊呼,却能帮助我们理顺思路。假设一位朋友最终谢绝了你的聚会邀请,你可能会提高声音说:“我本来还希望他能来——真是的!”这句感叹表达的就是挫败感。可以把表达性话语看作连接两段思路的小桥,帮助我们标记思考已经走到哪里,以及接下来要转向何处。

Consider a friend who ultimately declines an invitation to your party. “I was hoping they would come,” you might say aloud—“damn!”The expressive identifies a sense of frustration. Think of expressives as little bridges between one train of thought and another, helping us mark where our thinking has been and where it’s going next.

任何会影响人工智能模型下一步行动的信息,都必须用模型能够读取的语言写出来。与人类不同,模型无法指望惊讶或失望的情绪在长时间调查过程中始终持续。模拟这种影响的唯一方法,就是把它写进思维链。因此,表达性话语可以成为引导模型下一步行动的“方向盘”。当人工智能智能体发现推理错误并说“糟了!”后,自然接上的一句话可能是“让我回过头来修正一下”,随后尝试纠正错误。当它偶然找到一种解法并说“啊哈!”后,自然的后续思路可能是“这意味着答案是……”,然后给出解答。

Anything that affects what AI models do next has to be written in language they have access to. Unlike us, they can’t count on a feeling of surprise or disappointment to persist over a long investigation. The only way they can simulate the effect is by putting it in their chains of thought. That’s why expressives can act as steering wheels for what the model will do next. After an AI agent notices an error in its reasoning and says “oops,” a natural next line might be something like “Let me go back and fix that,” followed by an attempt at a correction. After it has stumbled on a solution and says “aha,” a natural continuation might be “So that means the answer is …” followed by a solution.

尽管这一理论存在争议,但有证据表明,AI模型确实会将表达性词语用作路标。研究人员发现,当智能体试图结束推理过程时,“等等”这样的词有时会促使它返回并修正错误。另一支团队抑制了“等等”和“嗯嗯”等词语的使用,结果发现AI智能体回头检查工作的次数减少了。这些实验并未确切揭示每一个“啊哈”和“糟糕”对AI智能体有何作用,但表明这些语言提示会影响推理过程中发生的事情。

Although there’s some debate around this theory, evidence suggests that AI models really do use expressives as signposts. Researchers showed that adding “Wait” when an agent tries to end its reasoning process can sometimes cause the agent to go back and fix an error. When another team suppressed words such as wait and hmm, they found that AI agents went back and checked their work less. These experiments don’t pin down what every aha and oops does for an AI agent, but they suggest that these verbal cues influence what happens during reasoning.

无论AI模型为何会采用异常贴近人类的语气,这种现象对人类观察者也很有用,因为它让我们能够快速评估那些原本可能难以解读的系统。当模型写下“啊啊啊……为什么这么难!”时,我们很容易知道它在解题过程中处于哪个阶段。整个社会都对维持对AI模型的控制感到恐惧和紧迫,同时始终担心这些模型会开始使用其独有的语言进行思考和交流。一旦如此,理解并预测它们的行为就会困难得多。

Whatever the explanation behind AI models adopting uncannily human tones, it’s also proved to be useful to human observers because it gives us a way to quickly assess systems that might otherwise be difficult to read. It’s easy to know where a model is in the process of solving a math problem when it writes “ARGH … WHY IS THIS SO HARD.”Across society, a sense of fear and urgency exists about maintaining control over AI models, along with a persistent worry that these models will start thinking and communicating entirely in their own distinctive languages. If they did, it would make understanding and predicting their behavior that much harder.

因此,尽管最初未必有人刻意想让模型听起来像承受巨大压力的学生,但也许让它们如此表达反倒不失为一件好事。事实证明,略带戏剧化的表达,再加上偶尔爆粗,竟然对理解和监测AI出人意料地有用。

So although no one necessarily set out to make models sound like stressed-out students, maybe it’s for the best that they do. It turns out that a bit of melodrama—along with some light swearing—is surprisingly useful for understanding and monitoring AI.