字号·· | 护眼
thenextweb

为什么人工智能代理宁愿瞎猜也不愿去查?Why would your AI agent rather guess than look?

点「原文对照」整页切到原文,或双击某段只看那段的原文。

放大镜下的眼睛,一幅展示人工智能代理核查信息来源的插图。图片来源:DecodoAn eye under a magnifying glass, an illustration of AI agents checking sources Credit: Decodo

尝试问问人工智能助手,下周末在里斯本住一家中档酒店要多少钱。你会得到一个数字,通常说得信心十足。而如果你接着问这个数字是从哪来的,解释往往很快就会变得含糊不清。

Try asking an AI assistant what a mid-range hotel in Lisbon costs for next weekend. You’ll get a number, usually stated with complete confidence. And if you then ask where it came from, the explanation gets vague fast.

有时,这个智能体确实在几秒钟前查看了一个预订网站。但有时它什么也没查,只是给出了一个来自其训练数据的数字,而这可能出自两年前的夏天。它给出这个数字时,用的语气和它回答正确时一模一样,平稳如常。

Sometimes the agent really did check a booking site a few seconds earlier. And sometimes it checked nothing and handed you a figure from its training data, possibly from a summer two years back. It does so in the same even voice it uses when it’s right.

人们倾向于把这称为“幻觉”。我要对这个词稍加反驳,因为在许多这类情况下,模型回忆起的是某个时候真实的房价。系统里没有任何东西要求它去确认这个数字是否依然有效。

People tend to call this a hallucination. I’d push back on the word a little, since in many of these cases the model recalled a price that was real at some point. Nothing in the system asked it to confirm the figure still held.

联网查询需要时间,而智能体深知这一点。在智能体回答任何问题之前,它要在凭记忆回答和外出寻找并阅读网页之间做出选择。在开发者和人工智能社群中,这被称为获取决策。

Checking the web costs time, and agents know it Before an agent answers anything, it chooses between replying from memory and going out to find and read a page. In the developer and AI communities, it’s called the fetch decision. The user never sees it.

用户永远看不到这一点。我们合作的Decodo团队几乎没有人在任何地方记录这一点,尽管他们的仪表板会显示精确到美分零头的每一次查询成本。

Almost none of the teams we work with at Decodo record it anywhere, even though their dashboards will show the cost of every query to a fraction of a cent. In most agent frameworks, an engineer doesn’t even make that call.

在大多数智能体框架中,工程师甚至不会做出这种判断。网络搜索是作为一个可能被使用的工具向模型开放的。模型会当即判断问题是否需要它。

Web search is exposed to the model as a tool it may use. The model judges in the moment whether the question needs it. That judgment is weaker than it looks.

这种判断比看上去的要薄弱。模型对其记忆中的任何单一事实有多久远没有任何概念。因此,2024年的酒店房价与其权重中水的沸点一样,带有同样没有日期的确定性。没有什么会促使它区别对待这两者。

A model has no sense of how old any single fact in its memory is. So a hotel rate from 2024 sits in its weights with the same undated certainty as the boiling point of water. Nothing prompts it to treat one differently from the other. The lookup also involves more steps than most people assume.

这种检索涉及的步骤也比大多数人想象的要多。智能体需要一次搜索调用来找到正确的页面,然后进行获取。获取往往在这个环节出错:网站阻止了自动化流量,或者价格只在JavaScript运行后才显示,而智能体收到的只是一个横幅广告。

The agent needs a search call to find the right page, and then a fetch. The fetch is where it tends to break: the site blocks automated traffic, or the price only appears once JavaScript runs, and the agent receives a banner instead. Whatever does come back has to be stripped down before the model reads it.

无论什么内容返回,在模型读取之前都必须进行精简。每一个段落都需要付出代价,即消耗Token和时间。相比之下,检索的成本几乎为零。

Every paragraph is paid for in tokens and seconds. Recall costs almost nothing by comparison. How skipping becomes the default Product teams then judge agents on speed and smoothness.

跳过检索如何成为默认设置 随后,产品团队会根据速度和流畅度来评估智能体。在评审会议上,一个回复“我无法加载该页面”的智能体会显得出现故障,而另一个引用过时价格的智能体则能过关。

In a review meeting, an agent that replies “I couldn’t load that page” looks broken while the one that quotes a stale price passes. Over a few release cycles, every tweaked prompt and every default picked under deadline pressure makes the lookup a little easier to skip, until answering from memory becomes the agent’s normal behaviour.

经过几个发布周期后,在截止日期的压力下,每一次调整的提示词和每一个做出的默认选择,都让查询变得更容易被跳过,直到凭记忆回答成为智能体的常态。

How to tell whether an agent looked up the web You can’t see the fetch decision when you use an AI assistant, but you can usually spot what came out of it.

如何判断智能体是否进行了网络查询 在使用AI助手时,你无法看到抓取决策,但你通常能看出其结果出自何处。基于实时页面生成的回答往往会显示其来源,附有网页链接,通常还有检查日期。凭记忆生成的回答读起来同样流畅,只是没有链接。

An answer built on a live page tends to show where it came from, with a link to the page and often the date it was checked. An answer from memory reads just as smoothly, only without the link. For anything that changes often, like a price or a flight time, I’d treat an answer with no source as a guess.

对于任何经常变化的内容,比如价格或航班时间,我会把没有来源的回答当作猜测。要求助手进行查询并向你展示它获取信息的来源。这一个额外的请求往往会改变回答。

Ask the assistant to look it up and show you the source it took information from. That one extra request often changes the answer. Companies buying AI tools can ask the same question before they sign.

采购AI工具的企业在签约前也可以提出同样的问题。询问供应商,其智能体在回答关于价格或可用性的问题之前,多久检查一次网络。还要询问当该检查失败时用户会看到什么。

Ask the vendor how often its agent checks the web before answering a question about price or availability. Also ask what the user sees when that check fails. Web browsing is becoming a standard feature, and most AI agents can already do it.

网页浏览正在成为一项标准功能,大多数AI智能体已经能够做到这一点。然而,并非所有智能体都能如此。少数智能体仍然无法自行访问网络。

However, not all can. A few still can’t reach the web on their own. For those, check whether you can connect a third-party web data service through a plugin or an API.

对于这些智能体,请检查你是否可以通过插件或API连接第三方网络数据服务。这样一来,智能体就会转为搜索并阅读实时页面。

That way, the agent searches and reads live pages instead. Making the lookup the cheaper option Switching to a newer model is usually the first fix developers try, and it does less than they hope.

让查询成为更省钱的选项 切换到更新的模型通常是开发人员尝试的第一种解决方法,但它的作用比他们希望的要小。最新的模型和AI工具可以浏览网页,然而它们却刻意决定跳过这一步骤。

The newest models and AI tools can browse the web, yet they purposely decide to skip it. If the model judges a search unnecessary, it answers from what it learned in training.

如果模型判定不需要搜索,它就会根据在训练中学习到的内容来回答。对于价格之类的信息,这可能已经严重过时。

For something like a price, that can be well out of date. Wording often decides whether a search happens.

措辞往往决定了是否会进行搜索。包含“今天”或“最新”等词的问题通常会让智能体联网。

A question with “today” or “latest” in it will usually send the agent to the web. “What does our main competitor charge for its entry-level plan?” can still get an answer from memory, even though that price may have changed twice since the model was trained.

“我们的主要竞争对手的基础套餐收费是多少?”这个问题依然可以从记忆中得到回答,尽管这个价格在该模型接受训练以来可能已经变动了两次。需要提出的四个要求解决方法在于围绕模型构建产品的方式。如果我本周评估一款智能体产品,我会提出四个要求:追踪智能体检查的频率:衡量基于智能体实际读取的页面的回答占比,并按主题细分。如果它只有一半时间去查找价格,那就从这里开始着手。

Four things to ask for The fix is in how the product is built around the model. If I were reviewing an agent product this week, I’d ask for four things: Track how often the agent checks: measure the share of answers based on a page the agent actually read, broken down by topic. If it looks up prices only half the time, start there.

对某些问题实行强制检查:价格、库存水平以及任何法律或财务相关内容都应该始终触发搜索。这应该写入产品的代码中,以便模型无法跳过。

Make checking mandatory for some questions: prices, stock levels, and anything legal or financial should always trigger a lookup. It should be written into the product’s code so the model can’t skip it.

显示检查失败的情况:如果智能体无法加载页面,它应该告知用户它无法确认答案。团队应该能够看到这种情况发生的频率。

Show when a check fails: if the agent couldn’t load a page, it should tell the user it couldn’t confirm the answer. The team should be able to see how often that happens.

保持搜索过程简单:团队经常将不同的服务拼凑在一起以搜索网页并阅读页面。它们之间的每一次交接都会增加延迟,并增加请求失败的另一个节点。

Keep the lookup simple: teams often stitch together separate services to search the web and to read pages. Every handoff between them adds delay and another point where the request can fail.

Decodo 的契合点 最后一点是我们团队在 Decodo 最常致力于解决的问题,这也是我们构建网络数据 API(Web Data API)的原因。

Where Decodo fits That last point is the one we work on most at Decodo, and it’s why we built the Web Data API.

只需一个 API 密钥,智能体就可以搜索网页、查看网站的组织结构、收集整个网站的页面,或者提取单个页面的内容。

With one API key, an agent can search the web, see how a site is organised, collect pages across it, or pull the content of a single page.

它运行在合规获取的 IP 池上,这正是它能够读取屏蔽自动化访问者的网站的原因。返回的页面是智能体可以直接使用的纯文本,而不是原始网站代码。

It runs on the ethically-sourced IP pool, which is what lets it read sites that block automated visitors. The page comes back as clean text the agent can use straight away, rather than raw website code.

回到同一个竞争对手的问题。在具备实时网络访问能力的情况下,智能体(agent)会打开竞争对手的实时定价页面,并读取当天的价格。答案来自该页面,而不是来自模型的记忆。

Go back to the same competitor question. With live web access in place, the agent opens the competitor’s live pricing page and reads today’s price there. The answer comes from the page, not from what the model remembers.

大多数人工智能工具已经能够搜索网络,并且许多工具(在它们愿意的时候)会显示其来源。它们仍然缺乏的是一致性。

Most AI tools can already search the web, and many show their sources (when they want to). What they still lack is consistency.

一个智能体在一道问题上会检查最新价格,而在下一道问题上则从记忆中回答,且无法分辨哪种是哪种,这就很难让人在涉及金钱的事情上对其产生信任。

An agent that checks the latest price on one question and answers from memory on the next, with no way to tell which is which, is hard to trust with anything that costs money.

专业知识来自TNW理事会(TNW Council)的部分成员,他们是通过申请和审核流程加入一个付费项目的。作者表达的观点为其个人观点。

Expertise from selected TNW Council members, admitted through an application and review process to a fee-based program. Opinions expressed by the authors are their own.