随着决策模型在各行业广泛应用，一家名为Musubi的公司提出了一个新想法：将这些模型用于内容审核。周二，Musubi宣布推出一款名为PolicyLM-1.7B的轻量级决策模型，该模型专为实时内容审核设计，并以开放权重的形式发布。
As decision models spread across the industry, a company called Musubi has a new idea for how to put them to work: moderating content. On Tuesday, Musubi announced a lightweight decision model made for real-time moderation called PolicyLM-1.7B, released with open weights.
这一理念是将用普通英语编写的内容政策应用到消息中，且耗时仅为50毫秒。Musubi的模型旨在成本和速度上与目前大多数社交平台审核所使用的AI分类系统相当，但由于它具备现代大语言模型的灵活性，无需进行特殊训练即可应用复杂政策。更为重要的是，政策变更时该模型无需重新训练，从而允许政策制定者根据需要进行多次迭代。
The idea is to take a content policy written in plain English and apply it to messages in under 50 milliseconds. Musubi’s model is designed to be similar in cost and speed to the AI classifier systems that power moderation on most social platforms — but because it has the flexibility of a modern LLM, it can apply complex policies without special training. Even more important, the model won’t need new training when the policy changes, allowing for human policy-setters to iterate as much as they need.
Musubi联合创始人兼首席人工智能官Filip Jankovic表示，这为平台管理员提供了一种主动标记内容的方式。
As Musubi co-founder and chief AI officer Filip Jankovic sees it, it gives platform managers a way to label content proactively.
“产品团队只是想更好地了解他们平台上的情况，尤其是在内容数量呈指数增长的情况下,” Jankovic说。“能够以非常可扩展、可定制的方式标记所有这些内容非常有用。”自从Typesafe AI于9月份发布Jev以来，决策模型在人工智能领域引起广泛关注，随后OpenAI和Amazon也推出了竞争对手的决策模型。不同于输出文本，决策模型输出的是结果概率，不过在这种情况下，该模型输出的是二元判断：即内容属于该类别或不属于该类别。
“Product teams just want a better understanding of what’s happening on their platform, especially as the amount of content is exponentially increasing,” Jankovic says. “Being able to label all of that in a very scalable, customizable way is extremely useful.”Decision models have become a hot topic in the AI world since the release of Typesafe AI’s Jev in September, which was shortly followed by competing decision models from OpenAI and Amazon. Instead of outputting text, a decision model outputs outcome probabilities, though in this case the model outputs a binary judgement: either the content is in the category or it isn’t.
通过将模型的输出限制在一组预定的选项中，决策模型能够比大语言模型更快速、更经济地运行，同时仍然保持transformer架构的灵活性。
By limiting the model’s output to a set of predetermined choices, decision models are able to run faster and cheaper than large language models, while still maintaining the flexibility of the transformer architecture.
早期应用场景之一是控制AI代理的不当行为——因此将同样的技术应用于人类不当行为也是顺理成章的事情。
One early use case is reining in misbehavior by AI agents — so it’s only natural to apply the same technology to human misbehavior.
值得注意的是，詹科维奇表示，他对决策模型的兴趣要追溯到Jev之前，起源于一个名为GLiNER（通用命名实体识别模型）的2024年项目，该项目采用了许多相同的技术。
Notably, Jankovic says his interest in decision models predates Jev, tracing it back to a 2024 project called GLiNER (Generalist Model for Named Entity Recognition) that deployed many of the same techniques.
然而，Musubi并不对这一比较感到戒备。相反，该公司热衷于利用人们对决策模型的新兴兴趣来突显内容审核的重要性。产品公告上写道：“如果Jev引起了你的注意，那么PolicyLM-1.7B就是同类型的模型，它专门用于内容审核，并且你可以自己运行它。”
Still, Musubi isn’t wary of the comparison. If anything, the company is eager to use the new interest in decision models to shine a light on content moderation. “If Jev caught your eye, PolicyLM-1.7B is the same kind of model, trained specifically for content moderation, that you can run yourself,” the product announcement reads.