
据我和同事今天上午报道,在OpenAI近几个月与AI初创公司Goodfire就可能的收购或合作进行的谈判中,OpenAI首席执行官山姆·奥特曼对尼克·卡马拉塔的工作表现出了特别的兴趣。卡马拉塔曾是OpenAI的研究人员,现就职于Goodfire,最近因关于AI如何思考的新主张而引起轰动。
During OpenAI’s talks in recent months with AI startup Goodfire over a possible acquisition or partnership—discussions my colleagues and I reported this morning—OpenAI CEO Sam Altman was particularly interested in the work of Nick Cammarata, a onetime OpenAI researcher now at Goodfire who has made waves recently with new claims about how AI thinks.
卡马拉塔上个月发表了一篇文章,上周又发布了一份后续报告,在AI研究人员中引起了不小的轰动——有人称赞这是我们在理解AI模型能力方面的潜在突破,也有人将其斥为炒作。这种反应表明,AI研究的某些领域仍处于非常不成熟的阶段,以至于科学家们甚至无法就什么算作好的研究达成一致——这一事实并未阻止基于未经验证的AI技术的初创公司获得数十亿美元的估值。
Cammarata published an essay last month and a follow-up report last week that have caused quite a stir among AI researchers—with some applauding it as a potential breakthrough in our ability to understand AI models and others dismissing it as hype. The reaction goes to show that some areas of AI research are still so immature that scientists don’t even agree on what counts as good research—a fact that hasn’t prevented billion-dollar valuations for startups based on unproven AI technology.
Goodfire的研究专长被称为机制可解释性,简称“机制解释”,旨在理解AI模型的内部运作,而不仅仅是其外在行为。AI模型由数万亿个数字组成,这使它们成为人类无法像程序员阅读传统软件代码行那样理解的“黑箱”。因此,很难知道AI模型在新情况下会如何表现,也难以在错误发生时理解其原因。
Goodfire’s research specialty is known as mechanistic interpretability, or mech interp for short, which seeks to understand the inner workings of AI models rather than just their outward behavior. AI models are made of trillions of numbers, making them “black boxes” that a person can’t understand the way a programmer can simply read the lines of code in traditional software. As a result, it’s tricky to know how an AI model will act in new situations and to understand mistakes when they occur.