编程语言的泛型如今可以借助 Mojo 核心贡献者埃文·奥瓦迪亚的实验性工作,在不同语言的编译器之间优雅往返。这位前谷歌工程师一直致力于将 Rust 打造成一门其他语言可以据以构建的库,并展示了一种在 Rust 与另一种语言之间传递泛型数据类型和函数的方法,而 C ABI 并不支持这种操作。“从技术角度来说,这是我几个月来读到的最引人入胜的东西。”Let’s Get Rusty YouTube 频道的主持人在介绍这项工作时兴奋地说道,并将其称为“Rust++”。这无疑是一次极客但来之不易的胜利。
Programming language generics can now ride in style from one language’s compiler to another and back again, thanks to experimental work from Mojo core contributor Evan Ovadia. This former Google engineer has endeavored to turn Rust into a library that other languages can build from, demonstrating a way to pass generic data types and functions between Rust and another language, an action the C ABI doesn’t support. “Technically, this is the most fascinating thing I’ve read in months,” enthused the host of the Let’s Get Rusty YouTube channel, explaining the work, which he characterized as “Rust++.”That's a geeky but hard-won victory to be sure.
奥瓦迪亚在一篇解释性博客文章中将自己的工作描述为连接两种不同语言的“黄金道钉”,借此指代美国第一条横贯大陆铁路线完工时最后钉入的那根道钉。库不必再被剥离成原始基元,只为塞进 C ABI 这辆粗陋的货车;如今,功能丰富的库可以沿着坚固的高速铁路,驶入陌生的生态系统。终于甩开 C ABI这项工作源于奥瓦迪亚打造自己语言 Valen 的尝试(Valen 源自他早先名为 Vale 的项目);他设想将 Valen 打造为速度更快、对开发者更友好的 Rust 版本。
Ovadia describes his work in an explanatory blog post as the “Golden Spike” that intertwines two different languages, referring to the final spike hammered into the United States’ first transcontinental railroad line. Instead of being stripped down into raw primitives just to be crammed into the coarse wagons of C ABI, libraries of rich capability can be sped to foreign ecosystems on sturdy, high-speed rails. Ditching the C ABI at last The work came about in Ovadia’s effort to create his own language, Valen (an offshoot of an earlier effort called Vale), which he envisions as a speedier, more dev-friendly version of Rust.
他推断,从零创造一门新语言的问题在于,他无法使用自己喜爱的 Rust 库(尤其是 Rust 那款棒极了的 wgpu 图形库)。
The problem with creating a new language from scratch, he reasoned, was that he couldn’t use his favorite Rust libraries (notably Rust’s bitchin’ wgpu graphics library).
如今,在用另一种语言编写的程序中使用某种语言库的一种常见方式,是通过 C ABI(应用程序二进制接口)进行调用。C ABI 是一种事实上的互操作层,规定了编译代码之间传递数据和调用函数的约定。(Rust 有自己的 ABI,但并不提供稳定性保证。)使用 C ABI 时,如果一个 Rust 库将其函数公开为 extern "C",并将其类型公开为 repr(C),那么其他语言就可以通过外部函数接口(FFI)访问它们。
Today, a common way to use a library from one language in a program written in another is through the C ABI (application binary interface), a de facto interoperability layer that defines conventions for passing data and function calls between compiled code. (Rust has its own ABI, though it does not offer stability guarantees.) With the C ABI, if a Rust library exposes its functions as extern "C" and exposes its types as repr(C), then they can be accessed from another language using a Foreign Function Interface (FFI).
一切都没问题,但 Ovadia 面临的问题是,C ABI 不理解泛型(即日后再提供的数据类型占位符)这一概念,因为 C 语言本身就不理解泛型。不过,Rust 大量使用泛型,Valen 也使用泛型,只是方式有所不同。Ovadia 写道:“因此,核心问题,也就是有待解开的核心谜团,是如何实现跨语言泛型?”跨语言泛型终于成为现实。
All well and good, but the problem Ovadia had was that the C ABI did not understand the concept of generics (or placeholders for data types to be provided later), because C itself doesn’t understand the concept of generics. But Rust uses generics quite a bit, and so does Valen, though not in the same way. “So the central question, the central mystery to solve, is how do we do cross-language generics?”Ovadia wrote.
让两个编译器协同工作极其棘手,因为编译器往往是相互隔离的实体,是各自独立世界中孤独的王者。因此,Ovadia 需要找到一种方法,让 Valen 能够调用 Rust 泛型函数,同时也能让 Rust 泛型函数回调 Valen,以获取所需的值。经过长时间深思熟虑,Ovadia 提出了一个分为两部分的跨语言泛型解决方案:第一,将 Rust 编译器作为库运行。
Cross-language generics at last Getting two compilers to collaborate is super tricky, as compilers tend to be isolated entities, solitary kings of their own individual worlds. So Ovadia needed to come up with a way for Valen to call a Rust generic function, but also have a Rust generic function that calls back into Valen for needed values. After much rumination, Ovadia arrived at a two-part solution to cross-language generics: first, run the Rust compiler as a library.
令人惊讶的是,Rust 已经具备这种能力(rustc_driver),Valen 可以借此将 rustc 直接嵌入自身的内存空间。第二步是修改 Rust,使其在遇到非 Rust 的 Valen 类型时,将控制权移交给 Valen 编译器,由后者处理相应部分,并等待编译结果。Ovadia 承认,修改 Rust 编译器是“核选项”。他写道:“说实话,这是整个项目中风险最高的部分。”
Surprisingly, Rust already possesses this capability (rustc_driver), which would allow Valen to embed rustc directly into its own memory space. The second step would be to patch Rust so that when it encounters non-Rust Valen types, it hands control over to the Valen compiler to handle its portion, and waits for the compiled result. Ovadia admits that patching the Rust compiler was the “nuclear option.”“Honestly, this was the riskiest part of the whole endeavor,” he wrote.
这个补丁本身只有大约 100 行代码,但它采用了取巧的做法,并假定 Rust 使用 LLVM 生成机器码,而情况并非总是如此。Ovadia 写道:“它确实能用,但绝对无法合并到上游。”尽管如此,Ovadia 实现的跨语言双向泛型,仍是编程语言互操作性领域的实质性进步。当前,Ovadia 全身心投入 Valen 的开发,不过他的跨语言泛型工作也开辟了新的可能:借助 Rust,为其他能够基于 Rust 库构建的新语言提供自举支持。
The patch itself is only about 100 lines of code, but it is hacky and assumes Rust is using LLVM for machine code generation, which isn’t always the case. “It works, but it's definitely not upstreamable,” Ovadia wrote. Nonetheless, Ovadia’s cross-language bidirectional generics are a genuine step forward in programming language interoperability. Ovadia has his hands full with Valen at this point, though his cross-language generics work opens the possibility of using Rust to bootstrap other new languages that can be built from Rust libraries.
只要对 Rust 本身稍作修改,对老旧不堪的 C ABI 的依赖或许有朝一日会成为历史。在 Reddit 和 Lobste.rs 程序员论坛上,观察者们一方面赞赏 Ovadia 着手攻克如此困难的问题,另一方面又对具体细节感到担忧。一位评论者写道:“我欣赏将 C 作为‘通用 FFI 语言’予以取代的举措。这是个非常大胆的选择,而且需要付出大量努力。”
Given a few modifications to Rust itself, the reliance on the creaky C ABI may one day be a thing of the past. On the Reddit and Lobste.rs programmer forums, observers praised Ovadia for tackling such a difficult problem, even as they fretted about the details. “I appreciate the move to replace C as the ‘universal FFI language,’” one commenter wrote. "It is a very bold choice and would require effort.”
另一位读者评论称,鉴于 Rust 自身类型系统的局限,它可能并不是实现互操作性的理想语言。不过,他们写道:“想象一下,如果你能在 Ada 之上构建绑定!那才叫惊艳。”®
Another reader commented that Rust may not be the ideal language for interoperability, given the limitations of its own type system. “However, imagine you could build bindings on top of Ada! Now that would be amazing,” they wrote. ®