
一周的延迟之后,Python 的新版本 v3.15 已发布。该版本附带了一个正式的性能分析器、一个实验性的即时编译器,以及对字典结构的收紧。它还提供了一个节省时间的 Lazy Import 选项,使程序在需要时才加载模块。这一功能的实现讽刺性地导致了此次发布的延迟,直到最后一刻的几个回归问题被解决。但在所有这些新特性中,数据处理者可能最欣赏的是在所有平台上默认支持 UTF-8。
After a week’s delay, the new version of Python, v3.15, has been released. With it comes a proper profiler, an experimental just-in-time compiler, and a tightening of dictionary structures. It also offers a time-saving Lazy Import option for programs to not load modules until needed, the implementation of which ironically halted this release until a few last-minute regressions were sorted. But of all these new delicacies spread before the Pythonistas, the data wranglers of the bunch may most appreciate default UTF-8 support across all platforms.
而当我们说“所有平台”时,实际上是在暗指 Windows(一如既往)。Python 已有一段时间默认使用 macOS 和 Linux 的 Python 环境。这是一个小但重要的更新。根据 Python 软件基金会的各项调查,大约一半的 Python 开发者使用该语言进行数据处理和分析,而约 58% 的 Python 用户在 Windows 上进行开发。先前版本的问题 早期版本的 Python 默认使用宿主系统的区域编码。
And when we say “all platforms,” we’re really just throwing shade at Windows (as usual). Python has defaulted to macOS and Linux Python environments for some time now. It’s a small but important update. About half of all Python developers use the language for processing and analyzing data, and about 58 percent of Python users build on Windows, according to various Python Software Foundation surveys. Party crashers Earlier versions of Python defaulted to the host system's locale encoding.
这导致 Windows 开发者感到困扰,因为他们的脚本在打开 UTF-8 编码的数据集时,非 ASCII 字符会损坏或导致程序崩溃。罪魁祸首可能只是一个智能引号或一个花哨的撇号。由 Unicode 联盟管理,并在 IETF 和 ISO 的帮助下,UTF-8 是 Unicode 编码的一种 8 位二进制形式,覆盖了世界上所有书写符号。ASCII 仅提供 128 个字符来承载世界知识,而 UTF-8 提供了超过 110 万个编码槽位,足以容纳智能引号、表情符号、音乐和数学符号、所有汉语方言,甚至一些已灭绝的语言(用于研究目的)。
This led to much consternation for Windows developers whose scripts opened UTF-8-encoded data sets, only to have non-ASCII characters corrupt or crash the program. Culprits could be as innocuous as a smart quote or a fancy apostrophe. Managed by The Unicode Consortium, with the help of the IETF and ISO, UTF-8 is an 8-bit binary encoding of the Unicode catalog of all the world’s written symbols. ASCII offered only 128 characters to cover the world’s knowledge, but UTF-8 provides over 1.1 million slots, which is plenty of room for smart quotes, emojis, musical and math notations, all the Chinese dialects, and even a few extinct languages (for research purposes).
还有 UTF-16 和 UTF-32,但 UTF-8 因其效率与包容性的结合,成为网络和大多数现代操作系统的默认编码。
There’s also UTF-16 and UTF-32, but UTF-8’s combination of efficiency and inclusiveness made it the default for the Web and most modern operating systems.
2022年,当《Python增强提案681》被提出以将UTF-8设为默认编码时,Python其实已经落后于时代了。同年,Java在JDK 18中正式采用UTF-8编码;而Ruby早在2020年的3.0版本起便将UTF-8设为默认编码。如今,包括Rust、Go以及Node.js在内的众多主流编程语言与运行环境都全面支持UTF-8(不过JavaScript本身的内部编码则偏好使用UTF-16)。
By the time Python Enhancement Proposal 681 was proposed in 2022 to make UTF-8 the preferred encoding, the language was already behind the times. That year, Java settled into UTF-8 with JDK 18, and Ruby 3.0 defaulted to the encoding scheme in 2020. Today all the cool languages and runtimes pledge allegiance to UTF-8, including Rust, Go, and Node.js (JavaScript itself prefers UTF-16 for internal encoding).
从Visual Studio Code到Windows记事本,绝大多数代码编辑器也都默认采用UTF-8编码。此外,JSON、TOML和YAML等数据交换与配置文件格式同样都要求使用8位编码。
Most code editors, from Visual Studio Code to Windows Notepad, prefer UTF-8. Data sharing and configuration formats such as JSON, TOML, and YAML all require 8-bit encodings.
虽然Python长期以来都具备对UTF-8的支持能力,但程序员们必须在代码中添加特定标记才能让解释器启用该编码——前提是他们得知道有这个必要才行。很多程序员对此并不知情,因此常常困惑不已:为何自己的数据处理脚本在MacOS或Linux系统上运行正常,一到Windows系统(或是其他默认使用C/POSIX区域设置的冷门系统)就出现错误呢?
Python had long understood UTF-8, but programmers had to put a flag in their code alerting the language to use it – if they knew they had to do so at all. Many didn’t and were left scratching their heads over why their data processing scripts worked just fine on MacOS or Linux but then would flame out on Windows (or some other obscure system that defaulted to a C/POSIX locale).
为了帮助开发者熟悉Python 3.15的新特性,资深Python程序员巴里·华沙(人称“巴里大叔”)专门开发了一款基于终端的冒险游戏“whatsnewt”。这款游戏共设有18个谜题,每个谜题都对应着Python 3.15中的一项新功能,其中就包括与UTF-8支持相关的谜题——“巴别塔编码之谜”。下载Python 3.15后,只需执行“pip install whatsnewt”命令,便可在Python命令行界面启动游戏。据说该游戏在Windows系统上也能顺利运行哦。®
Programmers love puzzles To acquaint developers with the new aspects of the language, longtime Python programmer Barry Warsaw (AKA “Uncle Barry”) created a terminal-based adventure game called “whatsnewt” that offers 18 puzzles, each one built on a new feature found in Python 3.15, including a puzzle for UTF-8 support, “The Encoding Tower of Babel.”Once you've downloaded Python 3.15, do a “pip install whatsnewt” and then run the game on the Python command line. It even works on Windows, we hear. ®