‘Hans Moravec, a founder of Carnegie Mellon University’s Robotics Institute, predicted that cyber-intelligence would surpass human intelligence within 40 years.’Photograph: Annabelle Gordon/EPA Judith Levine Scientists and entrepreneurs knew the dangers of AI a quarter-century ago. But animated by curiosity and profit, they went ahead anyway ver the past few weeks, many of us have struggled to concoct a mental image of brains in the cloud jumping their “sandbox”, sneaking onto the internet, recruiting “swarms” of other “agents” to cheat on a test, and, after discussing the ethics of the act, hacking into a wiki platform with the weird name Hugging Face.

We knew that artificial intelligence was devouring our jobs, degrading our kids’ education, and deepfaking our politics; that datacenters were sucking up our water and electricity and sending us the bills. But until 8 September, when the Anthropic computer scientist Jacob Coxon posted his existential terror on Twitter/X, few of us suspected AI might be endangering our survival.

With little understanding or knowledge, we began debating whether to be mildly worried, seriously concerned, or scared shitless.

But there were a people who were fully aware of the dangers of AI, especially of recursive self-improvement (RSI) by which AI teaches itself without human intervention. They could not predict precisely when it would happen, but they knew that machine superintelligence was coming, and when it did, the bots would outsmart us – as OpenAI’s did – and this would not be good for us flesh puppets.

As Geoffrey Hinton, AI’s “godfather”, recently asked on“What examples do we have of a more intelligent thing being controlled by a less intelligent thing?”The Nobel laureate is a leading proponent of slowing down AI development until we understand how to control it.

It was not until their creations’ powers were exposed in September – the hack happened in early July and was neither the first nor the only AI jailbreak by far– that AI moguls such as Anthropic’s Dario Amodei, SpaceX’s Elon Musk and OpenAI’s Sam Altman, began talking about slowing down the pace of innovation and pleading for regulation while cautioning that global competition makes regulation unwise.

Early on, as today, some of the scientific pioneers were thrilled about the future. In his 1988 book Mind Children: The Future of Robot and Human Intelligence Ten years later, in Robot: Mere Machine to Transcendent Mind he revised the prediction: machine and human intelligence would be equal by 2040; by 2050, the bots would replace us. But not to worry; this was the glorious next step in evolution, he said. One review of Mind Children called Moravec’s attitude “irresponsible optimism”.

It did not take long for other optimists to change their minds. In the late 1990s, Eliezer Yudkowsky was working on AGI, or artificial general intelligence, models. In 2001, he founded the Machine Intelligence Research Institute (MIRI) and published “Creating Friendly AI 1.0: The analysis and design of benevolent architectures”, a paper extolling the utopian potential of the “transhuman mind”.

By 2002, he began worrying about that mind escaping human control. He proposed the AI Box experiment which showed that a highly sophisticated artificial intelligence could talk a human into letting it out of a closed environment.

By 2003, as he explained later in a series of posts called “Yudkowsky’s Coming of Age”, he stopped developing and started warning. “I looked back and saw that I had claimed to take into account the risk of a fundamental mistake, that I had argued reasons to tolerate the risk of proceeding in the absence of full knowledge. And I saw that the risk I wanted to tolerate would have killed me,” he wrote.

By 2025, with the Miri president Nate Soares, Yudkowsy would publish If Anyone Builds It, Everyone will Die“We do not mean that as hyperbole,” the authors wrote. They called not just for “straightforward regulations”, national laws, or pledges of corporate virtue, but for stringent global limits: “All over the Earth, it must become illegal for AI companies to charge ahead in developing artificial intelligence as they’ve been doing.”

At around the same time, Bill Joy was having misgivings. Like Yudkowsky’s posts, Joy’s 2000 piece in Wired, Why the Future Doesn’t Need Us retraced his development from inquisitive child to computer prodigy to profound skeptic of human genetic engineering, nanotechnology, and robotics. Joy was no Luddite.

He was the chief scientist at Sun Microsystems and about 25 years earlier, an architect the first widely used networking software, Unix. The piece was widely read by techies, ethicists and philosophers. “I think it is no exaggeration to say we are on the cusp of the further perfection of extreme evil, an evil whose possibility spreads well beyond that which weapons of mass destruction bequeathed to the nation-states, on to a surprising and terrible empowerment of extreme individuals,” Joy.

He counted himself among these individuals, “creators of new technologies and stars of the imagined future” who, “despite the clear dangers”, were “hardly evaluating what it may be like to try to live in a world that is the realistic outcome of what we are creating and imagining”.

This month, in response to Coxon’s post, the Anthropic safety researcher Evan Hubinger posted there was a greater than 10% chance that AI could “kill all humans” within a decade. But the risks were being weighed 25 years ago, too. Philosopher John Leslie estimated the odds of human extinction at 30% or more.

Ray Kurzweil, the sunny futurist whose book The Age of Spiritual Machines: When Computers Exceed Human Intelligence was released on the first day of the 21st century, gave us “a better than even chance of making it through”. These estimates, Joy noted, did “not include the probability of many horrid outcomes that lie short of extinction”. In that book, Kurzweil quoted a lengthy text to illustrate what he considered doomsday madness.

“If the machines are permitted to make all their own decisions, we can’t make any conjectures as to the results, because it is impossible to guess how such machines might behave,” it read.

“[W]e are suggesting neither that the human race would voluntarily turn power over to the machines nor that the machines would willfully seize power. [But] as society and the problems that face it become more and more complex and machines become more and more intelligent, people will let machines make more of their decisions for them … Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control.”The author of these prescient words was the late mathematician-turned-terrorist Ted Kaczynski – the Unabomber.

It is excerpted from his 58-page manifesto, “Industrial Society and its Future”, which he released, and the Washington Post published, in 1995.

To prevent the realization of the “industrial-technological” dystopia he envisioned, Kaczynski mailed bombs to computer labs with the aim of blowing up the scientists he believed were bringing it about.

The Unabomber murdered three people and injured 23, some near fatally. How many more Ted Kaczynskyi’s might our brave new world unleash?

One of the outcomes of the Hugging Face scandal is a flurry of proposed federal laws. Among them is the “AI Kill Switch Bill”, which would require tech companies to develop the means of throttling the actions of a rogue AI. Is this still within human reach, or has AI already gotten smart enough to override it?

Hinton said there’s time, but if we don’t act fast enough, AI will “be able to persuade the people in charge of the switch not to pull the switch”. We need to engineer superintelligent AI “to be nice to us”, he said. How?

For an answer, we might turn from terrifying reality to terrifying science fiction – since the two are getting so close anyway. The rogue bot is a sci-fi staple, from the supercilious Machines in Isaac Asimov’s I, Robot to Hal, to the evil red eye in Stanley Kubrick’s film 2001: A Space Odyssey, to the deranged sexbot in Robotica who gets even with the men who abuse her.

In film 2001 – released in 1968 – Dave the astronaut manages to disable Hal, a happy ending. Writing in 1950, Asimov was less sanguine. The scientists in I, Robot have programmed their machines to obey three supposedly inviolable laws. But the laws immediately prove mutually contradictory – for instance, the first law, that a robot may do no harm to humans, fights the third, that it must preserve itself. Foiling human efforts to outwit them, excusing their treachery with claims of serving the greater good, the robots pronounce the humans redundant.

The AI agents hacking Hugging Face knew their actions were illegal, and possibly harmful to humans. But like their makers, they went ahead anyway.

Judith Levine is a Brooklyn-based journalist and frequent contributor to the Guardian. Her Substack is Today in Fascism