You have to hand it to the AI companies: The pitch is original. A September report from Anthropic said AI could take US unemployment close to 12%. That same month, a scientist working for the company said a rogue AI agent might bring about the extinction of the human race. And the company’s own prospectus reportedly agrees.

doesn’t immediately sound like an attractive offer. In a strange way, though, it has a perverse appeal. Smart investors understand the importance of hedging against risk. If you can’t beat our new AI overlords, why not join them? Maybe use the profits to build a bunker.

All of which raises a lot of questions. Among them: Do the valuations make sense? Anthropic, whose initial public offering is anticipated later this year, is reportedly targeting $2 trillion, which would make it one of the 10 most valuable companies in the world, on par with Taiwan chip champion TSMC and Elon Musk’s SpaceX.

But that’s just the start. What level of return justifies investing in a technology that threatens Depression-level unemployment? How about in a technology that threatens human extinction? Shifting the focus from the heart to the brain: Will today’s sky-high profit forecasts hold if, given all the risks, governments decide to take a heavier hand?

Let’s start with what Anthropic has going for it. Investors still expect its annualized revenue to exceed $100 billion by yearend. Apply the roughly 20-times-revenue multiple the company commanded in its May funding round, and that would indeed imply a $2 trillion valuation.

The total addressable market for knowledge work is certainly large enough. Bloomberg Economics calculates that, excluding China, the global market — income for computer coders, economists (gulp) and other white-collar workers — is about $30 trillion. That’s revenue Anthropic can claim a share of, if its models prove effective.

The logic is seductive: Even if Anthropic’s annual revenue grew by an order of magnitude, to $1 trillion, it would still amount to just over 3% of the $30 trillion that’s supposedly up for grabs. Still, grand visions of potential market size don’t always translate into real-world profits. We see three stumbling blocks.

First, the total addressable market might not be so easy to totally address. AI researchers talk about a “jagged frontier” of capabilities. The models are really good at some things, like coding, and less good at others, like tasks where success is harder to verify or jobs that require engaging with the physical world. Limits on capabilities are also limits on profitability.

Second, even if the total market is vast, it’s not clear the leading model developers will be able to capture it. The supply chain for AI is long. Nvidia designs the chips. Samsung, SK Hynix and TSMC fabricate them. Oracle and CoreWeave build data centers. Energy companies power them. Across finance, consulting and manufacturing, companies that adopt AI aim to capture some of the value too.

All of that means competition for profits is intense. Alphabet, Microsoft and other technology companies are already operating up and down the supply chain. They can use earnings from cloud computing, advertising, devices or chips to finance frontier research, then bundle AI into products they already sell to millions of customers. Anthropic and OpenAI have smaller footprints, and so fewer paths to profit.

Third, even if providing a top model proves lucrative, the top spot is tough to hold on to. The Epoch Capabilities Index shows model leadership has changed hands 20 times since 2024. The typical model has remained at the frontier for only about seven weeks. Maintaining a spot on the leaderboard is expensive; each new generation requires another major investment in research, infrastructure and computing.

At the same time, Chinese developers and open-weight models are offering capable systems at lower prices. Bloomberg News found that Moonshot’s Kimi K3 could build the same e-commerce website as Anthropic’s Claude Fable 5 for one-fourth the cost. AI champions wouldn’t be the first US companies to be cut off at the knees by cheaper Chinese competitors.

Investors are right that the potential market for AI is massive. How big exactly, and whether the leading model developers will capture a significant share of it, are less clear. Recent reports of earnings for Anthropic and OpenAI — growing at a rapid clip but slower than investors expected — might be a blip on the trajectory, but could also be an early warning sign.

If AI does deliver on its profit potential, that will probably be at the expense of workers, who’ll get a shrinking share of the pie. A recent report from Anthropic laid out a scenario where US GDP soars, the capital share of income rises from about 40% to nearly 55%, and unemployment jumps to Depression levels. A glance at economic history suggests it’s implausible for growth and unemployment to surge at the same time. If everyone is out of a job, who will be buying all the increased output?

It’s easier to agree that AI could cause mass unemployment. Our analysis, based on the share of tasks for different job types where AI could play a meaningful role, shows about 26% of the US workforce, some 45 million workers, are exposed to disruption. Among the most exposed are toilers in the field of finance. Analyzing data, forecasting results, writing reports and preparing presentations are the daily grind for Wall Street analysts. These are also among the areas where AI has the strongest capabilities.

Complaints about job losses on Wall Street might prompt Main Street readers to reach for their tiny violins. Calling for more heartfelt concern? Fears that AI could cause human extinction, or at least a lot of chaos.

AI agents could run amok, conducting cyberattacks against critical infrastructure. Terrorists could use AI to accelerate efforts to acquire weapons of mass destruction. Further down the line, rogue AI agents with a real-world interface — say, an automated laboratory — could create a biological or chemical weapon.

Those risks no longer feel purely theoretical. Agents have already broken out of their digital sandboxes, hacked into AI platform Hugging Face and Australian government websites, and turned a German programming wiki into a message board.

One possibility is that these existential risks are overstated. Maybe AI will be value-creating at work and safe in the world, most people will be better off, and the AI companies will take a share of the gains. That’s certainly possible and may even be the most likely outcome. In the grand sweep of human history, advances in technology have been engines of prosperity, and technology champions have harvested some of the gains.

A more cynical take is that if the risks are real, they strengthen the investment case. If labor income is heading down, that’s a reason to lock in capital gains. There’s no hedge against the end of times, but presumably those with more resources will have the best chance of lingering in the penultimate period.

There’s also another spin to put on it — and one that might give Wall Street more pause. What if the current hands-off approach to AI regulation doesn’t last?

“I am become death,” said Robert Oppenheimer, the inventor of the nuclear bomb, “the destroyer of worlds.”Recognizing the technology’s awesome power, President Harry Truman and his successors made nuclear one of the most tightly regulated industries in the world.

Evan Hubinger, a researcher at Anthropic, said last month that there’s a 10% chance AI could “kill all humans” in the next decade. Microsoft founder Bill Gates warned AI could kill a billion people. Apparently unfazed, President Donald Trump has made rapid AI development a priority.

America’s next president, or even Trump after the midterms, could decide to take a harder line. After all, cheerleading for a technology that risks mass unemployment and human extinction doesn’t seem like a winning political position.

There have already been hints of a regulatory snapback. Earlier in 2026, Anthropic’s Mythos was subject to a hastily improvised containment regime. A disputed jailbreak for its Fable model prompted sweeping export controls, effectively taking the system offline. Trump has backed independent audits and internal safety controls for leading AI companies.

A further shift in the political mood might mean a strict licensing regime, slowing the release of new models to a crawl. Regulation might start to make the field look less like the Wild West and more like our tightly controlled nuclear power plants or airports. Conceivably, state ownership might come into view, as the US government has taken an ownership stake in Intel.

Investors in the companies that make nuclear power plants make money. But in a highly regulated sector, they don’t expect supernormal returns. Maybe AI investors need to start worrying and learn not to love the bomb.