Better Red Than Killed by Robots

With sincere apologies to the ghost of Robert Frost, that is what the AI Futures Project is hoping will be the response to the two reports it has released on the future of AI.

The AI Futures Project is a nonprofit research group that does not want to call itself a think tank. It is led by Daniel Kokotajlo, an AI safety researcher who left OpenAI and gave up a $2m payday to maintain his freedom to speak freely about how AI is evolving.

In the first report, AI 2027, Kokotajlo’s group envisioned a future in which the “impact of superhuman AI over the next decade [is] enormous, exceeding that of the Industrial Revolution”. That would, of course, mean great upheaval.

The report foresaw the race to build data centers coming in the early part of this year. It predicted that the People’s Republic of China’s General Secretary Xi Jinping would “finally commit fully to the big AI push he had previously tried to avoid” in mid-2026. Xi did that in the middle of July.

Next Prediction: Job Losses

Having correctly forecast those two developments adds greater relevance to the group's prediction that AI will begin replacing some human jobs later this year in a process that only accelerates over time.

Early next year, according to the group, China steals the latest cutting-edge AI and replicates it. The US government responds by sanctioning cyber attacks on the PRC and getting more serious about rooting out spies. The AI Cold War arms race is on.

There are branching options for the reader. In one scenario, the US and China agree to slow things down, so they can figure out what dangers AI presents without militaristic pressure forcing them into grave errors.

In the other scenario, late next year an AI is created that is effectively conscious. The year after that, AIs in both the US and China are integrated with the two countries’ actual weapons-making systems. In the next decade, robots take over the world economy and exterminate humanity in a sneak attack.

Not Plan B

That report was published on 3 April 2025. Plan A was published this July and bears the subtitle AI 2040. It outlines what the group believes should happen instead.

In this new, somewhat rosier scenario, “humanity delays the development of superintelligence until 2040, makes all AI research public, allows dozens of companies globally to catch up to the frontier, and intentionally enters a regime of mutually assured compute destruction”.

With Plan A, Congress makes some halting attempts to regulate the development of AI now, and in 2028 AI is on the presidential ballot. Competition with the PRC looms over it.

Readers are invited to choose from many paths. Yet the only two with any depth to them are Plan A, a verified slowdown, and Plan B, fight China.

Having been primed by the previous report to see Plan B as the path to humanity's destruction, many readers will opt for Plan A.

The candidate who finds himself in the White House after the 2028 tilt says the following: “This mad race toward superintelligence must end. For too long, we have been pursuing lesser-evils and least-bad solutions. We need a Plan A. We can still proceed with AI development, but we must do it more cautiously, more transparently, and involving many more countries and companies.”

How that Works: In Theory

There is a temporary halt to AI training while the US and China establish a verification regime. Each side is allowed to inspect the other's data centers to verify its computing capacity, while governments around the world agree to slow the development of AI.

When things restart in 2030 or so, there are four guiding principles to AI development: (1) buy time; (2) total research transparency; (3) diffuse AI broadly; and (4) reversibility.

By 2031, the world is seeing significant AI developments. At the same time, using the precautionary principle frequently employed by European regulators, governments “require companies to write up detailed arguments for why their new AIs won’t cause an irreversible catastrophe”.

The group gets deep into the imagined details, including how to replace the federal government’s current income tax dependency with more of a robot-permitting tax base. It not only funds the government at necessary levels but also finances a universal basic income for displaced workers.

As for the mutually assured compute destruction plan, the US and China agreed to build “most of the new datacenters ... in third-party countries – especially Canada and Mongolia – to ensure their vulnerability in case the deal breaks down”. If the peace fails, troops of the respective countries would destroy their own centers to keep them from falling into enemy hands.

Does AI Have Limits?

It may be hard for non-experts, or indeed even experts, to assess the plausibility of any of this multidisciplinary forecasting. Much of it comes down to the question of what is intelligence.

If intelligence is simply processing power configured the right way, then many scenarios involving artificial general intelligence (AGI) or, in the AI Futures Project's preferred term, superintelligence either will or will not come true. Yet they are at least distantly possible.

Other researchers argue that in spite of all the improvements in large language models, there are both categorical limits to what AI can achieve and trade-offs involving a great deal of electricity use that will curb AI’s expansion.

Some clever AI entrepreneurs such as Elon Musk are aiming to get around this limitation by powering data centers in space. If Musk and his colleagues pull that off, the networked satellites will give AI all the power it needs to grow rapidly. Moreover, the devices will do so from a vantage point that is difficult for ground-based regulators to enforce.

Neither of these developments are ideal from the AI Futures Project’s perspective. Pity they did not see it coming.