Such things are not without precedent. In the early days of this century, the so-called dotcom bubble saw enormous investment in websites and internet platforms that were perceived to be the future of everything from pet grooming to holiday booking. In the end, the vast majority of those investments failed while only a few major companies – like Amazon and Google – turned the promise of e-commerce into global success stories.

The result, toward the tail end of Bill Clinton’s presidency and into the early Bush years, was a severe market correction. And that investment was as nothing compared to the scale of the present AI bubble.

Second Problem: What if AI Is, Wait for It ... Bad?

Bailey’s second concern, if anything, is more terrifying than the first, and in his own letter appears to be the more immediate financial-stability risk.

Simply put, he notes that the rapid advance of AI and the power now available to some AI models have the potential to empower genuinely threatening and cheap-to-launch cyberattacks. Last month, news broke of a major incident at OpenAI in which models being used during internal cybersecurity evaluations escaped their intended controls, gained internet access and compromised parts of Hugging Face’s systems.

In a typically understated reference to the incident, Bailey says: “Recent developments have also highlighted to me that many jurisdictions do not have the protocols in place to manage the development, release, and deployment of advanced frontier AI models, heightening risks for the financial sector and beyond.” Translated into plain English, he means that the G20 countries he is writing to are vulnerable to a sustained attack on their markets. This might manifest as cyber-theft or something much worse.

Doom and Despair

Taken together, Bailey’s warnings are stark: a great many investors have been seduced by the notion of AI as the next great frontier in human economic advancement, and trillions of dollars have been committed to what is, in essence, a bet that this advancement will deliver profits and prosperity.

Yet in reality, that advancement could just as easily deliver doom and despair.

That AI is capable of replacing much human labor is already evident to anybody who uses it professionally. Several AI models are already more than capable of taking on the role traditionally carried out by a personal assistant. In law, AI is already capable of basic drafting, contract review and other tasks traditionally given to trainees or junior associates. Even in media, AI is probably a more consistent sub-editor than most humans, and unlikely to miss a stray Oxford comma here or there.

Replacing human labor may deliver profits to AI companies and savings to their customers. But the replaced human beings do not simply vanish. They still have to be funded and financed through the welfare system, or other jobs found for them. It is also not readily clear, to put it mildly, that replacing vast swathes of human labor is socially or politically desirable, or that the consequences of doing so are being considered by policymakers.

It is also possible that the ultimate promise of AI is dramatically overstated because of the very paradox that the Hugging Face incident exposes: AI might well have terrifyingly efficient capabilities if developed to the max – but developing it to the max might mean creating a system that humans cannot control and from which supposed “advancements” might in fact be existentially dangerous. Should the public or governments reach that conclusion, then development might well end up stalled well before the “revolutionary” threshold is hit.

Horses Fall. All the Time.

That these risks are relatively obvious has not yet deterred investors from piling fortunes into their bet on AI. It is, to use an analogy, rather like betting all of one’s money on a heavily favored horse in a steeplechase: the odds are it will win or place, but there are fifteen fences to jump first, and horses fall all the time.

This is what Bailey can see: not merely that the horse might fall, but that the sheer amount of money being piled on that horse at the bookies will leave an awful lot of people without homes or assets should it meet its doom.

For governments, the problem is this: in a free market, they cannot direct people not to invest in AI or companies promising to own the future. They cannot restrict the flows of capital. They can only be prepared for what happens when thousands of investors come back from the racecourse, announcing that they have lost everything.

If AI fails, the bubble may burst. If AI succeeds, the consequences may be no less destabilizing.

It is, one might conclude, a good era in which to be a pessimist.