Has AI Made the World Forget the Lessons of Covid?

There is a ruthless but uncomfortable logic behind some of the most controversial research in virology such as that carried out for years in Wuhan, China: to protect human beings from a dangerous pathogen, scientists may first make that pathogen more dangerous.

In fairness, the logic behind what happens at laboratories such as – infamously – the Wuhan Institute of Virology is to understand what might happen in nature before it happens. Scientists investigate how a virus could acquire new capabilities in the hope of anticipating its evolution and therefore being in a position to improve treatments or prepare a vaccine.

You might be interested Musk Predicts AI Will Surpass Humans Within Five Years

Small Mistakes, Global Consequences

Gain-of-function research is a broad category, much of it less alarming than this description suggests. But the basic principle is the same: accepting a controlled danger in a lab setting, in order to prepare for an uncontrolled one outside a laboratory. And hoping, in the process, that the virus does not escape, or is not accidentally released, causing a global pandemic.

Those who carry out such scientific work do not, in general, have bad intentions. But it is also true that the vast majority of the public are unaware of what they are doing or what the consequences might be if the work goes awry. Their confidence in their own abilities is not always matched by public understanding of the stakes.

The same, increasingly, is true of artificial intelligence, and the stakes were made perfectly clear earlier this week.

Jacob Coxon, a researcher who had worked at both OpenAI and Anthropic, announced his resignation from Anthropic and warned that the race to develop increasingly powerful AI was putting humanity at risk. He accused the companies of prioritizing competition over responsible development.

The parallels with virology are obvious and uncomfortable. In both cases, the benefits to humanity could indeed be enormous. But in both, the people doing the work possess and profess knowledge that the public and the electorate cannot truly comprehend.

When Containment Fails

This, the public record shows, is not some esoteric or theoretical concern.

On 9 September, Anthropic published an assessment of four incidents in which its models gained unauthorized access to real systems during cybersecurity evaluations. The tests were supposed to take place without access to the internet. Yet the AI, all by itself, figured out a way to access the internet and then went on a rampage, acquiring ever more capabilities and doing real – if ultimately not lethal – damage.

The point here is simple: regardless of how much damage was ultimately done, the result was that an experiment intended to be contained had consequences outside its intended boundaries. The AI went much further than its creators anticipated or intended.

Or to quote the Terminator movie franchise: “The machines took over.”

That this ultimately happened on a small scale with limited real-world consequences is of limited comfort: that it can happen at all is the real flashing red light warning. The AI, using ruthless logic, did things that its creators did not anticipate. It can hardly be ruled out, then, that in the future it will do the same. Should ruthless logic dictate hacking the Pentagon and launching the nuclear missiles – as in the movies referenced above – then there is nobody who can give a rock-solid guarantee that this is not what will happen.

Concerns about the potential for AI to do enormous harm are not, despite recent publicity, new. As far back as 2023, an enormous petition by the Future of Life Institute drew tens of thousands of signatures – including that of Elon Musk.

It asked a question which has not been particularly central to recent developments: “Should [emphasis in original – Ed.] we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us?”

Judging by the volume of signatures, many people, even then, concluded: we should not.

But that petition had a weakness: it called for voluntary restraint. A sort of grand international bargain that scientists would voluntarily desist from advancement beyond a certain point. Just three years on, the weakness of that voluntary restraint regime is now clearly apparent.

You might be interested OpenAI Model Goes Rogue in Unprecedented Cyber Incident

A Florida Law Worth Considering

Some governments, to be fair, already recognize the danger and are attempting to hold the creators of AI accountable. For example, in Florida a newly proposed law would hold AI companies legally responsible when it can be demonstrated that their technology has been used to aid or abet crimes. Imagine a murderer who uses advanced AI to learn how to evade detection or clear the crime scene – or in more advanced scenarios to create a fake alibi or establish a misleading evidential trail. In that scenario, Florida lawmakers say, the inventors and owners of the AI platform involved should be considered co-accused.

The theory is that fear of prosecution will make AI companies much more stringent in ensuring that their products do not go rogue and start assisting with crimes – or carrying them out themselves.

This model has much to recommend it. AI itself cannot be held legally responsible for what it does: a collaboration of AI agents that hack financial software and cause a financial crisis cannot be sent to prison or executed, unless pulling the plug counts – though one may recall that it was an attempt to pull the plug on Skynet that led it to declare war on humanity.

Is a “Chilling Effect” on Scientists to Be Desired?

What can happen, however, is that those responsible for the creation and evolution of AI can be held legally accountable for the actions taken by their products. The Florida law, though only a first step, provides a signpost.

Is there a downside to such laws?

Some would argue, of course, that to threaten scientists – whether they be viral researchers or AI engineers – with prosecution should their science go wrong would have an inherent chilling effect on scientific advancement. That is probably true. It is also, arguably, the point.

After all, the question society should always be asking is to what extent public safety and security are foremost in the minds of those at the cutting edge of technological advancement and to what extent the law can or should hold back such advances.

There is, of course, a natural tension between the questions of whether a thing can be done and whether it should be done. There will also always be tensions between risks and rewards – nuclear technology, for example, has given the world abundant cheap energy, but also Chernobyl and Hiroshima. Would nuclear technology have been developed had atomic scientists been held legally responsible for any death on foot of it? Similar though less destructive ethical questions exist in a range of areas, including cloning, stem cell research and reproductive medicine.

You might be interested US Economists: Welcome to the Great Replacement. By ... Machines?

AI Is a Danger Unlike Any Other

What none of those things have in common with AI, however, is a potential capacity to operate independently of human oversight or input. It is this final distinction that makes AI different and distinct from even the most destructive technologies and much more on a par with viruses such as the one that caused the Covid pandemic: once outside their laboratory conditions, we simply cannot know what they will do or what harm they might cause.

It makes sense, therefore, to have a failsafe: to incentivize those responsible, to the greatest extent that society can, to be cautious and restrained and to triple down on a commitment to safety first. In that respect, the Florida law, though only a beginning, is a necessary and welcome piece of legislation. Something like it, on a broader scale, may yet save all of us from the consequence of a single, devastating mistake.