"we’re confronted with abilities that could end up killing us all.”
Does artificial intelligence really pose a greater risk to humanity than nuclear weapons?
Warnings were issued in the UK and US this week over the possibility that increasingly capable AI systems could eventually surpass human capabilities.
Former UK defence secretary Des Browne said a superintelligent AI could pose a threat “on the same, possibly, a greater scale” than nuclear weapons.
Meanwhile, Professor Stuart Russell, a computer scientist at the University of California, Berkeley, warned there could be a “Chernobyl-sized catastrophe” if humanity loses control of advanced AI.
The warnings came as MPs and peers at Westminster considered the potential risks of artificial superintelligence (ASI), a form of AI that would surpass human capabilities across a broad range of tasks.
The session was convened by Control AI, a lobbying group calling for international regulation of advanced AI. It is supporting a bill tabled by Labour MP Alex Sobel that seeks to ban the creation of ASI.
Beatrice Fihn, who won the 2017 Nobel Peace Prize for leading the International Campaign to Abolish Nuclear Weapons, told parliamentarians:
“It is not the first time we’re confronted with abilities that could end up killing us all.”
Concerns escalated when Anthropic alignment science lead Evan Hubinger said he believed there was a greater than 10% chance AI could “kill all humans” in the next decade.
Hubinger also warned that Anthropic did not have a plan to ensure ASI was aligned so that it would not cause harm.
His comments followed the resignation of Jacob Coxon, a 28-year-old researcher at Anthropic who had previously worked at OpenAI.
Coxon claimed that “neither company was acting responsibly” and said they were “gambling with our lives”.
He said the risks associated with advanced AI were understood at Anthropic, but claimed the companies were “locked in a race to get there first”.
Coxon added that he remained optimistic about the possibility of US AI laboratories coordinating to slow the pace of development.
Predictions about when ASI could be achieved vary significantly, ranging from several years to more than a decade.
The push for greater international coordination was also central to discussions at Westminster.
Labour MP Darren Jones has written to Andy Burnham and the heads of the United Nations and OECD calling for a “multinational treaty for the regulated and safe development of superintelligence – not a ban on innovation or scientific endeavour but a safety-first approach to the rapid development of this technology”.
Citing Coxon’s resignation, Jones added: “The debate ranges from the end of humanity to claims of ‘marketing hype’. Either way, governments must now step in.”
In the US, independent senator Bernie Sanders has also renewed his calls for Congress to regulate AI.
Sanders pointed to polling suggesting that 81% of Americans believe politicians should take action over the technology.
The Vermont senator said: “We can’t allow a handful of greedy people to play God and determine the future of humanity – our economy, environment, democracy, privacy and more – without public input.”
Control AI is funded by Jaan Tallinn, the co-founder of Skype, who describes himself as an “anti-extinctionist”.
Tallinn has invested in Anthropic and Google DeepMind while also campaigning for greater AI safety.
He has claimed some senior AI executives would accept the possibility of humanity being wiped out and estimated that 10-15% of AI employees believe AI could become a worthy successor to humanity.
Tallinn previously said: “A fairly known AI researcher said to me ‘Jaan, don’t worry about this. Humans are a disposable species’.
“From what I understand he was fine with becoming extinct.”
The Westminster session heard contrasting views about the severity of the risks posed by advanced AI.
Russell, who issued the Chernobyl warning, said: “The other possibility is a much larger catastrophe, in which humanity loses control irreversibly. But we have no say over whether we continue to exist.”
Browne said nuclear war had once appeared to him to be the most likely threat facing humanity when he was defence secretary.
He now believed “a superintelligent AI poses a threat on the same, possibly, a greater scale”.
However, other experts questioned whether such scenarios were realistic given the current capabilities of AI.
Dr Andrew Rogoyski, of the Surrey Institute for People-Centred AI, said:
“In reality, these systems are nowhere near as versatile as humans, let alone humans acting collectively.
“I suspect we’re heading towards ‘the great disappointment’ where advanced AI turns out to be too expensive and not useful enough to continue in its current form.”
David Barber, director of Sofair, a state-backed AI research lab warned against rejecting the technology because of potential risks:
“AI is not going to go away. It’s incredibly useful, whether or not you allow it in a fully unconstrained way to access the internet and various systems that’s potentially problematic.
“We may need to learn how to better control these things.
“There are vulnerabilities in the software frameworks that need to be patched. But that’s doable.”
“What we need as a country is to get to grips with the duality that is both an incredibly important and useful technology, and at the same time, it’s something that needs to be carefully thought about and carefully controlled.”
Sandra Wachter, a professor at the Oxford Internet Institute, said she did not believe in “Terminator scenarios”.
However, she warned that AI already posed significant risks through its environmental impact, the spread of misinformation and potential job displacement.
Gary Marcus, an AI industry commentator and academic, also distinguished between different forms of superintelligence:
“There is a difference between superintelligence that is aligned (if such a thing is possible) and superintelligence that is not.
“It is at least conceivable that the former might be net positive.
“So far we have neither, but a superabundance of hype combined with a striking lack of prudence on OpenAI’s part has gotten us where we are, with intense mistrust all around.”
