Are tech companies moving too fast in rolling out powerful artificial intelligence technology that could one day outsmart humans?

That’s the conclusion of a group of prominent computer scientists and other tech industry notables such as Elon Musk and Apple co-founder Steve Wozniak who are calling for a 6-month pause to consider the risks.

Their petition published Wednesday is a response to San Francisco startup OpenAI’s recent release of GPT-4, a more advanced successor to its widely-used AI chatbot ChatGPT that helped spark a race among tech giants Microsoft and Google to unveil similar applications.

The letter warns that AI systems with “human-competitive intelligence can pose profound risks to society and humanity” — from flooding the internet with disinformation and automating away jobs to more catastrophic future risks out of the realms of science fiction.

It says “recent months have seen AI labs locked in an out-of-control race to develop and deploy ever more powerful digital minds that no one – not even their creators – can understand, predict, or reliably control.”

“We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4,” the letter says. “This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium.”

A number of governments are already working to regulate high-risk AI tools. The United Kingdom released a paper Wednesday outlining its approach, which it said “will avoid heavy-handed legislation which could stifle innovation.” Lawmakers in the 27-nation European Union have been negotiating passage of sweeping AI rules.

The petition was organised by the nonprofit Future of Life Institute, which says confirmed signatories include the Turing Award-winning AI pioneer Yoshua Bengio and other leading AI researchers such as Stuart Russell and Gary Marcus. Others who joined include Wozniak, former US presidential candidate Andrew Yang and Rachel Bronson, president of the Bulletin of the Atomic Scientists, a science-oriented advocacy group known for its warnings against humanity-ending nuclear war.

Musk, who runs Tesla, Twitter and SpaceX and was an OpenAI co-founder and early investor, has long expressed concerns about AI’s existential risks. A more surprising inclusion is Emad Mostaque, CEO of Stability AI, maker of the AI image generator Stable Diffusion that partners with Amazon and competes with OpenAI’s similar generator known as DALL-E.

OpenAI, Microsoft and Google didn’t respond to requests for comment Wednesday, but the letter already has plenty of sceptics.

“A pause is a good idea, but the letter is vague and doesn’t take the regulatory problems seriously,” says James Grimmelmann, a Cornell University professor of digital and information law. “It is also deeply hypocritical for Elon Musk to sign on given how hard Tesla has fought against accountability for the defective AI in its self-driving cars.”

While the letter raises the spectre of nefarious AI far more intelligent than what actually exists, it’s not “superhuman” AI that some who signed on are worried about. While impressive, a tool such as ChatGPT is simply a text generator that makes predictions about what words would answer the prompt it was given based on what it’s learned from ingesting huge troves of written works.

Gary Marcus, a New York University professor emeritus who signed the letter, said in a blog post that he disagrees with others who are worried about the near-term prospect of intelligent machines so smart they can self-improve themselves beyond humanity’s control. What he’s more worried about is “mediocre AI” that’s widely deployed, including by criminals or terrorists to trick people or spread dangerous misinformation.

“Current technology already poses enormous risks that we are ill-prepared for,” Marcus wrote. “With future technology, things could well get worse.”