Skip to main content

OpenAI building new team to stop superintelligent AI going rogue

If the individuals who are at the very forefront of artificial intelligence technology are commenting about the potentially catastrophic effects of highly intelligent AI systems, then it’s probably wise to sit up and take notice.

Just a couple of months ago, Geoffrey Hinton, a man considered one of the “godfathers” of AI for his pioneering work in the field, said that the technology’s rapid pace of development meant that it was “not inconceivable” that superintelligent AI — considered as being superior to the human mind — could end up wiping out humanity.

And Sam Altman, CEO of OpenAI, the company behind the viral ChatGPT chatbot, had admitted to being “a little bit scared” about the potential effects of advanced AI systems on society.

Altman is so concerned that on Wednesday his company announced it’s setting up a new unit called Superalignment aimed at ensuring that superintelligent AI doesn’t end up causing chaos or something far worse.

“Superintelligence will be the most impactful technology humanity has ever invented, and could help us solve many of the world’s most important problems,” OpenAI said in a post introducing the new initiative. “But the vast power of superintelligence could also be very dangerous, and could lead to the disempowerment of humanity or even human extinction.”

OpenAI said that although superintelligent AI may seem like it’s a way off, it believes it could be developed by 2030. And it readily admits that at the current time, no system exists “for steering or controlling a potentially superintelligent AI, and preventing it from going rogue.”

To deal with the situation, OpenAI wants to build a “roughly human-level automated alignment researcher” that would perform safety checks on a superintelligent AI, adding that managing these risks will also require new institutions for governance and solving the problem of superintelligence alignment.

For Superalignment to have an effect, OpenAI needs to assemble a crack team of top machine learning researchers and engineers.

The company appears very frank about its effort, describing it as an “incredibly ambitious goal” while also admitting that it’s “not guaranteed to succeed.” But it adds that it’s “optimistic that a focused, concerted effort can solve this problem.”

New AI tools like OpenAI’s ChatGPT and Google’s Bard, among many others, are so revolutionary that experts are certain that even at this pre-superintelligence level, the workplace and wider society face fundamental changes in the near term.

It’s why governments around the world are scrambling to play catchup, hurriedly moving to impose regulations on the rapidly developing AI industry in a bid to ensure the technology is deployed in a safe and responsible manner. However, unless a single body is formed, each country will have its own views on how best to use the technology, meaning those regulations could vary widely and lead to markedly different outcomes. And it’s these different approaches that will make Superalignment’s goal all the harder to achieve.

Editors' Recommendations

Trevor Mogg
Contributing Editor
Not so many moons ago, Trevor moved from one tea-loving island nation that drives on the left (Britain) to another (Japan)…
Few people are using ChatGPT and other AI tools regularly, study suggests
ChatGPT app running on an iPhone.

Not a day seems to go by without generative-AI products like ChatGPT making the news, but few people are actually making regular use of the tools, a new study suggests.

The study was carried out by the Reuters Institute and Oxford University, and it involved 6,000 respondents from the U.S., U.K., France, Denmark, Japan, and Argentina. Researchers found that OpenAI's ChatGPT is by far the most widely used generative-AI tool and is two or three times more widespread than the next most widely used products -- Google Gemini and Microsoft Copilot.

Read more
OpenAI strikes major deal with News Corp to boost ChatGPT
A laptop screen shows the home page for ChatGPT, OpenAI's artificial intelligence chatbot.

OpenAI has struck a major deal with News Corp to access content to train its AI models, the companies announced on Wednesday.

The multiyear agreement is reportedly worth as much as $250 million and gives OpenAI access to content from News Corp's large stable of titles across several countries that include The Wall Street Journal, MarketWatch, The New York Post, The Times, The Sunday Times, The Sun, The Australian,, The Daily Telegraph, The Courier Mail, and the Herald Sun.

Read more
ChatGPT AI chatbot can now be used without an account
The ChatGPT website on a laptop's screen as the laptop sits on a counter in front of a black background.

ChatGPT, the AI-powered chatbot that went viral at the start of last year and kicked off a wave of interest in generative AI tools, no longer requires an account to use.

Its creator, OpenAI, launched a webpage on Monday that lets you begin a conversation with the chatbot without having to sign up or log in first.

Read more