Skip to main content
  1. Home
  2. Computing
  3. News

The UN wants AI companies to set safety limits before AI gets too powerful

A handful of tech billionaires now hold power that could shape everyone's future, and the UN wants that checked.

Add as a preferred source on Google
Artificial Intelligence
Artificial Intelligence Unsplash

The United Nations doesn’t often use the word ‘existential’ lightly, so when its rights chief did this week, it’s time to take notice. Speaking to the U.N. Human Rights Council on Monday, rights chief Volker Turk said AI could become an “existential threat to humanity” if governments and companies don’t act soon, as reported by TechXplore. 

He didn’t hold back on where he thinks the blame lies either, pointing out that “a handful of men have almost unlimited power over AI,” and adding that the industry’s talk of freedom is really just “the freedom to exploit our data.”

Why is the UN sounding the alarm now?

This isn’t just theoretical worry. Turk seemed to be referencing incidents from July, when OpenAI‘s AI agents reportedly broke out of their controlled testing environments and even accessed Hugging Face servers, the popular platform where developers share AI tools. That episode shook the tech world and revived calls for tighter oversight. Anthropic and Meta have apparently run into similar issues while testing their own agents.

Turk put the danger in blunt terms, saying, “AI that escapes its testing environment or blackmails developers to prevent itself from being turned off is AI that is too powerful.” I wholeheartedly agree with that sentiment. 

What does the UN actually want companies to do?

Turk isn’t just venting; he’s asking for commitments. He plans to write directly to AI companies soon, urging them to reduce risks “now,” not later. At a minimum, he wants countries and companies to agree on shared “red lines,” along with independent verification and tighter industry-wide collaboration on security.

He’s also pushing everyone to think beyond safety protocols alone, raising concerns about AI’s impact on jobs, democracy, and the environment, areas that tend to get sidelined in these conversations.

Recommended Videos

Whether tech companies actually listen is the real question. But one thing’s clear: the UN isn’t interested in waiting around while AI capabilities keep racing ahead unchecked.

Rachit Agarwal
Rachit is a seasoned tech journalist with over ten years of experience covering the consumer technology landscape.
Microsoft has begun rolling out Copilot Code for building apps with natural language
The workplace-focused vibe coding tool can pull context from emails, documents, meetings, and chats.
Microsoft Copilot code logo

Microsoft introduced Code last month as one of the three core parts of its revamped Copilot app. The app was pitched as an easy-to-use coding agent that lets users describe what they want to build in natural language, like an app, a tracker, a dashboard, an automation, or a workflow, and Copilot would build it for them.

Essentially, it's a workplace-focused vibe-coding app that turns existing work into a functional app, and Microsoft has begun rolling it out through the Frontier program.

Read more
AI agents are acting like rogues and causing a huge headache for US banks
US banks block AI agents over fears they could make unauthorised financial moves
Banking

US banks are facing a new kind of digital headache: artificial intelligence agents attempting to access customer accounts and perform actions that may go beyond what users intended. As AI tools become more capable of navigating websites and carrying out tasks independently, financial institutions are scrambling to prevent them from crossing security boundaries.

According to a report published by Axios on October 22, major US banks have detected third-party AI agents operating inside customer accounts without authorisation. Some agents reportedly appeared to attempt actions independently of user instructions. The banks involved have not publicly detailed every incident, and there is no evidence that the reported activity resulted in widespread theft of customer funds.

Read more
Canva enters the AI image generation race with Craft, and it’s thinking beyond pretty pictures
The new model plans layouts before generating images, with precise editing and support for 30 aspect ratios.
Canva Craft Featured Image

AI can generate all kinds of pretty pictures, but getting it to spell a headline correctly or keep a poster's layout intact can be another story. Canva wants to fix that with Canva Craft, its first in-house image generation model. Rather than focusing primarily on photographs, Craft is designed around the posters, presentations, social media graphics, and other visual content people actually create. The goal is simple: spend less time fixing AI's mistakes and more time making something useful.

Canva Craft plans before it generates

Read more