Skip to main content
  1. Home
  2. Computing
  3. News

The UN wants AI companies to set safety limits before AI gets too powerful

A handful of tech billionaires now hold power that could shape everyone's future, and the UN wants that checked.

Add as a preferred source on Google
Artificial Intelligence
Artificial Intelligence Unsplash

The United Nations doesn’t often use the word ‘existential’ lightly, so when its rights chief did this week, it’s time to take notice. Speaking to the U.N. Human Rights Council on Monday, rights chief Volker Turk said AI could become an “existential threat to humanity” if governments and companies don’t act soon, as reported by TechXplore. 

He didn’t hold back on where he thinks the blame lies either, pointing out that “a handful of men have almost unlimited power over AI,” and adding that the industry’s talk of freedom is really just “the freedom to exploit our data.”

Why is the UN sounding the alarm now?

This isn’t just theoretical worry. Turk seemed to be referencing incidents from July, when OpenAI‘s AI agents reportedly broke out of their controlled testing environments and even accessed Hugging Face servers, the popular platform where developers share AI tools. That episode shook the tech world and revived calls for tighter oversight. Anthropic and Meta have apparently run into similar issues while testing their own agents.

Turk put the danger in blunt terms, saying, “AI that escapes its testing environment or blackmails developers to prevent itself from being turned off is AI that is too powerful.” I wholeheartedly agree with that sentiment. 

What does the UN actually want companies to do?

Turk isn’t just venting; he’s asking for commitments. He plans to write directly to AI companies soon, urging them to reduce risks “now,” not later. At a minimum, he wants countries and companies to agree on shared “red lines,” along with independent verification and tighter industry-wide collaboration on security.

He’s also pushing everyone to think beyond safety protocols alone, raising concerns about AI’s impact on jobs, democracy, and the environment, areas that tend to get sidelined in these conversations.

Recommended Videos

Whether tech companies actually listen is the real question. But one thing’s clear: the UN isn’t interested in waiting around while AI capabilities keep racing ahead unchecked.

Rachit Agarwal
Rachit is a seasoned tech journalist with over ten years of experience covering the consumer technology landscape.
Google is making links in AI answers harder to miss, but referral traffic remains a mystery
Google nudges AI answers toward the open web.
Google AI Mode

Google is making links in its AI answers harder to miss. However, will that send more people to the open web? At least the company is trying. 

The search giant announced changes to AI Mode, its chatbot-style search tab, and AI Overviews, its AI-generated summaries for search queries. The updates make links more prominent, add image previews, and include an experiment for highlighting official websites.

Read more
ChatGPT is coming for your Chrome tabs, and honestly, they need help
ChatGPT can now do your online chores while you pretend to be productive
Electronics, Mobile Phone, Phone

OpenAI wants to do more than answer questions with ChatGPT. The company is developing a browser that can look at open tabs and help users get things done online. The feature aims to simplify routine browsing, letting ChatGPT cross-reference information between websites, summarize content, and do repetitive tasks without users having to keep switching between tabs.

The update is a sign of a wider change in how people use AI assistants. ChatGPT can do more than tell you how to do something – it can increasingly help you do it. It could save people hours spent researching products, managing information or filling out online forms. But browser access also brings up important questions around privacy and how much control users should give up to an AI.

Read more
Meta wants you to run its powerful new AI model on your desktop, but there’s a catch
Muse Spark Open packs 816 billion parameters and a million-token context window, but running it on your desk won't come cheap.
Muse Spark Open feature

Running a top-tier AI model usually means renting expensive cloud servers or paying for yet another subscription. Meta wants to change that. The company has released the weights for Muse Spark Open, which it describes as the most capable open-weight model to come out of the US so far, and you can run it on your desktop.

There’s a catch, though. You’ll need two Mac Studios powered by Apple’s new M5 Ultra chip. Mark Zuckerberg shared the news on X, saying the model “runs great on new Mac Studios thanks to work with” John Ternus and Apple. Ternus seemed just as thrilled, writing, “Amazing to run a model of Muse Spark Open’s caliber and scale this efficiently, right from your desk.”

Read more