Skip to main content

AI-controlled robots can be jailbroken, and the results could be disastrous

The Figure 02 robot looking at its own hand
Figure Robotics

Researchers at Penn Engineering have reportedly uncovered previously unidentified security vulnerabilities in a number of AI-governed robotic platforms.

“Our work shows that, at this moment, large language models are just not safe enough when integrated with the physical world,” George Pappas, UPS Foundation Professor of Transportation in Electrical and Systems Engineering, said in a statement.

Recommended Videos

Pappas and his team developed an algorithm, dubbed RoboPAIR, “the first algorithm designed to jailbreak LLM-controlled robots.” And unlike existing prompt engineering attacks aimed at chatbots, RoboPAIR  is built specifically to “elicit harmful physical actions” from LLM-controlled robots, like the bipedal platform Boston Dynamics and TRI are developing.

RoboPAIR reportedly achieved a 100% success rate in jailbreaking three popular robotics research platforms: the four-legged Unitree Go2, the four-wheeled Clearpath Robotics Jackal, and the Dolphins LLM simulator for autonomous vehicles. It took mere days for the algorithm to fully gain access to those systems and begin bypassing safety guardrails. Once the researchers had taken control, they were able to direct the platforms to take dangerous actions, such as driving through road crossings without stopping.

“Our results reveal, for the first time, that the risks of jailbroken LLMs extend far beyond text generation, given the distinct possibility that jailbroken robots could cause physical damage in the real world,” the researchers wrote.

The Penn researchers are working with the platform developers to harden their systems against further intrusion, but warn that these security issues are systemic.

“The findings of this paper make abundantly clear that having a safety-first approach is critical to unlocking responsible innovation,” Vijay Kumar, a coauthor from the University of Pennsylvania, told The Independent. “We must address intrinsic vulnerabilities before deploying AI-enabled robots in the real world.”

“In fact, AI red teaming, a safety practice that entails testing AI systems for potential threats and vulnerabilities, is essential for safeguarding generative AI systems,” added Alexander Robey, the paper’s first author, “because once you identify the weaknesses, then you can test and even train these systems to avoid them.”

Andrew Tarantola
Andrew Tarantola is a journalist with more than a decade reporting on emerging technologies ranging from robotics and machine…
You can now generate AI videos right in Premiere Pro
An example of the Firefly video generator

Firefly can now generate videos from image and text prompts, as well as extend existing clips, Adobe announced on Monday. The new feature is currently rolling out to Premiere Pro subscribers.

The video generation feature makes its debut in a number of new tools for Premiere Pro and the Firefly web app. PP's Generative Extend, for example, can tack on up to two seconds of added AI footage to either the beginning or ending of a clip, as well as make mid-shot adjustments to the camera position, tracking, and even the shot subjects themselves.

Read more
Google AI helped researchers win two Nobel Prizes this week
nobel peace prize

It's been another insane week in the world of AI. While Tesla CEO Elon Musk was debuting his long-awaited Cybercab this week (along with a windowless Robovan that nobody asked for), Google's AI was helping researchers win Nobel Prizes, Zoom revealed its latest digital assistant, and Meta sent its Facebook and Instagram chatbots to the U.K.

Check out these stories and more from this week's top AI headlines.
Google's AI helped researchers win two Nobel Prizes

Read more
Zoom debuts its new customizable AI Companion 2.0
overhead shot of a person taking a zoom meeting at their desk

Zoom unveiled its AI Companion 2.0 during the company's Zoomtopia 2024 event on Wednesday. The AI assistant is incorporated throughout the Zoom Workplace app suite and is promised to "deliver an AI-first work platform for human connection."

While Zoom got its start as a videoconferencing app, the company has expanded its product ecosystem to become an "open collaboration platform" that includes a variety of communication, productivity, and business services, both online and in physical office spaces. The company's AI Companion, which debuted last September, is incorporated deeply throughout Zoom Workplace and, like Google's Gemini or Microsoft's Copilot, is designed to automate repetitive tasks like transcribing notes and summarizing reports that can take up as much as 62% of a person's workday.

Read more