Skip to main content

Watch a self-driving artificial intelligence try to drive around ‘GTA V’

Riding a bike in Grand Theft Auto V.
Rockstar Games
Grand Theft Auto V might be a fun anarchy simulator, but it’s proving to have some practical uses in the real world, too. Some AI developers are using it to prototype their self-driving car algorithms and now you can watch one of them learning as it goes, live on Twitch.

Last year, the Darmstadt University of Technology in Germany and Intel Labs collaborated on a method of pulling visual information from Grand Theft Auto V, according to a report from Bloomberg. The game has now been modified such that it can be used in research projects designed to perfect self-driving programs.

The AI currently trying its luck at driving around Los Santos is called Charles. It uses pixel-level-analysis of on-screen information to make decisions about how to drive around the fictional city. In essence, it uses the same visual data that a human driver would. For an idea of what Charles “sees,” the developer has put together a short image library to explain it.

Crafted in Python — hence the Stream name, “Python Plays” — Charles currently doesn’t have any short-term memory, so doesn’t have much of an ability to learn. That’s why its driving is so poor right now and doesn’t seem to get much better no matter how long you view the stream. It does occasionally make it a few blocks without hitting anything and long-term viewers claim it is getting better, but it is far from being a safe or capable self-driving system.

Despite its obvious problems though, for a homebrew project by a single developer, the fact that it’s able to get around without any specialized information is reasonably impressive. You can imagine with time and further development, how an AI could become more proficient at driving by using a virtual toolkit like GTA V.

This kind of testing will hopefully help teach automated driving systems how to respond to unexpected obstacles with the same alertness and awareness as a skilled human driver. Anything can happen when you’re on the road, and it’s essential that an AI is able to respond appropriately, no matter what the eventuality.

At some point, self-driving technology will be advanced enough to stand up to trials on the same roads as every other human motorist. However, until we reach a certain level of sophistication, it will be the pedestrians roaming the streets in Grand Theft Auto V who bear the brunt of any unexpected occurrences.

Updated on 05-19-2017 by Jon Martindale: Added Twitch stream and information on it.

Editors' Recommendations

Brad Jones
Former Digital Trends Contributor
Brad is an English-born writer currently splitting his time between Edinburgh and Pennsylvania. You can find him on Twitter…
Grand Theft Auto V upgrade brings three graphics modes
Michael from Grand Theft Auto V.

After months of speculation, we finally know exactly what improvements are coming to Grand Theft Auto V in its upcoming PlayStation 5 and Xbox Series X versions on March 15. The news comes via the official Rockstar blog, where the developer shares specifics regarding graphical updates, character migration, and more.

Most notable among the current-gen console enhancements is a series of new graphics modes aimed at letting players tailor the experience to their own preferences. Fidelity Mode provides 4K resolution at 30 frames per second (fps) with ray tracing, while Performance Mode offers an upscaled 4K resolution with a targeted 60 fps. Lastly is Performance RT Mode, which acts as a combination of the other two modes, with upscaled 4K resolution, ray tracing, and a targeted 60 fps.

Read more
Grand Theft Auto V’s enhanced upgrade may not be free
Trevor from Grand Theft Auto V.

Grand Theft Auto V: Enhanced Edition, may not a free upgrade for players, according to Take-Two Interactive CEO Strauss Zelnick.

Answering a question during an investor relations call, Zelnick said that players would have to pay for the newest version of GTAV, saying that "there will be plenty of reasons to purchase it." It's not clear if he means that new players will have enough reasons to buy the game or if existing players will need to shell out more cash to play it on current-gen consoles.

Read more
GTA V has sold 160 million units, second only to Minecraft
The main characters of Grand Theft Auto V stand side-by-side.

Grand Theft Auto V has sold 160 million units since its launch, cementing it as the second bestselling game of all time behind Minecraft, according to Take-Two Interactive's latest investor presentation.

The figure for GTA V's sales is massive compared not only to other games by themselves, but entire franchises. According to Statista, by November 2020, the Assassin's Creed franchise had sold 155 million copies. That number doesn't include Assassin's Creed: Valhalla, which was released that same month, but even when you add the sales of that title in, GTA V's sales come extremely close to the total sales of the Assassin's Creed franchise.

Read more