DeepMind has an AI bot that maneuvers through mazes and grabs objects on its own

Google’s DeepMind release a paper this week called Reinforcement Learning with Unsupervised Auxiliary Tasks, which describes a method to increase the learning speed of artificial intelligence and the final performance of agents — or bots. This method includes adding two main additional tasks to perform while the AI trains, and builds on the standard deep reinforcement learning foundation, which is basically a trial-and-error reward/punishment method where AI learns from its mistakes.

The first added task for speeding up AI learning is the ability to understand how to control the pixels on the screen. According to DeepMind, this method is similar to how a baby learns to control his/her hands by moving them and watching those movements. In the case of AI, the bot would understand visual input by controlling the pixels, thus leading to better scores.

“Consider a baby that learns to maximize the cumulative amount of red that it observes. To correctly predict the optimal value, the baby must understand how to increase ‘redness’ by various means, including manipulation (bringing a red object closer to the eyes); locomotion (moving in front of a red object); and communication (crying until the parents bring a red object),” DeepMind’s paper states. “These behaviors are likely to recur for many other goals that the baby may subsequently encounter.”

The second added task is used to train the AI to predict what the immediate awards will be based on a brief history of prior actions. To enable this, the team provided equal amounts of previous rewarding and non-rewarding histories. The end result is that the AI can discover visual features that will likely lead to rewards faster than before.

“To learn more efficiently, our agents use an experience replay mechanism to provide additional updates to the critics. Just as animals dream about positively or negatively rewarding events more frequently, our agents preferentially replay sequences containing rewarding events,” the paper adds.

With these two auxiliary tasks added to the previous A3C agent, the resulting new agent/bot is based on what the team calls Unreal (UNsupervised REinforcement and Auxiliary Learning). The team virtually sat this bot in front of 57 Atari games and a separate Wolfenstein-like labyrinth game consisting of 13 levels. In all scenarios, the bot was given the raw RGB output image, providing it direct access to the pixels for 100 percent accuracy. The Unreal bot was rewarded across the board for tasks like shooting down aliens in Space Invaders to grabbing apples in a 3D maze.

Because the Unreal bot can control the pixels and predict if actions will produce rewards, it’s capable of learning 10 times faster than DeepMind’s previous best agent (A3C). Even more, it produces better performance than the previous champion as well.

“We can now achieve 87 percent of expert human performance averaged across the Labyrinth levels we considered, with super-human performance on a number of them,” the company said. “On Atari, the agent now achieves on average 9x human performance.”

DeepMind is hopeful that the work that went into the Unreal bot will enable the team to scale up all of its agents/bots to handle even more complex environments in the near future. Until then, check out the video embedded above showing the AI moving through labyrinths and grabbing apples on its own without any human intervention.


Exclusive: The Surface Hub 2S will revolutionize work. Here’s how it was made

Exclusive interviews with the designers, futurists, and visionaries behind the Surface Hub 2 paint a dramatic picture of how Microsoft thinks collaboration will change your office.

Walmart offers big price cuts on air fryers from La Gourmet and Farberware

Walmart made deep price cuts on air fryers from La Gourmet, Farberware, and others. Air frying is faster, healthier, and easier to clean than traditional deep frying. You also can use most air fryers for baking, roasting, and grilling.

If we get a Nintendo 64 Classic, it needs to have these games

The Nintendo 64 introduced a long list of top-tier games, but which were the iconic platform's best? From Mario Party to Ocarina of Time to NFL Blitz, check out our picks for the best N64 games.

Chromebooks are laptops, but they do things a little differently

Chromebooks are an intriguing branch of laptops that are often cheaper and faster than their Windows counterparts, but they are a little more limited. Intrigued? Here's everything you need to know about Chromebooks.

The hottest Nintendo Switch games you can get right now

The Nintendo Switch's lineup started off small, but games have steadily released as the console continues through its second year. Here are the best Nintendo Switch games available now.
Product Review

You won't buy Microsoft's Surface Hub 2S, but it could still change your life

The Microsoft Surface Hub 2S wants to change the way you collaborate at work. That’s a lofty goal most devices fail to achieve, but the unique Hub 2S could be an exception. And trust us – you’re going to want it.
Emerging Tech

How emotion-tracking A.I. will change computing as we know it

Affectiva is just one of the startups working to create emotion-tracking A.I. that can work out how you're feeling. Here's why this could change the face of computing as we know it.

Meet the mastermind behind Microsoft's massive new Surface Hub

Microsoft Chief Product Officer Panos Panay gives us an exclusive peek at the 85-inch Surface Hub 2, and explains how innovation and collaboration will transform your workplace.

Microsoft reveals details of Surface Hub 2S, coming in June at $9,000

The Surface Hub 2 could be the most expensive whiteboard ever made, but it should be a powerful and capable one. With the ability to connect several of the 50-inch displays together, the picture at least, should be gorgeous.

Report says 20% of all 2018 web traffic came from bad bots

Distil Networks published its annual Bad Bot Report this week and announced that 20% of all web traffic in 2018 came from bad bots. The report had other similarly surprising findings regarding the state of bots as well.

Learn to uninstall a Steam game and clear some space on your PC

Looking to learn how to uninstall Steam games? You've come to the right place. In this guide, we walk you through the process step by step, whether you want Steam to do it for you or handle the process manually.

Amazon strikes $100 off the price of Microsoft Surface Go tablets

If you've been eyeing Microsoft's Surface Go for its compact size and portability, now may be a great time to buy the tablet. Amazon has a $100 discount on the Surface Go, bringing the price of this slate down to just under $400.

Sweet 16: Wacom’s Cintiq 16 pen display makes retouching photos a breeze

Wacom’s Cintiq pen displays are usually reserved for the pros (or wealthy enthusiasts), but the new Cintiq 16 brings screen and stylus editing to an approachable price. Does it cut too much to get there?

Mueller report releases on CD, forces Congress to find PCs with disc drives

The Mueller report was released this week to Congress via CDs and congressional members had to find PCs with working disc drives to access the 400-page document. The redacted report was also released to the public on a website.