DeepMind has an AI bot that maneuvers through mazes and grabs objects on its own

Google’s DeepMind release a paper this week called Reinforcement Learning with Unsupervised Auxiliary Tasks, which describes a method to increase the learning speed of artificial intelligence and the final performance of agents — or bots. This method includes adding two main additional tasks to perform while the AI trains, and builds on the standard deep reinforcement learning foundation, which is basically a trial-and-error reward/punishment method where AI learns from its mistakes.

The first added task for speeding up AI learning is the ability to understand how to control the pixels on the screen. According to DeepMind, this method is similar to how a baby learns to control his/her hands by moving them and watching those movements. In the case of AI, the bot would understand visual input by controlling the pixels, thus leading to better scores.

“Consider a baby that learns to maximize the cumulative amount of red that it observes. To correctly predict the optimal value, the baby must understand how to increase ‘redness’ by various means, including manipulation (bringing a red object closer to the eyes); locomotion (moving in front of a red object); and communication (crying until the parents bring a red object),” DeepMind’s paper states. “These behaviors are likely to recur for many other goals that the baby may subsequently encounter.”

The second added task is used to train the AI to predict what the immediate awards will be based on a brief history of prior actions. To enable this, the team provided equal amounts of previous rewarding and non-rewarding histories. The end result is that the AI can discover visual features that will likely lead to rewards faster than before.

“To learn more efficiently, our agents use an experience replay mechanism to provide additional updates to the critics. Just as animals dream about positively or negatively rewarding events more frequently, our agents preferentially replay sequences containing rewarding events,” the paper adds.

With these two auxiliary tasks added to the previous A3C agent, the resulting new agent/bot is based on what the team calls Unreal (UNsupervised REinforcement and Auxiliary Learning). The team virtually sat this bot in front of 57 Atari games and a separate Wolfenstein-like labyrinth game consisting of 13 levels. In all scenarios, the bot was given the raw RGB output image, providing it direct access to the pixels for 100 percent accuracy. The Unreal bot was rewarded across the board for tasks like shooting down aliens in Space Invaders to grabbing apples in a 3D maze.

Because the Unreal bot can control the pixels and predict if actions will produce rewards, it’s capable of learning 10 times faster than DeepMind’s previous best agent (A3C). Even more, it produces better performance than the previous champion as well.

“We can now achieve 87 percent of expert human performance averaged across the Labyrinth levels we considered, with super-human performance on a number of them,” the company said. “On Atari, the agent now achieves on average 9x human performance.”

DeepMind is hopeful that the work that went into the Unreal bot will enable the team to scale up all of its agents/bots to handle even more complex environments in the near future. Until then, check out the video embedded above showing the AI moving through labyrinths and grabbing apples on its own without any human intervention.


In the age of Alexa and Siri, Cortana’s halo has grown dim

In a sea of voice assistants, Cortana has become almost irrelevant. The nearly five-year-old voice assistant is seeing little love from consumers, and here’s why it is dead.
Emerging Tech

Statistician raises red flag about reliability of machine learning techniques

Machine learning is everywhere in science and technology. But how reliable are these techniques really? A statistician argues that questions of accuracy and reproducibility of machine learning have not been fully addressed.

These are the must-have games that every Xbox One owner needs

More than four years into its life span, Microsoft's latest console is finally coming into its own. From Cuphead to Halo 5, the best Xbox One games offer something for players of every type.

Bentley Bentayga Speed surpasses Lamborghini Urus as world’s fastest SUV

The Bentley Bentayga Speed has wrested the title of world's fastest SUV from its cousin, the Lamborghini Urus. But the Bentley is just 1 mph faster than the Lamborghini. It requires 626 horsepower to achieve that top speed.

Tight on space? Here’s how to transfer photos from an iPhone to a computer

Never lose any of your cherished selfies or family vacation photos from your iPhone again by learning how to transfer photos from your iPhone to a computer, whether you want to use a cable or wireless transfer.

Wi-Fi helps connect all of our devices at high-speed, but what exactly is it?

What is Wi-Fi? It's a technology we all use everyday to connect all of our portable devices, but understanding how it works and how far it's come from its humble beginnings is another thing entirely.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Grow veggies indoors and shower more efficiently

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!

Are you one of the billions who have watched these super-popular YouTube videos?

Viral videos can quickly garner millions upon millions of views, but even they fall well behind the view counts on the most watched YouTube videos ever. Those have been watched billions of times.

Windows updates shouldn't cause problems, but if they do, here's how to fix them

Windows update not working? It's a more common problem than you might think. Fortunately, there are a few steps you can take to troubleshoot it and in this guide we'll break them down for you step by step.

Speed up your system by tweaking the startup application in Windows and MacOS

Bothered by programs that automatically start when you boot your computer, or want to add a new one to the list? Here's how to change your startup programs in Windows 7, 8, and 10, along with Apple's MacOS.

Marriott asking guests for data to see if they were victims of the Starwood hack

Marriott has created an online form to help you find out if your data was stolen in the massive Starwood hack that came to light toward the end of 2018. But take note, it requires you to submit a bunch of personal details.

Apple may go big with a redesigned 16-inch MacBook Pro, 31.6-inch 6K display

In a research note, well-respected Apple analyst Ming-Chi Kuo writes that there are indications the company could be releasing a 16-inch MacBook Pro, as well as a 31.6-inch 6K display in 2019.

Accidental Amazon listings provides peek at cost of GTX 1660 Ti PCs

An accidental listing on Amazon provided a peek at the final cost of some of the new Windows 10 desktops with the GeForce GTX 1660 Ti on board, with prices possibly in the $1,200 range.

The best Presidents’ Day sales 2019: Amazon, Walmart, Dell, and more

Presidents' Day sales are a great chance to score electronics, clothing, home and office stuff, and other goodies at a discount. We’ve smoked out a large handful of the best of these Presidents' Day deals, from tech to bedding, to help…