DeepMind has an AI bot that maneuvers through mazes and grabs objects on its own

Google’s DeepMind release a paper this week called Reinforcement Learning with Unsupervised Auxiliary Tasks, which describes a method to increase the learning speed of artificial intelligence and the final performance of agents — or bots. This method includes adding two main additional tasks to perform while the AI trains, and builds on the standard deep reinforcement learning foundation, which is basically a trial-and-error reward/punishment method where AI learns from its mistakes.

The first added task for speeding up AI learning is the ability to understand how to control the pixels on the screen. According to DeepMind, this method is similar to how a baby learns to control his/her hands by moving them and watching those movements. In the case of AI, the bot would understand visual input by controlling the pixels, thus leading to better scores.

“Consider a baby that learns to maximize the cumulative amount of red that it observes. To correctly predict the optimal value, the baby must understand how to increase ‘redness’ by various means, including manipulation (bringing a red object closer to the eyes); locomotion (moving in front of a red object); and communication (crying until the parents bring a red object),” DeepMind’s paper states. “These behaviors are likely to recur for many other goals that the baby may subsequently encounter.”

The second added task is used to train the AI to predict what the immediate awards will be based on a brief history of prior actions. To enable this, the team provided equal amounts of previous rewarding and non-rewarding histories. The end result is that the AI can discover visual features that will likely lead to rewards faster than before.

“To learn more efficiently, our agents use an experience replay mechanism to provide additional updates to the critics. Just as animals dream about positively or negatively rewarding events more frequently, our agents preferentially replay sequences containing rewarding events,” the paper adds.

With these two auxiliary tasks added to the previous A3C agent, the resulting new agent/bot is based on what the team calls Unreal (UNsupervised REinforcement and Auxiliary Learning). The team virtually sat this bot in front of 57 Atari games and a separate Wolfenstein-like labyrinth game consisting of 13 levels. In all scenarios, the bot was given the raw RGB output image, providing it direct access to the pixels for 100 percent accuracy. The Unreal bot was rewarded across the board for tasks like shooting down aliens in Space Invaders to grabbing apples in a 3D maze.

Because the Unreal bot can control the pixels and predict if actions will produce rewards, it’s capable of learning 10 times faster than DeepMind’s previous best agent (A3C). Even more, it produces better performance than the previous champion as well.

“We can now achieve 87 percent of expert human performance averaged across the Labyrinth levels we considered, with super-human performance on a number of them,” the company said. “On Atari, the agent now achieves on average 9x human performance.”

DeepMind is hopeful that the work that went into the Unreal bot will enable the team to scale up all of its agents/bots to handle even more complex environments in the near future. Until then, check out the video embedded above showing the AI moving through labyrinths and grabbing apples on its own without any human intervention.

Product Review

The competition was fierce, and this is the best TV of 2018

With stellar picture quality, excellent ease of use, and rich features, the LG C8 OLED is the best TV you can buy in 2018, but that doesn’t necessarily mean it is right for you. Check out our review to learn more.
Cars is helping Hyundai mine the artificial intelligence gold rush

In November 2018, Hyundai invested in a startup named We talked to the company's founder to learn more about what that means for consumers in the not-too-distant futures.
Movies & TV

'Stranger Things' season 3 teaser reveals the new episodes' titles

With a sophomore season as strong as its first, Stranger Things is now moving on to season 3. Here's everything we've learned so far about the Netflix series' upcoming third season.

You could spend $1,000 on an iPhone, or buy one of these awesome laptops instead

Finding a decent laptop is easy, but finding one under $1,000 is a bit tricky. Luckily, we've taken some of the guesswork out of picking out a budget laptop. Here are some of our favorites, the best laptops under $1,000.

Don't know what to do with all your old DVDs? Here's how to convert them to MP4

Given today's rapid technological advancements, physical discs are quickly becoming a thing of the past. Check out our guide on how to convert a DVD to MP4, so you can ditch discs for digital files.

Here’s how to install Windows on a Chromebook

If you want to push the functionality of your new Chromebook to another level, and Linux isn't really your deal, you can try installing Windows on a Chromebook. Here's how to do so, just in case you're looking to nab some Windows-only…

We want every laptop to be as thin as an iPhone. But is it practical?

The Acer Swift 7 is the thinnest notebook you can buy, and it feels like the notebook of the future. But it makes too many compromises along the way, and some weird design choices hold it back.

Supermicro investigation: no spy chips found on our motherboards

Supermicro announced the results of an investigation into the controversy surrounding its motherboards. The investigation was launched in response to reports that alleged the motherboards were compromised with malicious hardware.

Photographers can now customize the layout of Lightroom Classic controls

Tired of scrolling past Lightroom tools that you don't use? Adobe Lightroom Classic now allows users to reorganize the Develop panel. The update comes along with new sharing options in Lightroom CC, and updates to the mobile Lightroom app.

Best Products of 2018

Our reception desk has so many brown boxes stacked up, it looks like a loading dock. We’re on a first-name basis with the UPS guy. We get new dishwashers more frequently than most people get new shoes. What we’re trying to say is: We…

Windows 10 user activity logs are sent to Microsoft despite users opting out

Windows 10 Privacy settings may not be enough to stop PCs from releasing user activity data to Microsoft. Users discovered that opting out of having their data sent to Microsoft does little to prevent it from being released.

Intel's discrete graphics will be called 'Xe,' IGP gets Adapative Sync next year

Intel has officially dubbed its discrete graphics product Intel Xe, and the company also provided details about its Gen11 IGP. The latter will include adaptive sync support and will arrive in 2019.

Intel answers Qualcomm's new PC processors by pairing Core and Atom in 'Foveros'

Intel has announced a new packaging technology called 'Foveros' that makes it easier for the company to place multiple chips together on one package. That includes chips based on different Intel architectures, like Core and Atom.

Razer’s classic DeathAdder Elite gaming mouse drops to $40 on Amazon

If you're looking to pick up a new gaming mouse for the holidays, Amazon has you covered with this great deal on the classic Razer DeathAdder Elite gaming mouse with customizable buttons, RGB lighting, and a 16,000 DPI optical sensor.