Robot learns how to grab objects by analyzing them in simulated reality

Our hands are pretty great at picking up all manner of objects, while our brains are fine-tuned at working out exactly where and how to pick up an object most securely. That’s not easy for a robot, however. Faced with a world full of strange-shaped objects to pick up and manipulate, there’s no easy way of programming a robot to be able to know the precise grip it should employ to deal with every single object it might encounter.

That’s where researchers from the University of California, Berkeley come into play. They’ve developed a system called DexNet 2.0 that works out how to perform this task not by endlessly practicing in real life, but by analyzing the objects in virtual reality — courtesy of a deep learning neural network.

“We construct a probabilistic model of the physics of grasping, rather than assuming the robot knows the true state of the world,” Jeff Mahler, a postdoctoral researcher who worked on the project, told Digital Trends. “Specifically we model the robustness, or probability of achieving a successful grasp, given an observation of the environment. We use a large dataset of 1,500 virtual 3D models to generate 6.7 million synthetic point clouds and grasps across many possible objects. Then we can learn to predict the probability of success of grasps given a point cloud using deep learning. Deep learning allows us to learn this mapping across such as large and complex dataset.”

robot learns real world virtual reality dexnet

The most obvious application for DexNet would be to improve robots used in warehousing or manufacturing by enabling them to cope with new components or other objects, and be able to manipulate them by packing them into boxes for shipping or performing assemblies. However, as Mahler points out, the technology could also help improve the capabilities of home robots — such as those that can clean up items or be used for assistive care, such as bringing items to elderly folks who can’t otherwise reach them.

There’s still more work to be done, though. “The big thrust of research in the next year is related to having the robot grasp for a particular use case,” Mahler said. “For example, orienting a bottle so it can be placed standing up or flipping legos over to plug them into other bricks.”

Other specifics on the agenda include the ability to grasp objects in clutter and reorienting objects for assembly. The team also plans to release the necessary code to let users generate their own training datasets and deploy the system on their own parallel-jaw robot. This will take place later in 2017.

“We have some interest in commercialization, but are primarily interested in furthering research on the subject in the next 6-12 months,” Mahler concluded.


Nose cam no more. How Dell avoided a notch and fixed the XPS 13’s biggest flaw

The new Dell XPS 13 moves the webcam from the below the screen to the top, finally vanquishing the one obstacle facing thin, sleek laptop displays. We have the exclusive story on how it was done.
Home Theater

The seven best TVs you can buy right now, from budget to big screen

Looking for a new television? In an oversaturated market, buying power is at an all-time high, but you'll need to cut through the rough to find a diamond. We're here to help with our picks for the best TVs of 2019.
Emerging Tech

Awesome Tech You Can’t Buy Yet: heat-powered watches, phone cases with reflexes

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!

IBM’s Q System One is the rock star quantum computing needs

Quantum computing is intriguing. It's also confusing, distant, and unfathomable, which means people often skip right by the topic. IBM's Q System One may not be a scientific breakthrough, but it finally gives everyone a reason to stop and…
Emerging Tech

Why wait? Here are some CES 2019 gadgets you can buy right now

Companies come to CES to wow us with their cutting edge technology, but only a few products are slated to hit the market right away. Here is our list of the best CES 2019 tech you can buy right now.
Emerging Tech

Drones: New rules could soon allow flights over people and at night

With commercial operators in mind, the U.S. government is looking to loosen restrictions on drone flights with a set of proposals that would allow the machines greater freedom to fly over populated areas and also at night.
Emerging Tech

Yamaha’s new app lets you tune your motorcycle with a smartphone

It used to be that if you wanted to tune your motorcycle’s engine and tweak its performance, you needed specialized tools and even more specialized knowledge. Yamaha’s new Power Tuner app changes that.
Emerging Tech

Short film celebrates New Yorker’s amazing robot costumes

New York City resident Peter Kokis creates stunning robot costumes out of household trash. His designs are huge, heavy, and extremely intricate, and never fail to turn heads when he's out and about.
Emerging Tech

In a first for humankind, China is growing plants on the moon

Having recently landed a probe on the far side of the moon, China announced that it managed to grow the first plant on the moon, too. Here's why that matters for deep space travel.
Emerging Tech

Ford’s sweaty robot bottom can simulate 10 years of seat use in mere days

Ford has developed 'Robutt,' a sweaty robot bottom that's designed to simulate the effects of having a pair of human buttocks sitting on its car seats for thousands of hours. Check it out.
Emerging Tech

CES 2019 recap: All the trends, products, and gadgets you missed

CES 2019 didn’t just give us a taste of the future, it offered a five-course meal. From 8K and Micro LED televisions to smart toilets, the show delivered with all the amazing gadgetry you could ask for. Here’s a look at all the big…
Emerging Tech

Want to know which drones are flying near you? There’s an app for that

Want to know what that mysterious drone buzzing over your head is up to? A new system developed by AirMap, Google Wing, and could soon tell you -- via a map on your phone.
Emerging Tech

A Japanese hotel fires half its robot staff for being bad at their jobs

Japan’s oddball Henn na Hotel has fired half of its 243 robot staff. The reason? Because these labor-saving machines turned out to be causing way more problems than they were solving.
Emerging Tech

CERN plans to build a massive particle collider that dwarfs the LHC

CERN already has the world's biggest particle accelerator. Now it wants a bigger one. Meet the 9 billion euro Future Circular Collider that will allow physicists to extend their study of the universe and matter at the smallest level.