Skip to main content

Like a real-life mech, MIT engineers use VR to put you in the head of a robot

Operating Robots with Virtual Reality
Researchers from the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Lab (CSAIL) have come up with a smart way to combine two of the most exciting emerging technologies — virtual reality (VR) and robotics — by creating a smart new telepresence control system. Using one of Rethink Robotics’ Baxter robots, a VR headset, and maybe a page or two from mecha anime like the classic Mobile Suit Gundam Wing, they created a smart head-mounted “sensor display” system which puts human operators in the head of the robot they are controlling.

“A lot of jobs are difficult to do remotely, particularly in manufacturing and industry,” Jeffrey Lipton, one of the postdoctoral researchers who worked on the project, told Digital Trends. “A system like this could one day allow humans to supervise robots from a distance. This would enable employees to work from home, and could even open up manufacturing jobs to people with physical limitations, [such as those] who can’t lift heavy or bulky objects. Many industrial jobs are also terrible for human health — imagine servicing the inside of an airplane or working out on an oil rig. They can be dangerous, cramped and uncomfortable, but right now they need a human mind to understand, make decisions, and do movements. We think this model of teleoperation could allow us to keep humans safe and away from these sites while leveraging human mental capabilities.”

Jason Dorfman/MIT CSAIL
Jason Dorfman/MIT CSAIL

MIT’s smart system embeds the user in a virtual reality control room with multiple sensor displays, allowing the user to see everything the robot is seeing at any given moment. To execute tasks, the human then employs gestures — picked up courtesy of hand controllers — which are mirrored by the robot. Controls accessible by the human user appear virtually, rather than being physical controls. This allows them to change depending on what the robot has to carry out at any given time.

“We hope to extend this work to many different robots and scale up the trials to tasks beyond assembly,” Lipton continued, describing future research the CSAIL scientists hope to carry out. For more on the Baxter project, you can check out a research paper published earlier this year, titled “Baxter’s Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturing.”

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Valve Index VR goes on sale again Monday in time for Half-Life: Alyx
Valve Index VR

Valve announced that the Index VR headset will be back in stock for a limited time this Monday, March 9. The Steam creator revealed the return of the anticipated headset on Twitter after weeks of it being out of stock due to Coronavirus related issues.

https://twitter.com/valvesoftware/status/1235311669045272577?s=21

Read more
Watch MIT’s Mini Cheetahs limber up for the robot apocalypse
watch mits mini cheetahs limber up for the robot apocalypse mit

Testing 9 New Mini Cheetahs

The Biomimetics Robotics team at the Massachusetts Institute of Technology (MIT) recently took a number of its talented Mini Cheetah robots for some fun in the park.

Read more
Robotics company is offering $125K if you’ll let a robot use your face
will computers revolt preparing for the future of ai ex machina xxl

Ever wonder why all the T-800 model Terminators look like Arnold Schwarzenegger? Presumably at some point, in that fictional world, there was a real person who served as the model for Skynet’s advanced killing machines. In fact, he may have signed over the rights to his physical appearance in something like the deal that U.K.-based additive manufacturing company Geomiq is currently offering.

On the orders of a robotics company client, Geomiq is promising $125,000 to a person willing to submit their face as the model for a new line of humanoid robots. The robots are reportedly intended for some kind of elderly care and will be unveiled next year. While most entrants will not get the $125k prize, the person selected as the face model will scoop up the cash in exchange for waiving all rights to their appearance.

Read more