Skip to main content

To train robotic servants, scientists built a virtual world where chores never end

VirtualHome: Simulating Tasks for Robots

Before robots can really help out around the home, they may have to sharpen their skills in a virtual world where chores never end. That’s the aim of a new research project in which a Sims-like system called “VirtualHome” helps artificial intelligence (A.I.) characters perform everyday activities, one step at a time.

For us, VirtualHome looks like a Lynchian nightmare. For robots, it’s something of a training ground.

Humans have a talent for inference, and we take that for granted. If you were told to vacuum the rug, you’d presumably have no problem completing the task without having to break it down into each of its individual steps — like walk to the closet, open the closet, grab the vacuum, move the vacuum, plug it in, etc. Machines, on the other hand, need to process each one of these subtasks to get the job done.

The goal of VirtualHome is to help robots learn tasks by first experiencing them in a virtual system. In its current form, the VirtualHome avatar can perform 1,000 separate actions in eight different settings, from the living room to kitchen and home office. The project is led by researchers at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory (CSAIL), the University of Toronto, McGill University, and the University of Ljubljana in Slovenia.

“We were trying to find a way to model complex activities to better understand the steps needed to do them, so that we could better identify them in video and potentially teach robots to perform them,” Xavier Puig, a CSAIL doctoral student who led the research, told Digital Trends. “There are very few datasets that have videos of people or agents doing step-by-step household activities.”

Puig and his colleagues created programs that laid out each step of a specific task in skeletal detail. For example, the task of watching TV involves five subtasks: Walk to the TV, turn on the TV, walk to the sofa, sit on the sofa, and watch the TV.

The researchers gave these program instructions to the VirtualHome system, which had the character act out these tasks. Videos of the agent performing the activities would be used to further train robots, giving them a visual example of what these actions look like.

In a recent paper, the researchers demonstrated that, by reviewing instructions or a video demo, their virtual agents could reconstruct the steps and perform the task. They hope such a system could help train household robots and build up a database of tasks to support human-machine communication.

The findings will be presented this month at the Computer Vision and Pattern Recognition conference in Salt Lake City, Utah.

Editors' Recommendations

Dyllan Furness
Dyllan Furness is a freelance writer from Florida. He covers strange science and emerging tech for Digital Trends, focusing…
Meet the game-changing pitching robot that can perfectly mimic any human throw
baseball hitter swings and misses

Who’s your favorite baseball pitcher? Shane McClanahan? Sandy Alcantara? Justin Verlander? Whoever you said, two of the top sports-tech companies in the U.S. -- Rapsodo and Trajekt Sports -- have teamed up to build a robot version of them, and the results are reportedly uncannily accurate.

Okay, so we’re not talking about walking-talking-pitching standalone robots, as great a sci-fi-tinged MLB ad as that would be. However, Rapsodo and Trajekt have combined their considerable powers to throw a slew of different technologies at the problem of building a machine that's able to accurately simulate the pitching style of whichever player you want to practice batting against -- and they may just have pulled it off, too.

Read more
The best portable power stations
EcoFlow DELTA 2 on table at campsite for quick charging.

Affordable and efficient portable power is a necessity these days, keeping our electronic devices operational while on the go. But there are literally dozens of options to choose from, making it abundantly difficult to decide which mobile charging solution is best for you. We've sorted through countless portable power options and came up with six of the best portable power stations to keep your smartphones, tablets, laptops, and other gadgets functioning while living off the grid.
The best overall: Jackery Explorer 1000

Jackery has been a mainstay in the portable power market for several years, and today, the company continues to set the standard. With three AC outlets, two USB-A, and two USB-C plugs, you'll have plenty of options for keeping your gadgets charged.

Read more
CES 2023: HD Hyundai’s Avikus is an A.I. for autonomous boat and marine navigation
Demonstration of NeuBoat level 2 autonomous navigation system at the Fort Lauderdale International Boat Show

This content was produced in partnership with HD Hyundai.
Autonomous vehicle navigation technology is certainly nothing new and has been in the works for the better part of a decade at this point. But one of the most common forms we see and hear about is the type used to control steering in road-based vehicles. That's not the only place where technology can make a huge difference. Autonomous driving systems can offer incredible benefits to boats and marine vehicles, too, which is precisely why HD Hyundai has unveiled its Avikus AI technology -- for marine and watercraft vehicles.

More recently, HD Hyundai participated in the Fort Lauderdale International Boat Show, to demo its NeuBoat level 2 autonomous navigation system for recreational boats. The name mashes together the words "neuron" and "boat" and is quite fitting since the Avikus' A.I. navigation tech is a core component of the solution, it will handle self-recognition, real-time decisions, and controls when on the water. Of course, there are a lot of things happening behind the scenes with HD Hyundai's autonomous navigation solution, which we'll dive into below -- HD Hyundai will also be introducing more about the tech at CES 2023.

Read more