Skip to main content

To train robotic servants, scientists built a virtual world where chores never end

VirtualHome: Simulating Tasks for Robots

Before robots can really help out around the home, they may have to sharpen their skills in a virtual world where chores never end. That’s the aim of a new research project in which a Sims-like system called “VirtualHome” helps artificial intelligence (A.I.) characters perform everyday activities, one step at a time.

Recommended Videos

For us, VirtualHome looks like a Lynchian nightmare. For robots, it’s something of a training ground.

Humans have a talent for inference, and we take that for granted. If you were told to vacuum the rug, you’d presumably have no problem completing the task without having to break it down into each of its individual steps — like walk to the closet, open the closet, grab the vacuum, move the vacuum, plug it in, etc. Machines, on the other hand, need to process each one of these subtasks to get the job done.

The goal of VirtualHome is to help robots learn tasks by first experiencing them in a virtual system. In its current form, the VirtualHome avatar can perform 1,000 separate actions in eight different settings, from the living room to kitchen and home office. The project is led by researchers at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory (CSAIL), the University of Toronto, McGill University, and the University of Ljubljana in Slovenia.

“We were trying to find a way to model complex activities to better understand the steps needed to do them, so that we could better identify them in video and potentially teach robots to perform them,” Xavier Puig, a CSAIL doctoral student who led the research, told Digital Trends. “There are very few datasets that have videos of people or agents doing step-by-step household activities.”

Puig and his colleagues created programs that laid out each step of a specific task in skeletal detail. For example, the task of watching TV involves five subtasks: Walk to the TV, turn on the TV, walk to the sofa, sit on the sofa, and watch the TV.

The researchers gave these program instructions to the VirtualHome system, which had the character act out these tasks. Videos of the agent performing the activities would be used to further train robots, giving them a visual example of what these actions look like.

In a recent paper, the researchers demonstrated that, by reviewing instructions or a video demo, their virtual agents could reconstruct the steps and perform the task. They hope such a system could help train household robots and build up a database of tasks to support human-machine communication.

The findings will be presented this month at the Computer Vision and Pattern Recognition conference in Salt Lake City, Utah.

Dyllan Furness
Former Digital Trends Contributor
Dyllan Furness is a freelance writer from Florida. He covers strange science and emerging tech for Digital Trends, focusing…
Jaguar Land Rover, Nissan hit the brake on shipments to U.S. over tariffs
Range Rover Sport P400e

Jaguar Land Rover (JLR) has announced it will pause shipments of its UK-made cars to the United States this month, while it figures out how to respond to President Donald Trump's 25% tariff on imported cars.

"As we work to address the new trading terms with our business partners, we are taking some short-term actions, including a shipment pause in April, as we develop our mid- to longer-term plans," JLR said in a statement sent to various media.

Read more
DeepSeek readies the next AI disruption with self-improving models
DeepSeek AI chatbot running on an iPhone.

Barely a few months ago, Wall Street’s big bet on generative AI had a moment of reckoning when DeepSeek arrived on the scene. Despite its heavily censored nature, the open source DeepSeek proved that a frontier reasoning AI model doesn’t necessarily require billions of dollars and can be pulled off on modest resources.

It quickly found commercial adoption by giants such as Huawei, Oppo, and Vivo, while the likes of Microsoft, Alibaba, and Tencent quickly gave it a spot on their platforms. Now, the buzzy Chinese company’s next target is self-improving AI models that use a looping judge-reward approach to improve themselves.

Read more
Toyota shifts gears: 15 New EVs and a million cars by 2027
Front three quarter view of the 2023 Toyota bZ4X.

After years of cautiously navigating the electric vehicle (EV) market, Toyota is finally ramping up its commitment to fully electric vehicles.
The Japanese automaker, which has long relied on hybrids, is now planning to develop about 15 fully electric models by 2027, up from five currently. These models will include vehicles under the Toyota and Lexus brands, with production expected to reach 1 million units annually by that year, according to a report from Nikkei.
This strategy marks a significant shift for Toyota, which has thus far remained conservative in its approach to electric cars. The company sold just 140,000 EVs globally in 2024—representing less than 2% of its total global sales. Despite this, Toyota is aiming for a much larger presence in the EV market, targeting approximately 35% of its global production to be electric by the end of the decade.
The Nikkei report suggests the company plans to diversify its production footprint beyond Japan and China and expanding into the U.S., Thailand, and Argentina. This would help mitigate the impact of President Donald Trump’s 25% tariffs on all car imports, as well as reduce delivery times. Toyota is also building a battery plant in North Carolina.
For now, Toyota has only two fully electric vehicles on the U.S. market: The bZ4X  and the Lexus RZ models. The Japanese automaker is expected to introduce new models like the bZ5X and a potential electric version of the popular Tacoma pickup.
Separately, Toyota and Honda, along with South Korea’s Hyundai, all announced on April 4 that they would not be raising prices, at least over the next couple of months, following the imposition of U.S. tariffs. According to a separate Nikkei report, Toyota’s North American division has told its suppliers that it will absorb the extra costs of parts imported from Mexico and Canada. Another 25% for automotive parts imported to the U.S. is slated to come into effect on May 3.

Read more