Skip to main content

Watch Meta demonstrate full-body VR tracking with just a Quest headset

Meta Reality Labs is making big strides in avatar rendering with the latest advances combining machine learning (ML) with sensor data from Quest VR headsets to show your full body, including arms, legs, torso, and head. The result is a very realistic and accurate representation of the poses and movements of a person wearing a Quest 2 headset.

QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars

This is quite amazing since only the positions and orientations of the Quest VR headset and its two controllers were used to estimate leg motion and position. There are no tracking bands placed on the legs and no external cameras used for this remarkable system. Meta Research Scientist Alexander Winkler shared several videos on Twitter along with links to the scientific paper on arXiv and a YouTube video with more detail.

In a scientific sense, this is known as motion tracking from sparse sensors and ML is particularly adept at extracting meaningful information from very little, if there are enough dependencies between what is known and unknown. Since we swing our arms for balance when walking and running, arm movement is a good indicator of what the legs are usually doing. Combined with head tilt and direction, the ML system can predict most human motion very accurately.

Body tracking in VR using a Quest headset.
Image used with permission by copyright holder

More traditional approaches to tracking limbs rely on extra hardware, such as reflective markers placed on your legs and torso that are identified by external cameras, or bands with wireless beacons worn at various locations on your legs to transmit position and motion data.

While these methods work, those are typically sold as accessory items that cost more and aren’t well supported in most apps and games. If Meta can achieve such good results with a Quest 2 headset, developers will be more likely to build body tracking in their games and apps.

Despite these significant advances, Meta Reality Labs admits that more work needs to be done. If you move fast enough, the ML model fails to correctly identify your pose. Unusual stances are difficult for the computer to estimate and if the virtual environment has obstacles that don’t exist in reality, the movement won’t match. The overall effect seems to be very good, however, and it would be a nice upgrade to be able to see a full body instead of floating torsos when chatting with friends in VR.

Hopefully, this technology will be ready for launch soon. With Meta’s Quest Pro headset announcement expected in just a few weeks, the timing for full-body avatars seems perfect. The Meta Quest Pro can track eye movement and facial expressions, providing an improved sense of presence to your friends, family, and coworkers might be greatly enhanced in the near future.

Editors' Recommendations

Alan Truly
Alan is a Computing Writer living in Nova Scotia, Canada. A tech-enthusiast since his youth, Alan stays current on what is…
We have some bad news for Quest owners
The original Oculus Quest 1 appears on a white background.

If you've been holding onto your original Oculus Quest or Quest 1 VR headset, it's time to upgrade because support is ending soon. The Quest 1 came out in 2019, so this headset is nearly five years old and quickly approaching the end of support. In August 2024, the original Quest will stop receiving security updates and bug fixes.

Meta recently emailed developers with an alert that new VR games and apps will be blocked from supporting the Quest 1 next month. Updates to existing apps will also cease. The Quest 1 will still have access to a large library of games, but to enjoy the latest releases, it's time to upgrade. This news follows a slow pullback that started last year when social features were removed.

Read more
These are all the must-try apps for your Meta Quest 3
A profile shot shows Alan Truly wearing the Meta Quest 3.

The Meta Quest 3 is one of the coolest ways to experience next-gen gaming from the comfort of your home. Heck, you can ever wear the VR headset in public, and shame on those who judge! But gaming isn’t the only immersive capability your Quest 3 is capable of. Thanks to apps, you’ll almost be able to use your headset as a standalone computer or smartphone. Of course, there’s a little know-how that goes into just what apps to download, and where you can find them — which is where we come in.

For your education and entertainment, we’ve gone ahead and rounded up several of our favorite Meta Quest 3 apps. You’ll find options for everything from movie and TV show players to 3D model-making. There’s plenty of fun software to get into, so let’s not delay any further!
Web browsing in 3D

Read more
A cheaper version of the Meta Quest 3 now seems more likely than ever
A Meta Quest 3 sits on a wooden floor in moody backlighting.

Rumors have been circulating about Meta Quest 3 Lite, a budget model meant to replace the Quest 2, the world’s most popular VR headset. Meta will inevitably remove the Quest 2 from its lineup soon since it launched four years ago.

Meta launched the Quest 3, a VR headset with mixed reality features, in October 2023, and if you believe the rumors, it could be planning to add a new low-cost model to the lineup soon.

Read more