Skip to main content
  1. Home
  2. Computing
  3. News

Google wants to show off your face a bit better in VR and mixed reality

Mark Coppock
By

google research daydream labs creating realistic facial representation in vr and mixed reality
Image used with permission by copyright holder
Being around someone using a virtual reality system can be an eerie experience. The typical VR headset fully hides the user’s eyes, making it hard to follow along with the action and interact with anyone using a system. When you’re inside a VR experience with another user, today’s systems don’t pass along facial expressions and so the social aspects and overall experience are hindered.

Google Research is looking to resolve that problem with a new project it is working on with Daydream Labs. By combining 3D computer vision, machine learning, and advanced rendering methods, the project aims to create more of a realistic solution that can convey a user’s identity, focus, and face.

Recommended Videos

Basically, the solution uses a VR headset modified by SensoMotoric Instruments that includes a video camera and eye-tracking components to show a person’s facial variations while blinking and looking in certain directions. This information is overlaid inside the mixed reality environment using green screen technology, basically making it look like the VR headset is a scuba mask — the translucent effect is deliberate and intended to avoid the “uncanny valley” effect that can make avatars and robots come across as unnatural.

From there, this modified user representation is worked into VR and mixed reality environments. The experienced is completed by showing a more realistic and expressive view of the user including their facial expressions and the ability to see where the user is looking.

While the technology does nothing for the real-world headset user, it dramatically improves the experience inside a VR or mixed reality world. Users can interact more realistically, using facial expressions and nonverbal cues to greatly enhance the communication and social interaction possible in VR environments. While the gaming applications are obvious, more mundane and practical solutions like VR video conferencing would also be greatly improved.

Related

There is no word yet on when this technology will make it to shipping products. Anyone who wants to delve into the details can visit the Google Research blog to learn more about how the pieces connect to create a much more realistic virtual representation of the VR user.

Editors' Recommendations

Topics
Mark Coppock
Mark Coppock
Computing Writer
Mark has been a geek since MS-DOS gave way to Windows and the PalmPilot was a thing. He’s translated his love for…
Apple’s mixed reality headset could be as powerful as the MacBook Pro
A woman reaching out while wearing a VR headset.

Just yesterday we learned that the metaverse is supposedly “off-limits” for Apple’s planned mixed reality headset. Now, a fresh report has claimed the device will use the same 96W power adapter as the latest MacBook Pro and will come loaded with a dual-chip setup, suggesting the headset will have some serious firepower under the hood.

The device is expected to blend augmented reality (AR) and virtual reality (VR), the result being a mixed reality (MR) headset that offers the flexibility to switch between AR and VR as you see fit. Previous rumors have suggested it will be bristling with around a dozen cameras, use high-resolution displays, and could also incorporate advanced gesture-recognition tech.

Read more
The metaverse is ‘off limits’ for Apple’s upcoming VR headset
A person wearing a virtual reality headset.

News on Apple’s upcoming mixed reality headset has been accelerating in recent weeks, with several fresh reports shedding further details on the project. Now, reliable reporter Mark Gurman has chimed in with his thoughts on Apple’s vision for the device.

In his latest Power On newsletter, Gurman explained that Apple’s headset will be used primarily for short bursts of activity rather than prolonged sessions. This contrasts with other companies that are seeking to build a "metaverse" of immersive experiences that replicate real life in a digital realm.

Read more
Pimax’s 12K QLED VR headset wants to take virtual reality to the next level
Pimax's new 12k QLED VR headset.

It seems that virtual reality may be about to become even more real than ever before -- all thanks to a new VR headset. Pimax, a company that manufactures VR equipment, announced the upcoming release of a new 12K QLED VR headset that will feature technologies such as eye tracking, full-body tracking, and refresh rates of up to 200Hz. The headset, dubbed Pimax Reality 12K QLED, is part of the company's venture into the metaverse and a step toward bringing true realism to using VR.

During today's Pimax Frontier event, the company's representatives talked at length about the goals behind the product -- naturalness, self-awareness, and freedom. Pimax wants to bring these qualities into virtual reality and the metaverse, allowing people from all over the world to interact and explore virtual worlds together. While VR technology already allows for some of that to happen, Pimax wants to take it to the next level with its new invention -- the Reality 12K QLED VR headset.

Read more