Facebook wants to transport your physical self into the virtual world by allowing you to create photorealistic avatars for virtual reality headsets. The technology was unveiled by Facebook chief technology officer Mike Schroepfer during the second day of the company’s developer-centric F8 conference in Silicon Valley, California.
The company has made progress over the years to evolve its avatar creation technology to make digital representations more lifelike and emotive. Facebook originally represented avatars as a simple blue face, but the technology eventually allowed users to personalize their virtual selves with more details and lifelike features. The result is still a cartoon-like representation of an individual, and it is not dissimilar to Snap’s personalized Bitmoji or Samsung’s AR emoji. Creating a photorealistic avatar is a logical next step for Facebook’s VR journey, as the company hopes that advancing technologies will help blur the borders between the real and virtual worlds.
Facebook didn’t detail much about its work with photorealistic avatars. We learned from demos that the company is using motion-capture technology to map facial images from photographs to capture various points on a user’s face. By carefully mapping a user’s face and facial characteristics, Facebook can synchronize facial movements and expressions in real life with the photorealistic avatar in VR.
In late 2016, Facebook began experimenting with bringing more emotions into its VR avatar experience, according to Techristic. The company used body tracking to allow users to show that they were racing their fists in the air, demonstrating an angry expression, or a shoulder shrug. Photorealistic avatars with face tracking could allow Facebook to inject even more emotions into our digital personas in the virtual world. During his demo, Schroepfer showed that when an Oculus employee with a Rift headset spoke, his avatar’s mouth moved in unison with his speech.
Still, we don’t know when photorealistic avatars will arrive for consumers. New hardware may also be required, according to Upload VR, that will allow Oculus to track eye movements, which could allow our avatars to blink when we do. Similarly, Midmaze has been working on its sensor mask, which is designed to sit between your face and a VR headset, like the Oculus Rift, to track your facial movements and display them in real time in the virtual world.
Photorealistic avatars could help to further define and transform the virtual and augmented reality experiences that we have today, allowing users from remote locations to experience games and movies together as if they’re in the same room. If you look over and see a photorealistic image of your friend enjoying the same experience as you in the VR space, it would make virtual experiences feel more convincing.
