Skip to main content

State-of-the-art scanning technique digitizes humans, sticks them in VR

Using an array of high-resolution cameras, scientists at Fraunhofer Heinrich Hertz Institute (HHI) in Berlin, Germany have developed a state-of-the-art scanning technique to capture realistic images of a person and transmit them to a virtual reality environment. The technology, which will be on display at IFA 2017 in Berlin, may make human actors in virtual and augmented reality experiences even more lifelike.

“Today you usually use animated avatars to model a person,” Ingo Feldmann, head of the Immersive Media and Communication group at Fraunhofer HHI, told Digital Trends. “So you create a model of a person in a 3D map and somehow capture the motion from a real actor and then you try to animate this model.”

Related Videos

The method developed by Feldmann and his team instead record high-resolution video using industrial stereo cameras, which capture an image with two lenses, much like a pair of human eyes. The array is made up of 32 of these cameras (Feldmann was tight-lipped about about the camera models) allowing it to record a subject’s movements in 360 degrees, creating a detailed and dynamic representation.

“We film the person directly in 3D,” Feldmann said, “in what we call volumetric video. There are cameras arranged around this actor … and these cameras create 2D images. From these images we create a 3D structure or model that looks completely natural. There are no animations, it’s the real person, and they can be inserted into a VR or AR experience.”

Of course, there’s a lot that goes into inserting the recorded video into a virtual world but, to make the task easier, the researchers created algorithms that can quickly extract the video’s depth data to calculate how the 3D figure should appear. Feldmann said the camera array is capable of capturing some 300 million triangles per person — far too many for a VR device to accommodate — so they must also reduce the complexity of the model.

“We have to adapt the resolution to the device,” he said.

The scientists have now opened a studio at Fraunhofer HHI where people can come to record models for VR and AR.

IFA 2016 was host to another state-of-the-art scanner, Botscan, which used 70 high-resolution DSLRs to capture an incredibly detailed, albeit static, 360-degree image.

Editors' Recommendations

The best gaming monitors for 2023
Overwatch 2 running on the LG OLED 27 gaming monitor.

If you're looking for the best gaming monitor in 2023, Alienware's excellent 34 QD-OLED still takes the cake. However, it's not the perfect monitor for all gamers. We've reviewed dozens of monitors to find the top gaming displays you can buy right now, regardless of if you're chasing peak HDR experiences or high refresh rates for competitive titles.

We're focused specifically on gaming monitors here, which come with higher refresh rates and adaptive sync features like G-Sync and FreeSync. If you're looking for an all-around display, make sure to browse our list of the best monitors.

Read more
Watch ChatGPT come to life by powering this holographic AI companion
Looking Glass CEO, Shawn Frayne asks the holographic AI to complete the lyrics to the Rick Astley song Never Gonna Give You Up

ChatGPT is quickly being developed beyond its standard functionality on browsers and computer-based programs. One company has even created a "holographic AI companion" that uses the chatbot to bring its vision to life.

The company called Looking Glass recently shared on Twitter several demos of people interacting with its holographic AI companion, called Uncle Rabbit, which is able to communicate back-and-forth in real time with humans, while also completing tasks that people request.

Read more
This is how you can accidentally kill AMD’s best CPU for gaming
Someone holding the Ryzen 7 5800X3D in a red light.

It turns out that one of AMD's best gaming CPUs, the Ryzen 7 5800X3D, can accidentally be killed if you try to overclock it, and it's all because there are no limitations as to how far you can push the processor.

Igor Wallossek of Igor's Lab found that the software used for overclocking and overvolting Ryzen CPUs currently doesn't impose any limits when you try to ramp up the voltage. And that's a recipe for turning a fun performance boost into an overclocking nightmare.

Read more