Skip to main content

With 102 cameras, Metapixel can create photorealistic 3D models in 30 minutes

Robbie Cooper, co-founder of Metapixel, has a vision. “I want fully animated, believable characters in a VR game or environment that react in a totally natural manner. To have that feeling of wanting to reach out and touch them because they’re so real. That’s how a VR world should be.”

He’s not just dreaming about it: He and the Metapixel team are actually working towards making it possible. Using an incredible camera rig and some custom-made software, Metapixel creates astonishingly lifelike avatars with photo-realistic features, based on high-resolution images of an actual person, ready for use in games, movies, and beyond. We visited Metapixel’s office in Brighton, U.K. to see the technology in action, and have our own avatar created.

Photo-real avatar

The camera rig looks like a portal to another dimension; a single click of the shutter is all it will take to zap you across galaxies or through time. There are 102 Nikon D810 cameras mounted on it, all equipped with one of three different lenses — a 50mm, an 85mm, or a 72-300mm zoom.

There’s no galaxy zapping involved, of course, but once that shutter is released, each camera takes a 36-megapixel RAW-format picture of the subject from almost every conceivable angle. Those shots are then transferred to Metapixel’s computer system where they are processed in a matter of seconds. Over the course of the next hour or so, the software creates an amazing photo-real avatar on screen, looking exactly like the subject. It’s actually a technique called photogrammetry, where photos taken from different viewpoints are stitched together to form a single, 360-degree render similar to a 3D body scan.

And when I say exactly like the subject, I mean it. The result is uncanny. To test it out, I was invited to stand in the middle of the rig, and have my image immortalized forever. It’s likely the first and only time 102 cameras will be pointed in my direction at the same time. Told to stare at a single point and keep completely still, with my arms out in a slightly uncomfortable manner, I waited as the cameras were primed — then I was blinded by a flash as my photo was taken.

The more realistic the avatar looks and acts, the more you’re engaged with what’s going on.

Each picture is carefully checked for errors, odd lighting, and shadows (which can confuse the computer and ruin the end image), and for any missing images where a camera for some reason misfired. Metapixel’s own software, called Capturing Reality, then does all the heavy lifting. It pulls together each photo and maps the image on a mesh, building up the final avatar. It took 10 to 15 minutes to build up the first stage, then another 20 to add the texture and color.

You end up with the picture seen here, an avatar so real it’s frightening. It’s also considered only “normal” quality. There are other stages, which increase the level of detail even further. The whole process took less than an hour, including all the camera setup and the processing.

The possibilities

What can be done with your picture-perfect avatar? There are masses of exciting possibilities, but for Cooper, it’s all about virtual reality. “Combine this with really good artificial intelligence, and the world becomes real,” he said. For example as AI improves, he told me, avatars could understand where you’re “looking” in a VR game and respond accordingly. The more realistic the avatar looks and acts, the more you’re engaged with what’s going on.

He pointed out the work of animation studio 3Lateral, where by using face-tracking technology, photo-real avatars mirror the facial expressions and reactions of an actor, as proof of what’s already being worked on. To come face-to-face with what’s almost a living, breathing person in a VR game or environment has the potential to change the way we interact and empathize with in-game characters.

There’s also exciting potential in movies. An actor who is scanned this way becomes essentially immortal. Imagine actors continuing to work even after their death, or calling up a scan from years ago to act out a scene in which the actor is required to look younger — all this is possible with a digitally reproduced, identical-looking avatar. It goes a step beyond 3D animation, and can use the techniques seen in motion capture.

Combine the scan with a 3D printer and you’ve suddenly got absolutely perfect looking action figures, too. Bring the technology into museums and teaching environments, and you return to Cooper’s vision of characters you believe in. You may someday see and hear a historical person tell the tale of their lives, with true emotion registering on a face that appears to be completely human. It’s exciting stuff, and the possibilities keep occurring the more you think about it.

Almost reality

We’re still a while away from seeing all this in reality. Metapixel has come a long way very quickly; the company only built its first camera rig, with 30 cameras at the time, at the end of 2015. While it has expanded, the custom software is still being worked on, and there are upgrades to make the photo-taking process more efficient coming soon. Besides, VR itself — the key driving tech for Metapixel — is still in its infancy.

Metapixel is attracting plenty of attention for such a young company, and it is working with a movie production company, VR developers, and even conference organizers to supply personal, full-body scans on a USB stick to all participants. Imagine having the chance to use that avatar at a conference held in virtual reality in the future. The results are impressive and exciting for what’s really an early-stage technology. We can’t wait to see it evolve.

Editors' Recommendations

Andy Boxall
Senior Mobile Writer
Andy is a Senior Writer at Digital Trends, where he concentrates on mobile technology, a subject he has written about for…
How a former ILM designer now creates in 3D with just a laptop
iron man designer creates on msi laptop computex 2019 ilm single

The Star Wars prequels and original trilogy releases were some of the first big films to use 3D modeling to great effect for a wide audience. These types of films are still being made today, but the technology behind the designs have changed a lot in the past twenty years.

Nestled in MSI’s booth at Computex, we got a peek into just how far that process has come. In what was one of the most riveting demonstrations we’d ever seen at the show, a former designer at ILM (Industrial Light & Magic) explained just how much his process has changed over the years. 
3D designs in a matter of minutes
Some of Colie Wertz's conceptual sketches.

Read more
Even a true rock god can’t smash this unbreakable 3D-printed metal guitar
unbreakable 3d printed metal guitar

Sandvik Let’s Create: The Smash-Proof Guitar, tested by Yngwie Malmsteen

Rock stars love to smash guitars apart. Geeky engineers (and we say that with the utmost respect) love to build things. What happens when both parties meet? Probably something like the 3D-printed guitar developed by global engineering firm Sandvik. Not only does this all-metal shredding beast look like one of the coolest guitars we’ve seen in ages, but its creators claim that it is entirely unbreakable as well.

Read more
This $30 screen protector creates 3D views on a smartphone without the glasses
lucid holotech holoscreen 3d screen protector 3dpicturetaking copy

Can a simple screen protector deliver 3D content without the glasses -- or the expensive holographic smartphone? Lucid and Holitech think so. The Holoscreen is a smartphone screen protector that allows users to view 3D content without the glasses inside an app. The $30 accessory functions as a normal screen protector outside of the 3D app.

When viewing 2D content outside the app, the companies say that the screen protector doesn’t alter the usual smartphone screen, serving as just the usual thin-film style screen protector. The 3D images and video, on the other hand, are sharp and appear to almost pop out of the screen, according to Holitech and Lucid.

Read more