With 102 cameras, Metapixel can create photorealistic 3D models in 30 minutes

Robbie Cooper, co-founder of Metapixel, has a vision. “I want fully animated, believable characters in a VR game or environment that react in a totally natural manner. To have that feeling of wanting to reach out and touch them because they’re so real. That’s how a VR world should be.”

He’s not just dreaming about it: He and the Metapixel team are actually working towards making it possible. Using an incredible camera rig and some custom-made software, Metapixel creates astonishingly lifelike avatars with photo-realistic features, based on high-resolution images of an actual person, ready for use in games, movies, and beyond. We visited Metapixel’s office in Brighton, U.K. to see the technology in action, and have our own avatar created.

Photo-real avatar

The camera rig looks like a portal to another dimension; a single click of the shutter is all it will take to zap you across galaxies or through time. There are 102 Nikon D810 cameras mounted on it, all equipped with one of three different lenses — a 50mm, an 85mm, or a 72-300mm zoom.

There’s no galaxy zapping involved, of course, but once that shutter is released, each camera takes a 36-megapixel RAW-format picture of the subject from almost every conceivable angle. Those shots are then transferred to Metapixel’s computer system where they are processed in a matter of seconds. Over the course of the next hour or so, the software creates an amazing photo-real avatar on screen, looking exactly like the subject. It’s actually a technique called photogrammetry, where photos taken from different viewpoints are stitched together to form a single, 360-degree render similar to a 3D body scan.

And when I say exactly like the subject, I mean it. The result is uncanny. To test it out, I was invited to stand in the middle of the rig, and have my image immortalized forever. It’s likely the first and only time 102 cameras will be pointed in my direction at the same time. Told to stare at a single point and keep completely still, with my arms out in a slightly uncomfortable manner, I waited as the cameras were primed — then I was blinded by a flash as my photo was taken.

The more realistic the avatar looks and acts, the more you’re engaged with what’s going on.

Each picture is carefully checked for errors, odd lighting, and shadows (which can confuse the computer and ruin the end image), and for any missing images where a camera for some reason misfired. Metapixel’s own software, called Capturing Reality, then does all the heavy lifting. It pulls together each photo and maps the image on a mesh, building up the final avatar. It took 10 to 15 minutes to build up the first stage, then another 20 to add the texture and color.

You end up with the picture seen here, an avatar so real it’s frightening. It’s also considered only “normal” quality. There are other stages, which increase the level of detail even further. The whole process took less than an hour, including all the camera setup and the processing.

The possibilities

What can be done with your picture-perfect avatar? There are masses of exciting possibilities, but for Cooper, it’s all about virtual reality. “Combine this with really good artificial intelligence, and the world becomes real,” he said. For example as AI improves, he told me, avatars could understand where you’re “looking” in a VR game and respond accordingly. The more realistic the avatar looks and acts, the more you’re engaged with what’s going on.

He pointed out the work of animation studio 3Lateral, where by using face-tracking technology, photo-real avatars mirror the facial expressions and reactions of an actor, as proof of what’s already being worked on. To come face-to-face with what’s almost a living, breathing person in a VR game or environment has the potential to change the way we interact and empathize with in-game characters.

There’s also exciting potential in movies. An actor who is scanned this way becomes essentially immortal. Imagine actors continuing to work even after their death, or calling up a scan from years ago to act out a scene in which the actor is required to look younger — all this is possible with a digitally reproduced, identical-looking avatar. It goes a step beyond 3D animation, and can use the techniques seen in motion capture.

Combine the scan with a 3D printer and you’ve suddenly got absolutely perfect looking action figures, too. Bring the technology into museums and teaching environments, and you return to Cooper’s vision of characters you believe in. You may someday see and hear a historical person tell the tale of their lives, with true emotion registering on a face that appears to be completely human. It’s exciting stuff, and the possibilities keep occurring the more you think about it.

Almost reality

We’re still a while away from seeing all this in reality. Metapixel has come a long way very quickly; the company only built its first camera rig, with 30 cameras at the time, at the end of 2015. While it has expanded, the custom software is still being worked on, and there are upgrades to make the photo-taking process more efficient coming soon. Besides, VR itself — the key driving tech for Metapixel — is still in its infancy.

Metapixel is attracting plenty of attention for such a young company, and it is working with a movie production company, VR developers, and even conference organizers to supply personal, full-body scans on a USB stick to all participants. Imagine having the chance to use that avatar at a conference held in virtual reality in the future. The results are impressive and exciting for what’s really an early-stage technology. We can’t wait to see it evolve.

Emerging Tech

Awesome Tech You Can’t Buy Yet: Robo sidekicks, AC for your bed, and more

Check out our roundup of the best new crowdfunding projects and product announcements that hit the Web this week. You can't buy this stuff yet, but it sure is fun to gawk!
Movies & TV

'Prime'-time TV: Here are the best shows on Amazon Prime right now

Amazon Prime brings more perks than just free two-day shipping. Subscribers get access to a huge library of TV shows to stream at no extra cost. Here are our favorite TV shows currently available on Amazon Prime.

Sixth public beta of iOS 12 still lacks one key feature

At this year's Worldwide Developer Conference, Apple unveiled its latest operating system, iOS 12. From app updates to group FaceTime, ARKit 2.0, and more, here are all the new features in iOS 12.
Movies & TV

David Harbour is 'taking risks' with Jim Hopper in 'Stranger Things' season 3

With a sophomore season as strong as its first, Stranger Things is now moving on to season 3. Here's everything we've learned so far about the Netflix series' upcoming third season.

The Nixplay Iris might just make digital picture frames cool again

The digital picture frame's popularity has fizzled because of time-consuming updates and low quality -- but can a Wi-Fi connected frame change that? The Nixplay Iris is an 8-inch smart digital picture frame that wireless updates photos.

8 easy ways for you to transfer photos from an Android phone to a PC

If you haven't already, you should back up your photos to a computer. Here's how to transfer photos from an Android phone to a PC using third-party services and a wealth of storage devices.

A turn for the better: Loupedeck+ adds custom dials, more to Lightroom console

The Loupedeck+ improves on the original Lightroom console by adding welcome customization options and introducing support for Skylum Aurora HDR. What's even better is that it does this all at an even lower price.

These are the best camera straps you can get your hands on

Choosing the right camera strap for your needs can be a tough decision. To help sort through the junk, we've rounded up the best camera straps on the market for you to choose from.
Social Media

How to use Adobe Spark Post to spice up your social media images

Images are proven to get more likes than plain text -- but only if those images are good. Adobe Spark post is an AI-powered design program for non-designers. Here's how to use it to take your social media feeds to the next level.

18 seconds of fame: Giphy’s quirky new GIF film festival has a $10K prize

Have a knack for creating GIFs? Someone is going to win $10,000 for a video that's 18 seconds or less at the Giphy Film Festival. The new contest is set to recognize the top creator for the short micro-films in five different categories.

Google One subscriptions offer more cloud storage for low prices, other perks

Can't get enough storage on Google Drive, Photos, or Gmail? Google One is the new way to boost your cloud storage. But it's not just about more space -- Google One comes with a loads of benefits.

Sorry, DSLR, but Sony mirrorless is now the best-selling full-frame camera

Four out of every ten full-frame cameras sold in the United States during the first-half of 2018 had the Sony name. That makes Sony's mirrorless full-frame cameras the best-selling models.
Emerging Tech

Meet the Mantis Q: A drone you can control by yelling, waving, or even smiling

"Mantis, take a picture." Yuneec's new consumer drone, the Yuneec Mantis Q, responds to voice commands along with gestures and smiles. The 4K drone also integrates several different flight modes and safety features inside a one-pound…

Capture life in every direction with the best 360 cameras

While 360 cameras are still a new technology, that doesn't mean there's not a few that are worth a look. Whether you want to shoot from the middle or just need a simple, affordable option, here are the best 360 cameras on the market.