With 102 cameras, Metapixel can create photorealistic 3D models in 30 minutes

Robbie Cooper, co-founder of Metapixel, has a vision. “I want fully animated, believable characters in a VR game or environment that react in a totally natural manner. To have that feeling of wanting to reach out and touch them because they’re so real. That’s how a VR world should be.”

He’s not just dreaming about it: He and the Metapixel team are actually working towards making it possible. Using an incredible camera rig and some custom-made software, Metapixel creates astonishingly lifelike avatars with photo-realistic features, based on high-resolution images of an actual person, ready for use in games, movies, and beyond. We visited Metapixel’s office in Brighton, U.K. to see the technology in action, and have our own avatar created.

Photo-real avatar

The camera rig looks like a portal to another dimension; a single click of the shutter is all it will take to zap you across galaxies or through time. There are 102 Nikon D810 cameras mounted on it, all equipped with one of three different lenses — a 50mm, an 85mm, or a 72-300mm zoom.

There’s no galaxy zapping involved, of course, but once that shutter is released, each camera takes a 36-megapixel RAW-format picture of the subject from almost every conceivable angle. Those shots are then transferred to Metapixel’s computer system where they are processed in a matter of seconds. Over the course of the next hour or so, the software creates an amazing photo-real avatar on screen, looking exactly like the subject. It’s actually a technique called photogrammetry, where photos taken from different viewpoints are stitched together to form a single, 360-degree render similar to a 3D body scan.

And when I say exactly like the subject, I mean it. The result is uncanny. To test it out, I was invited to stand in the middle of the rig, and have my image immortalized forever. It’s likely the first and only time 102 cameras will be pointed in my direction at the same time. Told to stare at a single point and keep completely still, with my arms out in a slightly uncomfortable manner, I waited as the cameras were primed — then I was blinded by a flash as my photo was taken.

The more realistic the avatar looks and acts, the more you’re engaged with what’s going on.

Each picture is carefully checked for errors, odd lighting, and shadows (which can confuse the computer and ruin the end image), and for any missing images where a camera for some reason misfired. Metapixel’s own software, called Capturing Reality, then does all the heavy lifting. It pulls together each photo and maps the image on a mesh, building up the final avatar. It took 10 to 15 minutes to build up the first stage, then another 20 to add the texture and color.

You end up with the picture seen here, an avatar so real it’s frightening. It’s also considered only “normal” quality. There are other stages, which increase the level of detail even further. The whole process took less than an hour, including all the camera setup and the processing.

The possibilities

What can be done with your picture-perfect avatar? There are masses of exciting possibilities, but for Cooper, it’s all about virtual reality. “Combine this with really good artificial intelligence, and the world becomes real,” he said. For example as AI improves, he told me, avatars could understand where you’re “looking” in a VR game and respond accordingly. The more realistic the avatar looks and acts, the more you’re engaged with what’s going on.

He pointed out the work of animation studio 3Lateral, where by using face-tracking technology, photo-real avatars mirror the facial expressions and reactions of an actor, as proof of what’s already being worked on. To come face-to-face with what’s almost a living, breathing person in a VR game or environment has the potential to change the way we interact and empathize with in-game characters.

There’s also exciting potential in movies. An actor who is scanned this way becomes essentially immortal. Imagine actors continuing to work even after their death, or calling up a scan from years ago to act out a scene in which the actor is required to look younger — all this is possible with a digitally reproduced, identical-looking avatar. It goes a step beyond 3D animation, and can use the techniques seen in motion capture.

Combine the scan with a 3D printer and you’ve suddenly got absolutely perfect looking action figures, too. Bring the technology into museums and teaching environments, and you return to Cooper’s vision of characters you believe in. You may someday see and hear a historical person tell the tale of their lives, with true emotion registering on a face that appears to be completely human. It’s exciting stuff, and the possibilities keep occurring the more you think about it.

Almost reality

We’re still a while away from seeing all this in reality. Metapixel has come a long way very quickly; the company only built its first camera rig, with 30 cameras at the time, at the end of 2015. While it has expanded, the custom software is still being worked on, and there are upgrades to make the photo-taking process more efficient coming soon. Besides, VR itself — the key driving tech for Metapixel — is still in its infancy.

Metapixel is attracting plenty of attention for such a young company, and it is working with a movie production company, VR developers, and even conference organizers to supply personal, full-body scans on a USB stick to all participants. Imagine having the chance to use that avatar at a conference held in virtual reality in the future. The results are impressive and exciting for what’s really an early-stage technology. We can’t wait to see it evolve.

Emerging Tech

How emotion-tracking A.I. will change computing as we know it

Affectiva is just one of the startups working to create emotion-tracking A.I. that can work out how you're feeling. Here's why this could change the face of computing as we know it.
Movies & TV

'Prime'-time TV: Here are the best shows on Amazon Prime right now

There's more to Amazon Prime than free two-day shipping, including access to a number of phenomenal shows at no extra cost. To make the sifting easier, here are our favorite shows currently streaming on Amazon Prime.

This list of PlayStation 4 exclusives puts its competitors to shame

The PlayStation 4's game library and incredible selection of exclusive games could make anyone with an Xbox One or Nintendo Switch think twice. Here's our list of the latest and greatest PS4 exclusives.
Movies & TV

The best shows on Netflix right now (April 2019)

Looking for a new show to binge? Lucky for you, we've curated a list of the best shows on Netflix, whether you're a fan of outlandish anime, dramatic period pieces, or shows that leave you questioning what lies beyond.

Sweet 16: Wacom’s Cintiq 16 pen display makes retouching photos a breeze

Wacom’s Cintiq pen displays are usually reserved for the pros (or wealthy enthusiasts), but the new Cintiq 16 brings screen and stylus editing to an approachable price. Does it cut too much to get there?

China bans selfies at gigantic Aperture Spherical Telescope

You can't take a selfie with the world's largest single-dish radio telescope anymore, as the Chinese government has banned everything from smartphones to digital cameras in the surrounding 5-kilometer area.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Robots that eat landmines and clean your floors

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it's fun to gawk!

Light on price but rich on features, these are the best cameras for students

Need pro-level features on a budget? The best cameras for students mix advanced features with a more palatable price point. From $2K entry level full frame cameras to $600 budget picks, here are five of the best cameras for students.

After controversial video, China bans ‘Leica’ on social media

A video that referenced Tiananmen Square got the name of the camera company Leica banned from the social media platform Weibo. Leica says the video wasn't an officially sanctioned promotion.

Photography News: Instagram’s disappearing likes, the best photos of the year

In this week's Photography News, see why Instagram is testing a version that excludes the number of likes a post gets. Also, see the impressive winners from two photography contests and the latest features coming to the Fujifilm X-T3.

Panasonic Lumix S1R vs. Sony A7R III: Which pixel-shift powerhouse is better?

The Lumix S1R and Sony A7R III are different in design, but similar in performance, and both offer pixel-shift high resolution modes which pull even more detail out of their already high-resolution sensors. Here's how they compare.

Earn more likes on your photos with the best cameras for Instagram

Looking to snap better Instagrams? Instagram doesn't demand high-resolution files, but upgrading your camera can deliver better bokeh and low-light quality. Here our the best cameras for Instagram.

Capture life in every direction with the best 360 cameras

While 360 cameras are still a new technology, that doesn't mean there's not a few that are worth a look. Whether you want to shoot from the middle or just need a simple, affordable option, here are the best 360 cameras on the market.

These point-and-shoot cameras make your smartphone pics look like cave paintings

If your smartphone camera isn't giving you the results you want, maybe it's time to step up your game. The latest and greatest point-and-shoot cameras offer large sensors, tough bodies, and long lenses -- something no phone can touch.