With 102 cameras, Metapixel can create photorealistic 3D models in 30 minutes

Robbie Cooper, co-founder of Metapixel, has a vision. “I want fully animated, believable characters in a VR game or environment that react in a totally natural manner. To have that feeling of wanting to reach out and touch them because they’re so real. That’s how a VR world should be.”

He’s not just dreaming about it: He and the Metapixel team are actually working towards making it possible. Using an incredible camera rig and some custom-made software, Metapixel creates astonishingly lifelike avatars with photo-realistic features, based on high-resolution images of an actual person, ready for use in games, movies, and beyond. We visited Metapixel’s office in Brighton, U.K. to see the technology in action, and have our own avatar created.

Photo-real avatar

The camera rig looks like a portal to another dimension; a single click of the shutter is all it will take to zap you across galaxies or through time. There are 102 Nikon D810 cameras mounted on it, all equipped with one of three different lenses — a 50mm, an 85mm, or a 72-300mm zoom.

There’s no galaxy zapping involved, of course, but once that shutter is released, each camera takes a 36-megapixel RAW-format picture of the subject from almost every conceivable angle. Those shots are then transferred to Metapixel’s computer system where they are processed in a matter of seconds. Over the course of the next hour or so, the software creates an amazing photo-real avatar on screen, looking exactly like the subject. It’s actually a technique called photogrammetry, where photos taken from different viewpoints are stitched together to form a single, 360-degree render similar to a 3D body scan.

And when I say exactly like the subject, I mean it. The result is uncanny. To test it out, I was invited to stand in the middle of the rig, and have my image immortalized forever. It’s likely the first and only time 102 cameras will be pointed in my direction at the same time. Told to stare at a single point and keep completely still, with my arms out in a slightly uncomfortable manner, I waited as the cameras were primed — then I was blinded by a flash as my photo was taken.

The more realistic the avatar looks and acts, the more you’re engaged with what’s going on.

Each picture is carefully checked for errors, odd lighting, and shadows (which can confuse the computer and ruin the end image), and for any missing images where a camera for some reason misfired. Metapixel’s own software, called Capturing Reality, then does all the heavy lifting. It pulls together each photo and maps the image on a mesh, building up the final avatar. It took 10 to 15 minutes to build up the first stage, then another 20 to add the texture and color.

You end up with the picture seen here, an avatar so real it’s frightening. It’s also considered only “normal” quality. There are other stages, which increase the level of detail even further. The whole process took less than an hour, including all the camera setup and the processing.

The possibilities

What can be done with your picture-perfect avatar? There are masses of exciting possibilities, but for Cooper, it’s all about virtual reality. “Combine this with really good artificial intelligence, and the world becomes real,” he said. For example as AI improves, he told me, avatars could understand where you’re “looking” in a VR game and respond accordingly. The more realistic the avatar looks and acts, the more you’re engaged with what’s going on.

He pointed out the work of animation studio 3Lateral, where by using face-tracking technology, photo-real avatars mirror the facial expressions and reactions of an actor, as proof of what’s already being worked on. To come face-to-face with what’s almost a living, breathing person in a VR game or environment has the potential to change the way we interact and empathize with in-game characters.

There’s also exciting potential in movies. An actor who is scanned this way becomes essentially immortal. Imagine actors continuing to work even after their death, or calling up a scan from years ago to act out a scene in which the actor is required to look younger — all this is possible with a digitally reproduced, identical-looking avatar. It goes a step beyond 3D animation, and can use the techniques seen in motion capture.

Combine the scan with a 3D printer and you’ve suddenly got absolutely perfect looking action figures, too. Bring the technology into museums and teaching environments, and you return to Cooper’s vision of characters you believe in. You may someday see and hear a historical person tell the tale of their lives, with true emotion registering on a face that appears to be completely human. It’s exciting stuff, and the possibilities keep occurring the more you think about it.

Almost reality

We’re still a while away from seeing all this in reality. Metapixel has come a long way very quickly; the company only built its first camera rig, with 30 cameras at the time, at the end of 2015. While it has expanded, the custom software is still being worked on, and there are upgrades to make the photo-taking process more efficient coming soon. Besides, VR itself — the key driving tech for Metapixel — is still in its infancy.

Metapixel is attracting plenty of attention for such a young company, and it is working with a movie production company, VR developers, and even conference organizers to supply personal, full-body scans on a USB stick to all participants. Imagine having the chance to use that avatar at a conference held in virtual reality in the future. The results are impressive and exciting for what’s really an early-stage technology. We can’t wait to see it evolve.

Smart Home

Picture this: The Aura packs thousands of photos in a single frame (for a price)

Are you one of those people who miss the good old days of flipping through photo albums to see each and every favorite photo? If so, you might love the Aura digital photo frame. We tested the device and came away impressed.
Virtual Reality

Think virtual reality is just for games? These awesome apps will change your mind

Virtual reality isn't all about gaming. Swim with turtles, paint in 3D, and immerse yourself in some unique experiences the platform has to offer with our curated list of the best VR apps.
Movies & TV

'Prime'-time TV: Here are the best shows on Amazon Prime right now

There's more to Amazon Prime than free two-day shipping, including access to a number of phenomenal shows at no extra cost. To make the sifting easier, here are our favorite shows currently streaming on Amazon Prime.
Virtual Reality

Prototype Valve VR headset leaked: HTC Vive challenger confirmed?

Leaked images revealed that a Valve VR headset is in development, even amid Valve's partnership with HTC for the HTC Vive. Sources confirmed the device, which may be bundled with a Half-Life VR game.
Movies & TV

The best shows on Netflix, from 'The Haunting of Hill House’ to ‘The Good Place’

Looking for a new show to binge? Lucky for you, we've curated a list of the best shows on Netflix, whether you're a fan of outlandish anime, dramatic period pieces, or shows that leave you questioning what lies beyond.

Full frame or 4K for less than $1K? These 4 older cameras still have a lot to offer

Looking for a great camera deal? Sometimes, you might be better off buying one that's a few years -- last generation's professional models may not cost much more than today's entry-level models.

Get up close and personal with this telephoto lens for your phone

Moment is replacing its aging 60mm telephoto lens with a new 58mm tele lens, redesigned from the ground up for the latest iPhone, Pixel, and Galaxy phones. Mount it onto the phone via a case and get closer with 2x optical zoom.

Golf ball-sized Lume Cube Air is a pocketable LED for photos and video

Off-camera lighting for smartphones and GoPros just got even smaller. Meet the Lume Cube Air, a smaller portable LED light designed for photos and videos that weighs only about two ounces.

Edit portraits with A.I. and adjust focus in the new ON1 Photo RAW 2019 editor

ON1 Photo RAW 2019 now has a dedicated tab for portraits that automatically recognizes faces to help with retouching. The update also brings a new focus stacking tool, enhancements to layers, and improvements to local adjustments.

Taking shots in the dark with Night Sight, the Pixel’s newest photo feature

The Google Pixel range has always been the home of some of the mobile world's best phone cameras. That performance is now getting even better with the introduction of the low-light Night Sight mode.

Alpha Female: Sony awards five women grants to support artisan diversity

Women can face several challenges in launching a photography career -- Sony's latest initiative aims to help propel women in the industry forward. Sony recently announced the winners of the Alpha Female program.
Emerging Tech

The best drone photos from around the world

Most of today's drones come equipped with high-end cameras, which are quickly revolutionizing the world of aerial photography as we know it. Here are some of the best drone photos from around the world.
Product Review

Fewer pixels, better camera? The Nikon Z6 shows the beauty of restraint

The Nikon Z6 is the sibling to the new mirrorless Z7 -- but for some photographers, the cheaper Z6 may be the better option. Read where the $2,000 camera beats the $3,400 one (and where it doesn’t) in our Nikon Z6 review.
Social Media

Build a wish list and shop videos with Instagram’s latest shopping update

Eyeing a product on Instagram? Now there are more ways to shop from the social network. Instagram just rolled out options to save products in a collection as users can also now shop from videos.