Disney Research develops method to capture 3D facial performances with a single camera

disney research single camera facial performance capture
Disney Research
Capturing an actor’s facial performance in a three-dimensional environment is key to believable animations in movies and video games. The process of accurately modeling a performer’s face — and their full range of expressions — is no easy task, however. But with a new method developed by Disney Research, it’s about to become much simpler.

Unlike traditional facial-performance-capture techniques, which rely on multiple cameras to read depth information, this new method uses only a single camera. And no, it doesn’t require a light-field camera or any other non-traditional imaging device. In fact, the researchers even demonstrated it using both a GoPro and an iPhone, according to a press release first seen on Phys.org.

Single-camera facial-performance capture isn’t completely new, but existing methods require a complex computer model of an actor’s face to be developed ahead of time. That model has to be built with many different expressions programmed into it, otherwise the capture process will generate too many anomalies.

With Disney Research’s new method, information about bone structure and skin thickness is taken into account, which provides an “anatomically constrained” model, thereby keeping the range of available animations limited to what’s physically possible, without the need for as many pre-recorded expressions. The effectiveness of the method was tested by capturing an actor’s face being deformed by a jet of compressed air, something that simply wouldn’t be possible with other single-camera solutions.

Smaller, less complex hardware should make facial performance capture more accessible and easier to do. A single-camera system also allows actors increased freedom of movement during their performances. While typically there exists an inverse relationship between simplicity and capture quality, Disney researchers say their new method bucks the trend.

“No hardware setup could be simpler than our new one-camera method, yet we’ve shown that it can obtain results that rival, if not exceed, more traditional methods,” Markus Gross, vice president of Disney Research, said in the press release.

The project will be presented on July 24 at the ACM International Conference on Computer Graphics & Interactive Techniques (SIGGRAPH) in California.

Product Review

From tables to porches, Ring's Stick Up Cam watches wherever you want

Ring’s latest home security camera offers more versatility than ever before, with the ability to mount it, set it on a table, or put it inside or outside. We got a chance to test it out. Here’s how it went.
Photography

From DSLRs to mirrorless, these are the best cameras you can buy right now

From entry-level models to full-frame flagships, many cameras take great photos and video. The best digital cameras, however, push the industry forward with innovative sensors and improved usability, among other things. Here are our…
Product Review

Canon democratizes full-frame with the EOS RP, but keep your expectations low

At just $1,300, the RP is Canon's least expensive full-frame camera yet, but it was born into a world of high-end, high-cost lenses where it doesn't yet feel at home.
Photography

Nikon will bring eye-detection autofocus to the Z6 and Z7 in May

An upcoming firmware update will bring Eye AF to the Nikon Z6 and Z7 -- along with improved autofocus performance in low light. The update will also give the cameras support for the CFexpress format.
Emerging Tech

Descending at an angle could be key to landing heavier craft on Mars

Landing on Mars is a challenge: The heavier the craft, the more difficult a safe landing becomes. Scientists propose using retropropulsion engines and angling the craft to create a pressure differential to land heavier crafts in the future.
Emerging Tech

Ant-inspired walking robot navigates without GPS by using polarized light

What do you get if you cross Boston Dynamics and Ant-Man? You get Antbot, a robot from the French National Center for Scientific Research (CNRS) which uses ant-like navigation to move around without the aid of GPS.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Grow veggies indoors and shower more efficiently

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Emerging Tech

InSight’s heat probe will dig 16 feet beneath the surface of Mars

New images from NASA's InSight mission to Mars have confirmed that the lander succeeded in setting the Heat Flow and Physical Properties Package instrument onto the surface, from where a self-hammering spike will burrow downwards.
Emerging Tech

White spots on Ceres are evidence of ancient ice volcanoes erupting

Scientists are pouring over data collected by NASA's Dawn mission to learn about the dwarf planet Ceres and the bright white spots observed at the bottom of impact craters. They believe that these spots are evidence of ice volcanoes.
Emerging Tech

NASA to launch SPHEREx mission to investigate the origins of our universe

NASA is launching an ambitious mission to map the entire sky to understand the origins of the universe. The Spectro-Photometer for the History of the Universe, Epoch of Reionization and Ices Explorer (SPHEREx) mission will launch in 2023.
Emerging Tech

Probes exploring Earth’s hazardous radiation belts enter final phase of life

The Van Allen probes have been exploring the radiation belts around Earth for seven years. Now the probes are moving into the final phase of their exploration, coming closer to Earth to gather more data before burning up in the atmosphere.
Emerging Tech

How can digital art created on obsolete platforms be preserved?

As the lines between art and technology continue to blur, digital art experiences become more commonplace. But these developments are raising an important question for art conservationists: How should digital artworks be preserved?
Emerging Tech

Statistician raises red flag about reliability of machine learning techniques

Machine learning is everywhere in science and technology. But how reliable are these techniques really? A statistician argues that questions of accuracy and reproducibility of machine learning have not been fully addressed.
Emerging Tech

Chandra X-ray telescope uncovers evidence of the universe’s missing matter

Where is all of the matter in the universe? NASA's Chandra telescope has uncovered evidence of hot gas strands in the vicinity of a quasar which could explain the missing third of matter which has puzzled astronomers for years.