How digital face mapping helped time (and Falcon) fly in The Winter Soldier

Five films are nominated for an Academy Award in the “Visual Effects” category this year, and they each offer a nice look at the amazing tricks filmmakers and their effects teams can pull off on the big screen. In recognition of these five films and one of our favorite Oscar categories, we’re putting the spotlight on one “Visual Effects” nominee each day leading up to Sunday’s broadcast, and taking a closer look at what made them stand out.

First up is one of the two Marvel Studios movies nominated this year, Captain America: The Winter Soldier, which made innovative use of digital face-mapping techniques to send one character flying through the air, and make another look her age.

In the past few years, superhero movies have become a popular outlet for visual-effects studios to show off their own superpowers. But amid all the explosions and larger-than-life sequences that offer no shortage of eye candy, it’s often the more subtle tricks of the trade that provide the biggest challenge – and generate the most pride when they’re pulled off successfully.

Marvel Studios’ Captain America: The Winter Soldier was packed to the brim with intense action sequences that culminated in a sequence featuring several flying aircraft carriers engaging in a massive firefight high above Washington, D.C. However, one of the most important VFX techniques used in the film had nothing to do with creating fiery havoc in the skies, and everything to do with getting up-close and personal with various cast members’ faces.

Early in the film, Captain America (Chris Evans) is shown visiting his former girlfriend, Peggy Carter, who has aged more than 70 years since he last saw her during the events of Captain America: The First Avenger. The pair share a tender moment as the still-young superhero and the woman he lost to time ponder what might have been, had he not been lost in the frozen sea for so many decades.

“Let’s try something that we’ve never done before.”

Actress Hayley Atwell admitted that when she intially read the script for the film, she knew that her character’s advanced age would likely mean she would be “up to the eyeballs in prosthetics.”

For Lola VFX, the visual-effects studio tasked with aging the 33-year-old actress more than 60 years, that was indeed the initial plan. But after several makeup and prosthetic trials, plans changed. The team originally planned to digitally merge Atwell’s makeup-free performance with sequences of her performing the same scene in various stages of makeup and prosthetics, but the process never quite blended the way the artists hoped.

“With each iteration of the test, the make-up artist would make the appliances thinner and thinner, and by the end was some of the best prosthetics I’d seen,” recalled Lola visual-effects supervisor Edson Williams in an interview with FX Guide. “But it still wasn’t quite what we were looking for.”

After hiring an older actress to perform the same scene in order to get a better idea of where the makeup went wrong, the team decided to try a new approach to making Peggy Carter look her age.

“I said, ‘Let’s try something that we’ve never done before,’” said Williams. “I wanted to take the performance of the elderly woman that we had shot in a rig with eight cameras and project the skin onto the original Hayley footage that had been shot on set.”

Digital effects artists used features from an older actress superimposed upon the 33-year-old actress playing Peggy Carter to convincingly age her decades. Marvel/Lola VFX

The end result was an emotional scene in which many of Atwell’s subtle gestures – the movement of her eyes and mouth, for example – are retained and visible beneath the digital overlay of the older actress’ skin and facial features, wrinkles and all. Perfecting the technique required them to record multiple versions of the older actress performing the scene, mimicking Atwell’s delivery of the lines, then having the sequences time-adjusted to fall perfectly in sync with each other before the digital mapping could even begin.

Once the scenes were in sync, portions of the older actress’ face could be mapped to Atwell’s face using digital editing techniques that matched tiny portions of animated points on each face and blended them together.

“We matched the lighting, the timing and performance and literally stole the creases and cracks and put them onto Hayley,” said Williams of the process.

For the film’s visual-effects supervisor, Dan DeLeeuw, that quiet scene is one of the elements he’s most proud of in The Winter Soldier.

“There’s only so much you can do to an actor before they just can’t take it anymore.”

“What we end up with, I think, is probably a game-changer in approaching old-age makeup,” he told HitFix during a recent interview.

That scene wasn’t the only part of the film to benefit from facial-mapping effects, either. One of the film’s most action-heavy sequences also required some extensive use of digital editing of an actor’s face.

During the climactic mid-air battle involving the massive S.H.I.E.L.D. helicarriers, Anthony Mackie’s character stages a death-defying assault on one of the hovering battleships using high-tech wings that allow him evade a heavily armed aircraft with fantastic acrobatic maneuvers. The fast-moving, action-heavy set piece could easily have been relegated to a full-on digitally rendered sequence, but instead, Mackie can be seen setting his jaw in mid-dive, grimacing at close calls, and otherwise clearly involved in the action.

“There’s only so much you can do to an actor before they just can’t take it anymore,” said Deleeuw of the sequence. “Initially most of the shots with Mackie were close-ups. We’ll shoot the close-up with him on the wire. Some of those shots where we couldn’t get him to hang right, we would just kind of remove his body and just keep his face – so even the close-up shots are digi doubles.”

In order to get the facial models just right, Mackie was required to stand in a “dome” of cameras that could record every angle of his expressions in order to properly replicate them as his character spins, flips, and careens through the sky. These facial models were then blended with the digitally created portions of the sequence, and later merged with the practical effects and wire-work sequences.

Actor Anthony Mackie performed some stunts “on the wire” in real life, and appears in other flying scenes by having his face digitally mapped to a virtual stand-in.
Actor Anthony Mackie performed some stunts “on the wire” in real life, and appears in other flying scenes by having his face digitally mapped to a virtual stand-in. Marvel

“We spent a lot of time in previs [a type of mockup] making sure we could imagine, for example, how we would shoot him if we were just going to photograph him,” recalled Deleeuw during an interview with Animation World Network. “If you truly had a guy with a rocket pack, and you had a rocket pack to follow him around, how would you photograph him? We tried to maintain that sense of realism and then seamlessly get those visual effects into the film.”

All of these techniques came together to form a wild, airborne chase sequence that helped make Mackie’s character one of the film’s standout stars – and the latest and greatest new superhero to emerge from the Marvel movie-verse.

Of course, if Captain America: The Winter Soldier and its visual-effects team takes home an Academy Award this year, Mackie’s high-flying Falcon won’t be Marvel Studios’ only new hero.

Come back tomorrow for another installment of our Oscar Effects series that looks at each of this year’s Academy Award visual-effects nominees.

Movies & TV

Oscar-winning FX master explains why ‘First Man’ is a giant leap for filmmaking

Paul Lambert, the Oscar-winning visual effects supervisor on First Man, reveals the innovative techniques that blended old footage with modern movie magic to make the Apollo 11 mission to the moon resonate with audiences 50 years later.
Movies & TV

The best new movie trailers: ‘Velvet Buzzsaw,’ ‘IO,’ Fyre Festival, and more

Everyone loves a good trailer, but keeping up with what's new isn't easy. That's why we round up the best ones for you. This week, it's trailers for a Fyre Festival documentary, as well as Velvet Buzzsaw and Iron Sky: The Coming Race.
Home Theater

The best movies on Netflix in December, from 'Buster Scruggs’ to 'Roma'

Save yourself from hours wasted scrolling through Netflix's massive library by checking out our picks for the streamer's best movies available right now, whether you're into explosive action, witty humor, or anything else.
Movies & TV

The best shows on Netflix, from 'Haunting of Hill House’ to ‘Norsemen’

Looking for a new show to binge? Lucky for you, we've curated a list of the best shows on Netflix, whether you're a fan of outlandish anime, dramatic period pieces, or shows that leave you questioning what lies beyond.
Home Theater

From the Roku Ultra to the Fire TV Cube, these are the best streaming devices

There are more options for media streamers than ever, so it’s more difficult to pick the best option. But that’s why we're here. Our curated list of the best streaming devices will get you online in no time.
Movies & TV

NBCUniversal will launch its own streaming service in 2020

NBCU is prepping a streaming service filled with its original content for a debut sometime next year, meaning that Michael, Dwight, and the rest of the Scranton crew might be moving to a new home.
Home Theater

Sling TV offers free shows, a la carte subscription channels to Roku users

If you are among a select group of Roku users, Sling TV has added free TV shows and the option to subscribe to individual channels without having to subscribe to the company's base level of channels.

Break out the Wii: ‘Just Dance’ movie reportedly in the works

Sony's Screen Gems has reportedly acquired the film rights to the Just Dance video game series from Ubisoft, which previously helped bring the game Assassin's Creed to the big screen in 2016.
Movies & TV

Apple’s first original feature film reunites Bill Murray, Sofia Coppola

The Lost in Translation team of Sofia Coppola and Bill Murray will reunite with their A Very Murray Christmas collaborator Rashida Jones for On the Rocks, Apple and studio A24's first official feature together.
Movies & TV

You can get a screaming deal on LG OLED TVs just in time for the Super Bowl

LG is offering steep discounts on its critically acclaimed OLED TV models just in time for the Super Bowl, letting discerning viewers watch their favorite athletes with higher picture quality than ever.
Movies & TV

Jason Reitman’s ‘Ghostbusters’ sequel already has a creepy teaser trailer

Jason Reitman, son of Ghostbusters director Ivan Reitman, has secretly co-written a follow-up to the 1984 supernatural comedy. Filming begins this summer, but it's not clear if the original Ghostbusters will reprise their roles.
Movies & TV

From premiere date to footage: Here's all we have on 'Game of Thrones' season 8

With the eighth and final season looming, Game of Thrones fever has officially become a pandemic. Our list of all the relevant news and rumors will help make the wait more bearable, if you don't mind spoilers.
Movies & TV

Netflix recruits Steve Carell for the Trump-inspired comedy ‘Space Force’

Steve Carell, Netflix, and The Office showrunner Greg Daniels are teaming up for Space Force, a workplace comedy poking fun at the Trump White House's plans to establish a sixth branch of the U.S. military.
Home Theater

Cutting the cord? Let us help you find the best service for live TV streaming

There's a long list of live TV streaming services available to help you cut the cord and replace your traditional TV subscription. Each is different in important ways, and this guide will help you find the best one for you.