Skip to main content

The Irishman’s FX team invented bleeding-edge tech to subtly de-age De Niro

How The Irishman’s Groundbreaking VFX Took Anti-Aging To the Next Level | Netflix

Netflix and acclaimed director Martin Scorsese initially seemed like an odd pairing for The Irishman, the filmmaker’s adaptation of Charles Brandt’s 2004 novel I Heard You Paint Houses. Any uncertainties about that collaboration were swiftly dismissed by the film’s success both on the big screen and on the streaming service, however, and it’s now regarded as frontrunner in the upcoming awards season.

Chronicling the life of alleged mafia hitman Frank Sheeran and his possible connection to the disappearance of Teamsters union leader Jimmy Hoffa, The Irishman features impressive performances by leading men Robert De Niro, Al Pacino, and Joe Pesci as they portray a trio of real-world characters whose lives intertwined over the course of several generations. The film follows the characters across a wide range of ages using visual effects, but it doesn’t rely on the usual high-tech helmets, motion-capture suits, or facial markers, opting instead for an innovative new technology.

Leading the visual effects team was Industrial Light & Magic’s Pablo Helman, who previously worked with Scorsese on 2016’s Silence and has twice been nominated for an Academy Award for his work in visual effects. Digital Trends spoke with Helman about the challenges of developing and testing the visual effects technology used in The Irishman, and maybe most importantly, convincing one of the most celebrated directors of all time to trust digital de-aging techniques on some of Hollywood’s most famous faces.

The Irishman Netflix Production

Digital Trends: Going back to the early days of this project, how did you convince Martin Scorsese that this kind of digital de-aging was not only possible, but would work with his style of filmmaking?

Pablo Helman: It all comes with trust. Working on films, you rely on each other for all kinds of things. That’s one part of it. We also talked about technology, and Marty is open to trying new things when they make sense. So I made a good argument as to why it would be a good idea to to try to make the movie that way.

What was the gist of that discussion about the technology?

Well, we talked about different methodologies. There was no way to make this movie by bringing in another actor and replacing his face because of the way the movie was going to be edited, back and forth and back and forth over time. It would be impossible to replace the bodies over and over again. So that option went away.

The Irishman Netflix Robert De Niro

The second option was to cast a younger actor. Marty wanted to keep the connection between the characters (at) different ages, and you want the audience to also keep that connection, too — so it was going to be very difficult having the three main actors, plus three younger actors, plus all the makeup they each had to wear to keep that connection, and so on. So, there was really no other way to do it. So, once we were thinking we’d need to go with the digital de-aging, we had to show him what that would look like.

How did you settle on a particular type of digital de-aging? There are so many ways studios are handling this type of effect lately.

Marty said, “If we’re going to go with de-aging, Robert De Niro is not going to be somebody wearing all of that technology on his face. He’s not going to be wearing [facial] markers. He’s not going to be wearing a helmet with little cameras. He’s not going to be wearing a gray pajama suit, [and] he’s going to want to be on set. So if you can figure it out, then we can do it.”

Robert De Niro in the Irishman
Image used with permission by copyright holder

I got really excited about that because it was a great opportunity for us to push the technology toward the natural place where we want it to go. If you think about the history of performance capture, it started with tracking the bodies manually. Then we put markers on actors’ bodies and tracked them that way, because you could see so much more, and we eventually trained the computer to track those markers. And then markers migrated to the face, and we got computers to track those markers, too. So the natural progression is for us to push for no markers whatsoever. I jumped at trying to push filmmaking in that direction, but first I had to show it to Marty.

What were the early tests like?

Young Robert De Niro
Image used with permission by copyright holder

I proposed bringing De Niro in and having him reenact a scene from Goodfellas, because that was a way to lay the groundwork, and it was something Marty knew really well. So [De Niro] comes in and does the scene as a 74-year-old actor, and we were able to change him to look the way he did 30 to 40 years ago. At that point, they were going to trust me.

Which scene from Goodfellas did you have De Niro reenact?

It was the Pink Cadillac scene, after they steal millions of dollars and everybody starts buying stuff. De Niro’s character told everybody not to buy anything with the money, but then one of the characters shows up with a new pink Cadillac, and De Niro says, “What the fuck is wrong with you?”

We chose this scene because he was so over the top in it. Those extreme performances are so difficult to work on, and we wanted to test the behavioral likeness of the de-aging effect — because it’s not just about looking like a 40-year-old man, it’s also the way he behaves. That makes him who he is. That was going to be very difficult to catch without markers, so that test proved we could do this without markers and without interfering with the actor.

Image used with permission by copyright holder

So what did you come up with to capture all of those details without markers?

Well, if we don’t have markers, then the only things we do have are the actor, who is a 3-D object in front of the camera, and the lighting that hits the actor. So we set out to capture the lighting and the textures created by that lighting, and make some 3-D geometry out of it. To do that, we had to acquire the most information possible. We had one camera, the director’s camera, to catch the action. But if we had a way to capture the action from different points of view, with different cameras, we could triangulate from each of the cameras to create that 3-D geometry.

Image used with permission by copyright holder

So we came up with a three-camera rig. The center camera is still the director camera, and then to the left and right of the center camera were what we call “witness cameras,” which are infrared cameras. The software we created, Flux, takes a look at the information that comes through the three cameras and then triangulates whatever is in front of the camera to create 3-D geometry from it.

So you got three cameras’ worth of shading and depth and information from different perspectives?

Exactly. At the end of the day, the more information you have, the better the chances of creating something out of nothing.


You’ve worked on some films with massive, spectacle-driven visual effects, as well as films with a subtle approach to visual effects. The Irishman falls into the latter category. How does your approach to visual effects change when you take on a project like this, where subtlety is so important?

You have to take a look at things you overlook when you’re dealing with bigger atmospheric effects or things (that) are action-driven. With this, the camera is not moving, the framing is forehead to chin, and there is nowhere to hide. Everything in front of you is about performance, and you have to look at the performances of the actors and deconstruct that performance to do what you need to do.

So what ends up happening is, you start to understand what makes a concerned look, or a happy look, and what makes all of these different emotions come through. You start to understand that the chin moves a specific way and the nose or eyebrows all move in specific ways based on the emotion coming through, too. You really need to understand what makes that character look like the iconic actor that you’re going by.

The Irishman Production Still
Image used with permission by copyright holder

How does the technology catch and interpret all of that information?

So, this piece of software we created and wrote did away with markers, but it gave us all the pixels that the camera sees in somebody’s face. Now, instead of 200 markers, you have thousands of markers, constantly moving. So the software takes a look at that and captures all kinds of things like, for instance, the way the face moves during dialogue.

Image used with permission by copyright holder

A lot of the time [with visual effects used on a person’s face], you lose the weight of the dialogue, because when we put a consonant together with a phoneme, our face vibrates in a specific way. It’s like a rhythm. With markers, you don’t have access to that rhythm, because the software is not sensitive enough. Without markers, though, the computer is more sensitive to those kinds of things, and it captures them for you. Now, the dialogue and the whole performance starts to come together, because you see everything moving together at the same time.

And when two actors are together and their lines are hitting on the other person the right way, and the other person is understanding what they’re saying and reacting to it, and their eyes and faces and bodies are registering all of it, that also affects the rhythm of the performance. If you can capture all of that, then you end up with a fabulous shot.

Image used with permission by copyright holder

You’ve mentioned in a few interviews that you want The Irishman to be a referendum on visual effects technology in films. What do you mean by that?

So, what I meant was that technology is there to allow for the performance to come through. The director wants to have control, and the actors also want to have control. So it would be great for the technology we use to be away from where the performance originates. Technology needs to service the story, not the other way around.

the irishman vfx pablo helman
Image used with permission by copyright holder

No director likes to be told, “You can’t move the camera that way,” or “You can’t move the actors that way.” You shouldn’t have to tell a director not to light the actors in a specific way, or that they have to wear a helmet or a suit for technology reasons. So the idea is that, little by little, as technology gets better and better, we stay away from things that could affect the performance.

At the end of the day, the performances and the stories that are told by the performances are the things that the audience needs to get. That’s when the audience gets connected to characters: When the performances are as true as possible.

Editors' Recommendations

Rick Marshall
A veteran journalist with more than two decades of experience covering local and national news, arts and entertainment, and…
Everything leaving Max (formerly HBO Max) in October 2023
Eve and Villanelle from Killing Eve both looking shocked.

All good things must come to an end: romances, seasons, even free Twitter (sorry, X), etc. That's never been more true for streaming content libraries. Every month, as more movies and shows are added to each of the many streamers out there, it seems just as much content leaves as well.

Max isn't immune to this vicious cycle, and in October 2023, a fair amount of films and shows are leaving the service for an undetermined amount of time. From a couple of Exorcist movies to the Dolly Parton classic 9 to 5 to the so bad, it's good teen soap Dawson's Creek, there's a lot of good stuff leaving, so check out the list below and try to watch some of these gems before they leave Max for good (or at least a long while).

Read more
The best new shows to stream on Netflix, Hulu, Max (HBO), and more
A man in a suit with his hands up, others pointing guns at him in a scene from The Continental: From the World of John Wick.

This week offers a great example of why having multiple streaming subscriptions is a good idea. Because some of the best new shows to stream are all over the place. If you love action, Peacock's The Continental: From the World of John Wick brings some of that John Wick flavor to TV. Horror fans will naturally gravitate towards the new season of American Horror Story on Hulu. As for comedy fans, Netflix is bringing the laughs with the fourth and final season of Sex Education.

Netflix, Amazon Prime Video, Hulu, Max, Apple TV+, Disney+, Paramount+, and Peacock will keep you swimming in new shows, especially if you've taken the drastic step of subscribing to all of them. If you want to save some money, go for the cheaper ad-supported tiers. But whichever way you prefer to go, we've got you covered with our weekly roundup of the best new shows to stream.
When you're done here, check out the best new movies to stream this week, as well as the best shows on Netflix, best shows on Hulu, best shows on Amazon Prime Video, and best shows on Disney+.

Read more
The 50 best shows on Netflix in September 2023
The cast of Sex Education.

Not every Netflix original series gets to go out on its own terms, but the hit comedy Sex Education is coming to a proper end this week with the release of the fourth and final season. That's why we're welcoming it back to our roundup of the best shows on Netflix. And it shouldn't be too long before Sex Education is once again one of the 10 most popular shows on Netflix.

This week's other new addition is the fifth season of New Amsterdam, a medical drama that has now come to its conclusion. That makes it the perfect binge-watch after the summer of Suits, as new fans will have a lot of episodes to enjoy.

Read more