In some parallel universe, there’s a version of Casino Royale with Hugh Jackman playing everyone’s favorite suave British agent, James Bond. And one in which Matthew McConaughey took the Leo role in Titanic. And DiCaprio and Brad Pitt co-starred in Brokeback Mountain. And Saved by the Bell’s Tiffani Thiessen played Rachel in Friends.
The entertainment industry isn’t exactly short on “what if?” scenarios in which actors came close to, but were ultimately passed over, playing iconic roles. For more than 99% of movie history, fans have been able to do little more than squirrel away this trivia for use in pop quizzes. That is until the arrival of deepfakes.
Springing to life in the past couple of years, deepfakes use artificial intelligence technology to combine and superimpose new images and videos onto existing source footage using machine learning. That could mean anything from face swaps to mapping one person’s body onto someone else’s movements. The results can be jaw-droppingly realistic, which is why many people rightfully worry about its potential to be used for malicious hoaxes.
One tech enthusiast and movie buff thinks different, though. Operating under the YouTube username “Ctrl Shift Face,” this high-tech Hollywood fan has used deepfake technology to create some astonishing remixes of iconic movie scenes — complete with all new actors. Ever wanted to see The Shining starring Jim Carrey instead of Jack Nicholson? Sly Stallone in Terminator 2: Judgement Day? Heck, he’s even broken with the movie theme by dropping David Bowie into Rick Astley’s infamous song-turned-meme Never Gonna Give You Up.
“The Bowie one is my favorite,” its creator told Digital Trends. “I wanted to Rickroll people and blow them away at the same time. Bowie fitted the role of Rick Astley, and had interesting facial features for a deepfake.”
“The more the network learns, the more detailed the result will be.”
So who exactly is this prankster mogul who can put some of the world’s biggest actors into movies they never actually appeared in with a few clicks of the mouse? He’s not telling. “I don’t want to share my real name really,” Ctrl Shift Face’s creator explained. “It’s not important, I think. I let my work speak for itself.” He did mention a few personal details, such as his age (31), his place of birth (Slovakia, although he no longer lives there), and the fact that his job involves computer graphics on some level.
Ultimately, however, the fact that we don’t know who is behind Ctrl Shift Face raises an important point about the world of deepfakes. No longer confined to a few top Hollywood special effects houses, this kind of smart CGI wizardry is now available to all comers. It’s code that’s freely available online, capable of placing any actor alive or dead into any movie you can think of. No multi-million dollar salary negotiation or air-conditioned trailer conversation necessary!
“I’m not a coder, just a user,” said Ctrl Shift Face’s creator. “I don’t know the details about exactly how the software works. The workflow works like this: You add source and destination videos, then one neural network will detect and extract faces. Some data cleanup and manual extraction is needed. Next, the software analyzes and learns these faces. This step can sometimes take a few days. The more the network learns, the more detailed the result will be. In the final step, you combine these two and the result is your deepfake. There’s sometimes a bit of post-process needed as well.”
The future of fakes
While his favorite video is, as noted, the Bowie one, the one he feels is objectively the best is the Jim Carrey version of The Shining. Placing the face of the actor behind Ace Ventura: Pet Detective on the body of Jack Nicholson shouldn’t work, but it really, really does. “I had the idea to put a comedic actor into a serious dark scene,” he said. “Jim on Jack seemed like a good match.”
In the near future, it’ll be feasible to realistically recreate an actor’s speech using similar training data.
As for his least favorite? Terminator 2’s face swap with Stallone taking the role made famous by Arnold Schwarzenegger. “I guess my videos mirror my sense of humor,” he said. “For T2, there’s this joke from Last Action Hero, where it’s a sort of alternate universe where Sly played the Terminator.” But he doesn’t feel the results entirely worked, due to his choice to edit the audio to incorporate an impressionist doing a Stallone impersonation.
“I wasn’t able to edit it in seamlessly so the background music and ambient sounds have this noticeable jump,” he noted. “My version of the movie had had only 2.1 audio. Later, I realized if I had used 5.1 audio I could separate the audio channels and easily edit the new voice in. Another big regret was that the Bad to the Bone closing song really should have been Eye of the Tiger. It would have been a far better video than it is now.”
He said that he has heard from studios but “I can’t talk details.” That means we don’t know whether the remixes have been held in high regard or the total opposite. Given Hollywood’s enthusiasm about digitally de-aging actors, or even reincarnating deceased ones by putting their heads onto other actors’ bodies, it seems this technology will increasingly find its way into big-budget blockbusters. But it’s less clear if Tinseltown is happy that such tools are available for everyone.
Things could be about to get even more convincing, too. While the attempt to replace Arnie’s audio on Terminator 2 didn’t work as planned, there are efforts afoot in research labs to pioneer the audio equivalent of deepfakes. That means that, in the near future, it’ll be feasible to realistically recreate an actor’s speech using similar training data.
“That’s out of my expertise,” Ctrl Shift Face’s creator said. “But I may collaborate with people that can do these types of AI voices in the future.”
