A team of researchers at Carnegie Mellon University have developed an artificial intelligence system that automatically transfers content from one film into the style of another. In the vein of “deep fakes”, the A.I.-augmented videos infamous for superimposing one person’s face onto another’s body, the CMU system presents another case for how difficult it could be to distinguish fiction from reality in the future.
The CMU researches have named their system Recycle-GAN, after a class of algorithms that help make deep fakes possible by applying the style of one image or video to another.
In a video released early this month, the researchers demonstrated how a source video of Barack Obama speaking can be processed to make it seem as though Donald Trump is mouthing the word. Or a monologue from John Oliver can be transformed into one from Stephen Colbert. Recycle-GAN isn’t limited to human faces either. The researchers also show how the system can make a daffodil bloom with the same mechanics as a hibiscus.
The end result isn’t perfect — a slew of digital artifacts around the edges of the edited faces make it clear that things aren’t exactly as they seem. Still, it’s pretty impressive.
“Recycle-GAN encodes both spatial and temporal information,” Aayush Bansal, a CMU Ph.D. student who worked on the project, told Digital Trends. “Spatial constraints enable it to learn transformation from one domain to another, and the temporal information helps in better learning stylistic information and improve the spatial transformation.”
Bansal said he was motivated to develop Recycle-GAN by an urge to, in a sense, resurrect the dead. “One of my life goals is to bring back … Charlie Chaplin in our movies,” he said. As such, Bansal sees the system as a tool for artists, such as moviemakers, and data-hungry researchers.
Bansal acknowledged that bad actors could exploit a tool like Recycle-GAN to perform troubling manipulations similar to those we’ve seen with deep fakes, including fake news videos and fake porn. However, he hopes his team’s approach could provide a solution for identifying deep fakes rather than fuel to the fire.
“Our approach enables generation of data which could be used to train a simple machine learning model that can discriminate between real and fake,” he said. “Generating this fake data was hard earlier because most deep fakes out there require human intervention or manual supervision, and as such we could never get an automatic way to detect them. However, now we have an automatic way to generate such data that we can train models that could detect fake content with some reliable accuracy.”
Bansal presented his team’s work at the European Conference on Computer Vision in Germany earlier this month.
