Skip to main content

These amazing audio deepfakes showcase progress of A.I. speech synthesis

Visual deepfakes, in which one person’s face is spliced onto another person’s body, are so 2019. Here in 2020, deepfake technology trends have shifted a bit, and now the cool kids are using the technology is to create impressive “soundalike” audio tracks.

While these have plenty of scary potential when it comes to fake news and the like, for now, it seems that creators are perfectly happy to use them for more irreverent purposes, such as getting famous figures to perform songs they never had any real involvement with.

Here are five of the weirdest and best — including one made specifically for Digital Trends that you won’t find anywhere else.

Jay-Z raps ‘We Didn’t Start the Fire’

Jay-Z covers "We Didn't Start the Fire" by Billy Joel (Speech Synthesis)

No, this audio deepfake of Jay-Z rapping Billy Joel’s We Didn’t Start the Fire didn’t start any fires when it comes to showcasing this vocal synthesis tech. But, having triggered one of the first legal complaints about its usage (by Jay-Z’s record label), YouTube deepfake audio creator Vocal Synthesis helped raise awareness of these tools for a lot of people.

The vocal reproduction of Jay-Z’s voice isn’t perfect in his unofficial cover of Joel’s 1989 smash hit. But, in the breathy staccato style used by Jay, some of the more awkward vocal glitches are masked pretty well. This is a great showcase of deepfake audio in action: Its strengths, its weaknesses, and its eerie abilities to take a piece of text we immediately associate with one person and turn it into something that sounds convincingly like it came out of someone else’s mouth.

The queen recites The Sex Pistols

Queen Elizabeth II reads "God Save the Queen" by Sex Pistols (Speech Synthesis)

Another Vocal Synthesis creation, Queen Elizabeth II (that’s the current queen) reading the Sex Pistol’s 1977 single God Save the Queen is the kind of brilliant meta-parody the internet does so well. The song’s title is, of course, taken from the national anthem of the same name; repurposed to fit lyrics resentful of the English class system and the idea of a monarchy. The original song was famously banned from broadcast by both the BBC and United Kingdom’s Independent Broadcasting Authority.

The Queen Elizabeth voice synthesis on this particular creation wavers in and out, sounding more like a stitched-together tapestry of different samples than one cohesive reading. But is there anything more punk in its conception than a homemade DIY creation which turns, literally, the voice of authority against itself? Brilliant stuff.

Bill Clinton ponders if ‘Baby Got Back’

Bill Clinton reads "Baby Got Back" by Sir Mix-A-Lot (Speech Synthesis)

He likes big butts and he can’t deny. There’s something of a subgenre among deepfake audio makers of getting former U.S. presidents to lend their instantly recognizable voices to perform an array of musical numbers.

Bill Clinton playing Sir Mix-a-Lot doesn’t do it for you? How about George W. Bush performing 50 Cent’s In Da Club. Or maybe you’d just settle for a medley of former POTUS’s spitting NWA’s F*ck Tha Police? (At least the last two of these are NSFW, although in the age of working from home such things may no longer apply!)

Frank Sinatra and Ella Fitzgerald get their ‘La La Land’ on

Jukebox AI regenerates "city of stars" using Frank Sinatra's voices and music style.

So far, all of these have concentrated on synthesizing vocals only. That’s a good start, but an artist’s voice is just one part of their repertoire. What if you could use deepfake audio technology to not just reproduce a person’s voice, but also to learn their other musical stylings and use this to dream up a whole new piece of music?

This is the basis of Open AI’s Jukebox, a music-generating neural network that generates music — including, in its own words, “rudimentary singing … in a variety of genres and artist styles.” Unsurprisingly, this powerful tool is already being put to work, as evidenced by the above collaboration between Frank Sinatra and Ella Fitzgerald singing City of Stars from 2016’s Oscar-winning movie La La Land. The results aren’t perfect, but they definitely give a taste of where all of this is going.

Nirvana interprets ‘Clint Eastwood’

Top 4 Music Deep Fakes in the Style of Nirvana (sorta) sing Clint Eastwood by Gorillaz

In a piece created especially for Digital Trends, the folks at generative A.I. group Dadabots, CJ Carr and Zack Zukowski, whipped up a deepfake audio of legendary grunge band Nirvana riffing on Clint Eastwood, the 2001 single from the British virtual band Gorillaz.

“We used the pretrained, 5 billion-parameter Jukebox model,” Carr told Digital Trends. “It’s been trained on 7,000-plus bands, including Nirvana’s discography. We ran models on multiple Linux servers, set them to grunge and Nirvana, with the hook from Clint Eastwood as lyrics, then generated 27 different 90-second clips on our V100s, and picked our favorite top four.”

As Carr notes, there is still a degree of human creativity involved because they need to select the best pieces. A lot of the time, Carr said, the music clips sound less like one specific band and more like a generic group in that genre. Nonetheless, it’s pretty fascinating stuff.

“Sometimes it invents its own lyrics, [such as] ‘I got sunshine in my head,’ Carr said. “Sometimes the band goes into a breakdown. It kinda has a mind of its own. The realism and room for its own creativity is astonishing. I feel like we’re just scratching the surface on how to manipulate it.”

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
A.I. system seeks to turn thoughts of people unable to talk into speech
Cognitive Dissonance

For the first time, neuroengineers have developed a system capable of translating thoughts directly into recognizable speech, marking an important step toward more advanced brain-computer interfaces for people who lack the ability to speak.

The system, which was created by researchers at Columbia University, works by monitoring a person’s brain activity, identifying brain signals, and reconstructing the words the individual hears. Powered by speech synthesizers and artificial intelligence, the technology lays the groundwork for helping individuals who are unable to speak due to disability regain their capacity to communicate verbally.

Read more
IBM’s latest A.I. can help doctors detect how quickly a disease is progressing
ibm probabilistic ai huntingtons diseases doctor seeing patient

While there obviously shouldn't be an outcry for robots to take the jobs of human physicians, that doesn’t mean artificial intelligence can’t play an important role when it comes to helping doctors carry out certain aspects of their jobs -- such as recognizing early symptoms of serious diseases that can go unnoticed. A delay in diagnosis can make it more difficult to implement effective treatments or even find cures.

This is something that IBM wants to help with, courtesy of a new probabilistic A.I. model that can be used to better understand patients’ individual conditions and, crucially, how they are likely to develop. It can give information about a wide range of symptoms -- from shaking hands to mood swings -- and then use this data to help identify the biomarkers of diseases including diabetes, Huntington’s disease, Alzheimer’s, Parkinson’s, and more. The A.I. can help doctors understand more about the progression of these neurological disorders, as well as pinpoint how advanced they are.

Read more
Deepfake A.I. puts a young Harrison Ford into ‘Solo: A Star Wars Story’
solo a star wars story deepfake harrison ford

Solo | A Derpfakes Story

One of the recurring criticisms of Solo: A Star Wars Story often involves the difficulty of accepting anyone other than original Han Solo actor Harrison Ford in the film's title role. Imagining a young Ford replacing Solo star Alden Ehrenreich is a little easier now, though, thanks to some advanced image-editing and synthesis techniques.

Read more