Skip to main content

These amazing audio deepfakes showcase progress of A.I. speech synthesis

 

Visual deepfakes, in which one person’s face is spliced onto another person’s body, are so 2019. Here in 2020, deepfake technology trends have shifted a bit, and now the cool kids are using the technology is to create impressive “soundalike” audio tracks.

Recommended Videos

While these have plenty of scary potential when it comes to fake news and the like, for now, it seems that creators are perfectly happy to use them for more irreverent purposes, such as getting famous figures to perform songs they never had any real involvement with.

Here are five of the weirdest and best — including one made specifically for Digital Trends that you won’t find anywhere else.

Jay-Z raps ‘We Didn’t Start the Fire’

Jay-Z covers "We Didn't Start the Fire" by Billy Joel (Speech Synthesis)

No, this audio deepfake of Jay-Z rapping Billy Joel’s We Didn’t Start the Fire didn’t start any fires when it comes to showcasing this vocal synthesis tech. But, having triggered one of the first legal complaints about its usage (by Jay-Z’s record label), YouTube deepfake audio creator Vocal Synthesis helped raise awareness of these tools for a lot of people.

The vocal reproduction of Jay-Z’s voice isn’t perfect in his unofficial cover of Joel’s 1989 smash hit. But, in the breathy staccato style used by Jay, some of the more awkward vocal glitches are masked pretty well. This is a great showcase of deepfake audio in action: Its strengths, its weaknesses, and its eerie abilities to take a piece of text we immediately associate with one person and turn it into something that sounds convincingly like it came out of someone else’s mouth.

The queen recites The Sex Pistols

Queen Elizabeth II reads "God Save the Queen" by Sex Pistols (Speech Synthesis)

Another Vocal Synthesis creation, Queen Elizabeth II (that’s the current queen) reading the Sex Pistol’s 1977 single God Save the Queen is the kind of brilliant meta-parody the internet does so well. The song’s title is, of course, taken from the national anthem of the same name; repurposed to fit lyrics resentful of the English class system and the idea of a monarchy. The original song was famously banned from broadcast by both the BBC and United Kingdom’s Independent Broadcasting Authority.

The Queen Elizabeth voice synthesis on this particular creation wavers in and out, sounding more like a stitched-together tapestry of different samples than one cohesive reading. But is there anything more punk in its conception than a homemade DIY creation which turns, literally, the voice of authority against itself? Brilliant stuff.

Bill Clinton ponders if ‘Baby Got Back’

Bill Clinton reads "Baby Got Back" by Sir Mix-A-Lot (Speech Synthesis)

He likes big butts and he can’t deny. There’s something of a subgenre among deepfake audio makers of getting former U.S. presidents to lend their instantly recognizable voices to perform an array of musical numbers.

Bill Clinton playing Sir Mix-a-Lot doesn’t do it for you? How about George W. Bush performing 50 Cent’s In Da Club. Or maybe you’d just settle for a medley of former POTUS’s spitting NWA’s F*ck Tha Police? (At least the last two of these are NSFW, although in the age of working from home such things may no longer apply!)

Frank Sinatra and Ella Fitzgerald get their ‘La La Land’ on

Jukebox AI regenerates "city of stars" using Frank Sinatra's voices and music style.

So far, all of these have concentrated on synthesizing vocals only. That’s a good start, but an artist’s voice is just one part of their repertoire. What if you could use deepfake audio technology to not just reproduce a person’s voice, but also to learn their other musical stylings and use this to dream up a whole new piece of music?

This is the basis of Open AI’s Jukebox, a music-generating neural network that generates music — including, in its own words, “rudimentary singing … in a variety of genres and artist styles.” Unsurprisingly, this powerful tool is already being put to work, as evidenced by the above collaboration between Frank Sinatra and Ella Fitzgerald singing City of Stars from 2016’s Oscar-winning movie La La Land. The results aren’t perfect, but they definitely give a taste of where all of this is going.

Nirvana interprets ‘Clint Eastwood’

Top 4 Music Deep Fakes in the Style of Nirvana (sorta) sing Clint Eastwood by Gorillaz

In a piece created especially for Digital Trends, the folks at generative A.I. group Dadabots, CJ Carr and Zack Zukowski, whipped up a deepfake audio of legendary grunge band Nirvana riffing on Clint Eastwood, the 2001 single from the British virtual band Gorillaz.

“We used the pretrained, 5 billion-parameter Jukebox model,” Carr told Digital Trends. “It’s been trained on 7,000-plus bands, including Nirvana’s discography. We ran models on multiple Linux servers, set them to grunge and Nirvana, with the hook from Clint Eastwood as lyrics, then generated 27 different 90-second clips on our V100s, and picked our favorite top four.”

As Carr notes, there is still a degree of human creativity involved because they need to select the best pieces. A lot of the time, Carr said, the music clips sound less like one specific band and more like a generic group in that genre. Nonetheless, it’s pretty fascinating stuff.

“Sometimes it invents its own lyrics, [such as] ‘I got sunshine in my head,’ Carr said. “Sometimes the band goes into a breakdown. It kinda has a mind of its own. The realism and room for its own creativity is astonishing. I feel like we’re just scratching the surface on how to manipulate it.”

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Topics
Google just gave vision to AI, but it’s still not available for everyone
Gemini Live App on the Galaxy S25 Ultra broadcast to a TV showing the Gemini app with the camera feature open

Google has just officially announced the roll out of a powerful Gemini AI feature that means the intelligence can now see.

This started in March as Google began to show off Gemini Live, but it's now become more widely available.

Read more
This modular Pebble and Apple Watch underdog just smashed funding goals
UNA Watch

Both the Pebble Watch and Apple Watch are due some fierce competition as a new modular brand, UNA, is gaining some serous backing and excitement.

The UNA Watch is the creation of a Scottish company that wants to give everyone modular control of smartwatch upgrades and repairs.

Read more
Tesla, Warner Bros. dodge some claims in ‘Blade Runner 2049’ lawsuit, copyright battle continues
Tesla Cybercab at night

Tesla and Warner Bros. scored a partial legal victory as a federal judge dismissed several claims in a lawsuit filed by Alcon Entertainment, a production company behind the 2017 sci-fi movie Blade Runner 2049, Reuters reports.
The lawsuit accused the two companies of using imagery from the film to promote Tesla’s autonomous Cybercab vehicle at an event hosted by Tesla CEO Elon Musk at Warner Bros. Discovery (WBD) Studios in Hollywood in October of last year.
U.S. District Judge George Wu indicated he was inclined to dismiss Alcon’s allegations that Tesla and Warner Bros. violated trademark law, according to Reuters. Specifically, the judge said Musk only referenced the original Blade Runner movie at the event, and noted that Tesla and Alcon are not competitors.
"Tesla and Musk are looking to sell cars," Reuters quoted Wu as saying. "Plaintiff is plainly not in that line of business."
Wu also dismissed most of Alcon's claims against Warner Bros., the distributor of the Blade Runner franchise.
However, the judge allowed Alcon to continue its copyright infringement claims against Tesla for its alleged use of AI-generated images mimicking scenes from Blade Runner 2049 without permission.
Alcan says that just hours before the Cybercab event, it had turned down a request from Tesla and WBD to use “an icononic still image” from the movie.
In the lawsuit, Alcon explained its decision by saying that “any prudent brand considering any Tesla partnership has to take Musk’s massively amplified, highly politicized, capricious and arbitrary behavior, which sometimes veers into hate speech, into account.”
Alcon further said it did not want Blade Runner 2049 “to be affiliated with Musk, Tesla, or any Musk company, for all of these reasons.”
But according to Alcon, Tesla went ahead with feeding images from Blade Runner 2049 into an AI image generator to yield a still image that appeared on screen for 10 seconds during the Cybercab event. With the image featured in the background, Musk directly referenced Blade Runner.
Alcon also said that Musk’s reference to Blade Runner 2049 was not a coincidence as the movie features a “strikingly designed, artificially intelligent, fully autonomous car.”

Read more