Skip to main content

These amazing audio deepfakes showcase progress of A.I. speech synthesis

Visual deepfakes, in which one person’s face is spliced onto another person’s body, are so 2019. Here in 2020, deepfake technology trends have shifted a bit, and now the cool kids are using the technology is to create impressive “soundalike” audio tracks.

While these have plenty of scary potential when it comes to fake news and the like, for now, it seems that creators are perfectly happy to use them for more irreverent purposes, such as getting famous figures to perform songs they never had any real involvement with.

Here are five of the weirdest and best — including one made specifically for Digital Trends that you won’t find anywhere else.

Jay-Z raps ‘We Didn’t Start the Fire’

Jay-Z covers "We Didn't Start the Fire" by Billy Joel (Speech Synthesis)

No, this audio deepfake of Jay-Z rapping Billy Joel’s We Didn’t Start the Fire didn’t start any fires when it comes to showcasing this vocal synthesis tech. But, having triggered one of the first legal complaints about its usage (by Jay-Z’s record label), YouTube deepfake audio creator Vocal Synthesis helped raise awareness of these tools for a lot of people.

The vocal reproduction of Jay-Z’s voice isn’t perfect in his unofficial cover of Joel’s 1989 smash hit. But, in the breathy staccato style used by Jay, some of the more awkward vocal glitches are masked pretty well. This is a great showcase of deepfake audio in action: Its strengths, its weaknesses, and its eerie abilities to take a piece of text we immediately associate with one person and turn it into something that sounds convincingly like it came out of someone else’s mouth.

The queen recites The Sex Pistols

Queen Elizabeth II reads "God Save the Queen" by Sex Pistols (Speech Synthesis)

Another Vocal Synthesis creation, Queen Elizabeth II (that’s the current queen) reading the Sex Pistol’s 1977 single God Save the Queen is the kind of brilliant meta-parody the internet does so well. The song’s title is, of course, taken from the national anthem of the same name; repurposed to fit lyrics resentful of the English class system and the idea of a monarchy. The original song was famously banned from broadcast by both the BBC and United Kingdom’s Independent Broadcasting Authority.

The Queen Elizabeth voice synthesis on this particular creation wavers in and out, sounding more like a stitched-together tapestry of different samples than one cohesive reading. But is there anything more punk in its conception than a homemade DIY creation which turns, literally, the voice of authority against itself? Brilliant stuff.

Bill Clinton ponders if ‘Baby Got Back’

Bill Clinton reads "Baby Got Back" by Sir Mix-A-Lot (Speech Synthesis)

He likes big butts and he can’t deny. There’s something of a subgenre among deepfake audio makers of getting former U.S. presidents to lend their instantly recognizable voices to perform an array of musical numbers.

Bill Clinton playing Sir Mix-a-Lot doesn’t do it for you? How about George W. Bush performing 50 Cent’s In Da Club. Or maybe you’d just settle for a medley of former POTUS’s spitting NWA’s F*ck Tha Police? (At least the last two of these are NSFW, although in the age of working from home such things may no longer apply!)

Frank Sinatra and Ella Fitzgerald get their ‘La La Land’ on

Jukebox AI regenerates "city of stars" using Frank Sinatra's voices and music style.

So far, all of these have concentrated on synthesizing vocals only. That’s a good start, but an artist’s voice is just one part of their repertoire. What if you could use deepfake audio technology to not just reproduce a person’s voice, but also to learn their other musical stylings and use this to dream up a whole new piece of music?

This is the basis of Open AI’s Jukebox, a music-generating neural network that generates music — including, in its own words, “rudimentary singing … in a variety of genres and artist styles.” Unsurprisingly, this powerful tool is already being put to work, as evidenced by the above collaboration between Frank Sinatra and Ella Fitzgerald singing City of Stars from 2016’s Oscar-winning movie La La Land. The results aren’t perfect, but they definitely give a taste of where all of this is going.

Nirvana interprets ‘Clint Eastwood’

Top 4 Music Deep Fakes in the Style of Nirvana (sorta) sing Clint Eastwood by Gorillaz

In a piece created especially for Digital Trends, the folks at generative A.I. group Dadabots, CJ Carr and Zack Zukowski, whipped up a deepfake audio of legendary grunge band Nirvana riffing on Clint Eastwood, the 2001 single from the British virtual band Gorillaz.

“We used the pretrained, 5 billion-parameter Jukebox model,” Carr told Digital Trends. “It’s been trained on 7,000-plus bands, including Nirvana’s discography. We ran models on multiple Linux servers, set them to grunge and Nirvana, with the hook from Clint Eastwood as lyrics, then generated 27 different 90-second clips on our V100s, and picked our favorite top four.”

As Carr notes, there is still a degree of human creativity involved because they need to select the best pieces. A lot of the time, Carr said, the music clips sound less like one specific band and more like a generic group in that genre. Nonetheless, it’s pretty fascinating stuff.

“Sometimes it invents its own lyrics, [such as] ‘I got sunshine in my head,’ Carr said. “Sometimes the band goes into a breakdown. It kinda has a mind of its own. The realism and room for its own creativity is astonishing. I feel like we’re just scratching the surface on how to manipulate it.”

Editors' Recommendations

Topics
Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
If you use a VPN, don’t skip this important Windows 11 update
Microsoft Surface Laptop Go 3 rear view showing lid and logo.

It's not you; Windows is causing the issues this time. If the VPN on your Windows 11 or Windows 10 computer is having a hard time connecting, it is likely because of Microsoft's April security updates for Windows 11 (KB5036893 for) and Windows 10 (KB5036892), which have been reported to be the cause of the problems.

But there's good news. According to Microsoft, a patch is now available to fix the VPN problems users are experiencing.

Read more
This new Google Sheets feature is going to save so much time
Google Sheets is open in the Safari browser on a MacBook Air.

After Google I/O 2024, Google continues to roll out features that bolster its productivity apps -- this time, specifically with Google Sheets. As picked up by The Verge, Google has announced a much simpler way to generate easily formatted tables in the Sheets app. This new Sheets feature has been around for many years in Excel and has recently reached Google. Better late than never.

The option is called Convert to table, and you can use it by opening a Sheets document and clicking Format > Convert to table when the option reaches you later this month or early next. With this new option, Google aims for a more Excel-type experience by adding filters for each column. The rows also get visual separators, saving you time by not having to select the rows manually to turn them gray. The Convert to table feature also brings filters and column types and makes the drop-down menu creation easier.

Read more
The iPhone 16 Pro Max may get a very important battery upgrade
An iPhone 15 Pro Max laying face-down outside, showing the Natural Titanium color.

iPhone 16 dummy models Sonny Dickson / X

The iPhone 16 still has many months to go before its anticipated announcement in the fall, but we’ve already gotten a slew of rumors, leaks, and speculation about its specs and capabilities. One of the latest rumors about the iPhone 16 Pro Max comes from analyst Ming-Chi Kuo and it regards a new battery Apple is putting into the device.

Read more