Skip to main content

New A.I. can identify the song you’re listening to by reading your brain waves

song identifying ai
Krishna P. Miyapuram

From Journey’s “Don’t Stop Believin’” to Queen’s “Bohemian Rhapsody” to Kylie Minogue’s “Can’t Get You Out Of My Head,” there are some songs that manage to successfully worm their way down our ear canals and take up residence in our brains. What if it was possible to read the brain’s signals, and to use these to accurately guess which song a person is listening to at any given moment?

That’s what researchers from the Human-Centered Design department at Delft University of Technology in the Netherlands and the Cognitive Science department at the Indian Institute of Technology Gandhinagar have been working on. In a recent experiment, they demonstrated that it is eminently possible — and the implications could be more significant than you might think.

For the study, the researchers recruited a group of 20 people. and asked them to listen to 12 songs using headphones. To aid with their focus, the room was darkened and the volunteers blindfolded. Each was fitted with an electroencephalography (EEG) cap that’s able to noninvasively pick up the electrical activity on their scalp as they listen to the songs.

This brain data, alongside the corresponding music, was then used to train an artificial neural network to be able to identify links between the two. When the resulting algorithm was tested on data it hadn’t seen before, it was able to correctly identify the song with 85% accuracy — based entirely on the brain waves.

“The songs were a mix of Western and Indian songs, and included a number of genres,” Krishna Miyapuram, assistant professor of cognitive science and computer science at the Indian Institute of Technology Gandhinagar, told Digital Trends. “This way, we constructed a larger representative sample for training and testing. The approach was confirmed when obtaining impressive classification accuracies, even when we limited the training data to a smaller percentage of the dataset.”

Reading minds, training machines

This isn’t the first time that researchers have shown that it’s possible to carry out “mind-reading” demonstrations that would make David Blaine jealous, all using EEG data. For instance, neuroscientists at Canada’s University of Toronto Scarborough have previously reconstructed images based on EEG data to digitally re-create face images stored in a person’s mind. Miyapuram’s own previous research includes a project in which EEG data was used to identify movie clips viewed by participants, with each one intended to provoke a different emotional response.

song identifying ai
Krishna P. Miyapuram

Interestingly, this latest work showed that algorithms that proved very effective at guessing the songs being listened to by one participant, after being trained on their specific brain, wouldn’t work so well when applied to another person. In fact, “not so well” is a gross understatement: The accuracy in these tests plummeted from 85% to less than 10%.

“Our research shows that individuals have personalized experiences of music,” Miyapuram said. “One would expect that the brains respond in a similar way processing information from different stimuli. This is true for what we understand as low-level features or stimulus-level features. [But] when it comes to music, it is perhaps the higher-level features, such as enjoyment, that distinguish between individual experiences.”

Derek Lomas, assistant professor of positive A.I. at Delft University of Technology, said that a future goal of the project is to map the relationship between EEG frequencies and musical frequencies. This could help answer questions such as whether greater aesthetic resonance is accompanied by greater neural resonance.

To put it another way, is a person who is “moved” by a piece of music going to show greater correlations between the music itself and the brain response, making it possible to accurately predict how much a person enjoys a piece of music simply by looking at their brain waves? While everyone’s response to music may be subtly different, this could help shed light on why humans seek out music to begin with.

The road to brain-computer interfaces

“For near-term applications [in the next two years], we imagine a music recommendation engine that could be based on a person’s brain response,” Lomas told Digital Trends. “I currently have a student working on algorithmically generated music that maximizes neural resonance. It’s quite eerie: The maximum neural resonance is not the same as maximum aesthetic resonance.”

song identifying ai
Krishna P. Miyapuram

In the medium term, Lomas suggested it could lead to powerful applications for gaining information on the “depth of experience” enjoyed by a person engaging with media. Using brain-analysis tools, it may (and, indeed, should) be possible to accurately predict how deeply engaged a person is while, say, watching a movie or listening to an album. A brain-based measure of engagement could then be used to hone specific experiences. Want to make your movie more engaging for 90% of viewers? Tweak this scene, change that one.

“In the long term — 20 years — this field of work may enable methods for transcribing the contents of the imagination,” Lomas continued. “For example, transcribing thoughts to text. That’s the big future of [brain-computer interfaces.]”

As Lomas noted, we are still a distance away from that eventual goal of a brain-computer interface. Nonetheless, work such as this suggests that there’s plenty of tasty lower-hanging fruit on that tree before we finally fell it.

A paper describing this research, titled GuessTheMusic: Song Identification from Electroencephalography, was recently presented at CODS-COMAD 2021.

Editors' Recommendations

Lossless Bluetooth audio? Qualcomm says it’s coming in 2022
A man listens to his cell phone audio via bluetooth headphones.

Qualcomm has revealed its next-generation Bluetooth chips for makers of wireless earbuds and headphones, and the company predicts that 2022 will be the year that we see a whole slew of new innovations like lossless hi-res audio, audio sharing, lower latency, and Bluetooth broadcasting come to our phones and personal audio gear.

The giant mobile chipmaker says that its two new sound platforms, S3 and S5, will be provided to customers like Yamaha, Cambridge Audio, Master & Dynamic, and Audio-Technica -- just to name a few -- and that we can expect to see the first S3- and S5-based products by the second half of 2022.

Read more
Amazon Music is now available on Vizio TVs
Amazon Music on a Vizio TV.

Amazon Music is now available as a native app on Vizio televisions. The inclusion of the app for SmartCast -- Vizio's built-in operating system -- means that you'll be able to stream more than 2 million songs for free on the third-largest music streaming service in the world without having to resort to Bluetooth or AirPlay or some other workaround. You'll also be able to stream Amazon Music Unlimited, which ramps things up to more than 75 million high-definition songs.

“With millions of users across the country, Vizio is an obvious choice for Amazon Music to expand our service into the homes of music fans,” Karolina Joynathsing, director of business development for Amazon Music, said in a press release. “We look forward to our customers being able to easily listen to their favorite music on Vizio smart TVs starting today.”

Read more
How to master your equalizer settings for the perfect sound
An equalizer on Apple Music's desktop app.

Contrary to what those bottle DJs on YouTube would have you believe, mixing sound isn't just twiddling fake nobs and sliders to make you look cool on the internet. Coaxing the best possible balance of tones out of a piece of music is an art form in and of itself, whether you're a producer, engineer, DJ, or just a music lover with an iPhone and a Spotify account.

The equalizer, or EQ, has come a long way since your dad's graphic EQ with the tiny little sliders that you never quite understood -- but somehow messing with them made his Zeppelin records sound "rad." But for most devices you'll encounter these days, it's all done digitally.

Read more