Skip to main content

Groundbreaking A.I. can synthesize speech based on a person’s brain activity

Speech synthesis from neural decoding of spoken sentences

Scientists from the University of California, San Francisco have demonstrated a way to use artificial intelligence to turn brain signals into spoken words. It could one day pave the way for people who cannot speak or otherwise communicate to be able to talk with those around them.

The work began with researchers studying five volunteers with severe epilepsy. These volunteers had electrodes temporarily placed on the surface of their brains in order to locate the part of the brain responsible for triggering seizures. As part of this work, the team was also able to study the way that the brain responds when a person is speaking. This included analyzing the brain signals that translate into movements of the vocal tract, which includes the jaw, larynx, lips, and tongue. An artificial neural network was then used to decode this intentionality, which was in turn used to generate understandable synthesized speech.

While still at a relatively early stage, the hope is that this work will open up some exciting possibilities. A future step will involve carrying out clinical trials to test the technology on patients who are physically unable to speak (which was not the case with this demonstration). It will also be necessary to develop an Food and Drug Administration-approved electrode device with the kind of high channel capacity (256 channels in this latest study) required to capture the necessary level of brain activity.

This isn’t the first time we’ve covered impressive brain-computer interfaces at Digital Trends. In 2017, researchers from Carnegie Mellon University developed technology that used A.I. machine learning algorithms to read complex thoughts based on brain scans, including interpreting complete sentences in some cases.

A similar project, carried out by researchers in Japan, was able to analyze fMRI brain scans and generate a written description of what that person was viewing — such as “a dog is sitting on the floor in front of an open door” or “a group of people standing on the beach.” As this technology matures, more and more examples of similarly groundbreaking work will no doubt emerge.

A paper describing UC San Francisco’s recent work, titled Speech Synthesis From Neural Decoding of Spoken Sentences, was recently published in the journal Nature.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
A.I. translation tool sheds light on the secret language of mice
ai sheds light on mouse communication

Breaking the communication code

Ever wanted to know what animals are saying? Neuroscientists at the University of Delaware have taken a big leap forward in decoding the sounds made by one particular animal in a way that takes us a whole lot closer than anyone has gotten so far. The animal in question? The humble mouse.

Read more
Deep learning A.I. can imitate the distortion effects of iconic guitar gods
guitar_amp_in_anechoic_chamber_26-1-2020_photo_mikko_raskinen_006 1

Music making is increasingly digitized here in 2020, but some analog audio effects are still very difficult to reproduce in this way. One of those effects is the kind of screeching guitar distortion favored by rock gods everywhere. Up to now, these effects, which involve guitar amplifiers, have been next to impossible to re-create digitally.

That’s now changed thanks to the work of researchers in the department of signal processing and acoustics at Finland’s Aalto University. Using deep learning artificial intelligence (A.I.), they have created a neural network for guitar distortion modeling that, for the first time, can fool blind-test listeners into thinking it’s the genuine article. Think of it like a Turing Test, cranked all the way up to a Spınal Tap-style 11.

Read more
Mind-reading A.I. analyzes your brain waves to guess what video you’re watching
brain control the user interface of future eeg headset

Neural networks taught to "read minds" in real time

When it comes to things like showing us the right search results at the right time, A.I. can often seem like it’s darn close to being able to read people’s minds. But engineers at Russian robotics research company Neurobotics Lab have shown that artificial intelligence really can be trained to read minds -- and guess what videos users are watching based entirely on their brain waves alone.

Read more