Skip to main content

Amazing new headset translates thoughts into speech for vocally impaired wearers

Vicky Just/University of Bath

“In a nutshell,” said Scott Wellington, “we’re hoping to create a technology that can take your imagined speech — that is, you think of a word or a sentence, without moving or speaking at all — and translate your brain signals into synthesized speech of that same word or sentence.”

That’s quite a mission, but Wellington, a Ph.D. researcher at the University of Bath’s Center for Accountable, Transparent and Responsible Artificial Intelligence, may just be up to the job.

For the past several years, via his previous work at the University of Edinburgh and a startup called SpeakUnique, Wellington has been working on an ambitious, but potentially game-changing, project: Creating personalized synthetic voices for those who have impaired speech or entirely lost the ability to speak as a result of neurodegenerative conditions like Motor Neurone Disease (MND).

“The goal is to create a new technique that allows more fluent communication by either supporting or, even better, altogether replacing the need to type out what you want to communicate, by using the brain signal to do the ‘typing’ instead.”

Synthetic voices for people with potentially debilitating conditions like MND have been around for years. Famously, the late theoretical physicist Stephen Hawking communicated using a synthesized computer voice, created for him by a Massachusetts Institute of Technology engineer named Dennis Klatt, as far back as 1984. The voice, a default male named “Perfect Paul,” could be operated using a handheld clicker that would enable him to choose words from a computer. Later, when Hawking lost the use of his hands, he switched to a system that detected his facial movement.

Vicky Just/University of Bath

Wellington’s work would be a step forward from this. For one thing, where recordings exist or suitable sound parts could be made, he could piece together a synthetic personalized voice that sounds like the person it’s being used for. Furthermore, this voice could be controlled entirely through the user’s thoughts — all using a humble, commercially available gamer’s headset.

Promising developments

“There have already been some promising developments in the field from researchers around the world, but these have all used a process called electrocorticography, which requires a craniotomy,” Wellington said.

A craniotomy, as he points out, is invasive brain surgery. The goal of his work at the University of Bath is to achieve the effect of “imagined speech recognition,” but without the need for someone to cut open your head and plant sensors onto the surface of your brain.

“For people who have lost their natural speech, one of the biggest causes of frustration is the inability to communicate their thoughts to friends and family with the same speed and naturalness as they had previously,” he said. “For instance, for people in advanced stages of MND, eye-tracking technologies can allow people with severely impaired motor control to use text-to-speech systems to communicate at around 10 words a minute, and that’s if they’re fluent users of the technology. You and I can speak 10 words in a few seconds. You can see why this is one of the biggest causes of frustration for people with motor impairment who have lost their speech.”

In the University of Bath setup, the gaming headset employed is equipped with an EEG (electroencephalography) system to detect the wearers’ brain waves. These are then processed by a computer that uses neural networks and deep learning to identify the intended speech of the user.

“We’ve been able to translate these imagined sounds with a promising degree of accuracy.”

“The goal is to create a new technique that allows more fluent communication by either supporting or, even better, altogether replacing the need to type out what you want to communicate, by using the brain signal to do the ‘typing’ instead,” Wellington said. “With the latest developments in engineering, machine learning, and artificial intelligence, I believe we’re at the stage to begin to make this a reality.”

To train the system, volunteers wore the EEG device while a recording of their own speech was played for them. At the same time, they had to imagine saying the sound, as well as vocalize the sound. While it would be accurate to describe the system as reading thoughts, it would still require the user to silently verbalize the words they wanted to say. (The plus side of this is that there’s no risk of it accidentally reading a wearers’ most private thoughts.)

The future’s bright, but manage expectations

Wellington was clear that he wants to “manage expectations.” Taking the noisy signal of brain waves and trying to pick up the all-important signal contained in it is tough. He likened it to trying to have a phone conversation with a person who is outside in heavy wind — or even a hurricane. “If they’re shouting the same word over and over, yes, probably you’ll get it,” he said. “But a natural, full sentence? Probably not.”

Vicky Just/University of Bath

This will hopefully change as the project advances and they get better at extracting information from the brain signal. New machine learning techniques should push the capabilities of gaming headsets for better imagined natural speech reception. One challenge, which will prove worthwhile in the end, is that the researchers want to make sure that whatever hardware they use is affordable, practical, and mobile.

“[So far] we’ve managed to achieve some success in decoding imagined speech sounds from the brain signal,” Wellington said. “That is, imagine you were sounding out the English language phonically, as children do in school: ‘Aah,’ ‘buh,’ ‘kuh,’ ‘duh,’ ‘ehh,’ ‘guh,’ and so forth. We’ve been able to translate these imagined sounds with a promising degree of accuracy. Of course, this is far from natural speech, but does already allow for a brain-computer interface that can translate a small ‘closed’ vocabulary of distinct words quite reliably. For example, if you wanted the device to speak, from your thoughts, the words for ‘up,’ ‘down,’ ‘left,’ ‘right,’ ‘start,’ ‘stop,’ ‘back,’ ‘forwards,’ [that would be possible].”

Wellington noted that he is excited about developments like Elon Musk’s Neuralink hardware, a “brain chip” that could be implanted beneath the skull, which could prove extremely transformative for work such as this. “As you can imagine, I was left wanting to know what we could achieve if such a device were implanted over the speech- and language-processing regions of the brain,” he said. “There’s certainly an exciting future ahead for this research!”

The work was presented at the Interspeech virtual conference in late October 2020.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Spider-Man: Across the Spider-Verse features an incredible video game Easter egg
Miles Morales falls through a multiverse portal in Spider-Man: Across the Spider-Verse.

Spider-Man: Across the Spider-Verse is out now and it contains a wealth of Easter eggs for video fans to obsess over. It's full of references to Spidey's history, from his comic books to cartoons and beyond. The animated sequel even includes some nods to the franchise's video game history, leading to one of its best (and most obscure) references: Spider-Man for the Atari 2600.

(A quick heads up if you're afraid of spoilers: This article contains some very light story details for Spider-Man: Across the Spider-Verse.)

Read more
Best gaming laptop deals: Portable gaming rigs from $600
The Alienware x17 R2 Gaming Laptop viewed from the back.

If you're keen to game while on the move, you need a sweet gaming laptop. Regular laptop deals won't cut it as you need something with a dedicated graphics card and maybe even something with some cool gamer looks too. Gaming laptops aren't cheap but that doesn't mean there aren't some great deals around. If you're looking to invest in a cool new gaming laptop, we've picked out the current best deals around right now. We've considered different budgets as well as different requirements so there's something for everyone below. Take a look and see what appeals to you.
Lenovo IdeaPad Gaming 3 -- $600, was $900

Known as one of the best laptop brands for both gaming and business purposes, Lenovo knows how to make a robust system. The Lenovo IdeaPad Gaming 3 packs a lot into its great price tag. It has an AMD Ryzen 5 5600H processor, 8GB of memory and 256GB of SSD storage. More storage space would be nice but if you're fine just having a few games installed at once, this won't slow you down. It also has an Nvidia GeForce RTX 3050 Ti graphics card which is great for playing the latest games if you don't mind tweaking the detail level. As well as that, there's a 15.6-inch full HD screen, white backlit keyboard and a cool-looking style to the laptop.

Read more
Nintendo Switch’s weirdest launch game is getting a surprise sequel
Key art for Everybody 1-2-Switch!

Nintendo has quietly announced Everybody 1-2-Switch!, a sequel to one of the oddest launch titles for Nintendo Switch.
1-2-Switch! was a Nintendo Switch launch title that served as a tech demo of sorts for the Joy-Con controllers, with players completing minigames based on direction from actors in live-action clips. It's a very quirky and critically divisive Nintendo Switch game, so it's a bit surprising to see a follow-up. According to the eShop listing for Everybody 1-2-Switch!, this game lets players use their Joy-Cons or smartphones to complete a variety of team-based minigames. And that's about all we've seen or know about this $30 game ahead of its June 30 launch alongside some new pastel Joy-Cons.

Everybody 1-2-Switch!'s announcement was quite unorthodox for Nintendo. It simply tweeted about the game's existence and opened up preorders on the eShop. This all happened with no reveal trailer less than a month before the game's launch. It's unknown why Nintendo's taking this approach instead of including the game in a Nintendo Direct or other major showcase, but a 2022 report from Fanbyte may reveal the answer. According to that report, Everybody 1-2-Switch! tested horribly, and Nintendo worried that the game could "damage the company’s reputation as a great software developer." 
Almost a year after that report, it seems that the game has improved enough to release, or maybe Nintendo is just willing to take the hit after the impressive launch and critical reception of The Legend of Zelda: Tears of the Kingdom. Either way, this is a weird sequel to a weird Nintendo Switch launch game that's being mutedly marketed so it doesn't elicit much hype. We'll just have to wait and play it for ourselves if we want to know whether Everybody 1-2-Switch! is an enjoyable game or not.
Everybody 1-2-Switch! will be released physically and digitally for Nintendo Switch on June 30.

Read more