Skip to main content

Facebook is developing AR glasses to help deaf people hear better

Usually, the news that a research lab is being turned into a bar would suggest that it has failed and its researchers are throwing in the towel. But in the case of Facebook Reality Labs Research, a lab dedicated (in part) to coming up with creative new ways to improve hearing for members of the deaf or hard of hearing community, it showed that its work was paying off.

Facebook’s stated mission is to help people connect and communicate with one another. This is why, while it might seem off-mission to tackle this project, it actually fits perfectly into the company’s core competencies.

Image used with permission by copyright holder

The problem Facebook has been trying to tackle is how to improve audio pickup for individuals in noisy environments. Although many people with hearing loss, depending on its severity, find it possible to hear and interact in one-on-one situations in quiet locations, noisy environments like bars or busy social settings can be a whole lot tougher. With that in mind, Facebook has been exploring how it can combine beamforming technology, deep learning, noise cancellation, and — perhaps most surprisingly — augmented reality (AR) to improve this situation.

In a recent project, Facebook scientists have been exploring how augmented reality glasses could help determine which sounds people want to amplify by looking at, well, where they’re looking. By matching a person’s focus, in terms of what they’re looking at, to audio inputs, AR glasses can figure out which sounds a person wants to hear and which they don’t — and then dial up or down the sound channels accordingly. As part of its experiments in this area, Facebook turned the workspace where much of this investigation is carried out into a fake bar to explore how it would work in a place with multiple noisy sound inputs.

Image used with permission by copyright holder

“Many people with hearing loss don’t use hearing aids — in part — because they don’t work well in everyday situations like a noisy restaurant, a conversation involving multiple people at a loud party, or in a moving car,” Thomas Lunner, research lead for Facebook’s Hearing Science lab, told Digital Trends. “In conducting our enhanced hearing research for AR glasses, we realized that some of the discoveries we were making could be relevant to solving this problem. In [our new research], we show how augmented reality could supplement the hearing aid, through a system that understands what you want to listen to, isolates and enhances the sounds you want to hear, and reduces distracting background noise. By then sending the enhanced AR-processed signal to the hearing aid, you also get a system that adjusts for your unique hearing ability.”

Playing to Facebook Research’s strengths

Lunner is no novice when it comes to this area of research. He’s a renowned hearing scientist whose work laid the basis for what became the world’s first digital hearing aid in 1995. This research, while still in early stages, taps into a number of areas of research for Facebook, such as image recognition, natural language processing, and a whole lot more.

“Hearing sciences is an area we’re just starting to explore, so this technology is not available on any of our products, but we’re very excited about the possibilities with AR glasses,” Lunner continued. “We think they could one day help to deliver all sorts of enhancements, including helping anyone who has hearing loss or just has trouble understanding people at a party.”

Image used with permission by copyright holder

Michael Abrash, chief scientist at Facebook Reality Labs, told Digital Trends that he believes that both augmented and virtual reality could have profound implications (and applications) in the accessibility space.

“AR and VR have enormous potential to benefit everyone, but they have especially high potential for people with varying physical abilities,” Abrash said. “For example, AR glasses may one day be able to enhance contrast in order to help people with poor vision see better, and may be able to provide visual feedback about ambient sounds and speech to the hearing impaired. As another example, we believe that VR — and eventually AR — will one day be able to create ultra-realistic virtual spaces and avatars that will enable those with mobility challenges to teleport to the office or to their grandmother’s living room.”

Image used with permission by copyright holder

Facebook is, of course, best known for its social network. But just like Google parent company Alphabet, which explores self-driving cars, mapping systems and drone deliveries alongside search, Facebook is interested in far more than just news feeds and timelines. In the past, it has shown considerable interest in both VR and AR — with everything from its acquisition of Oculus to the creation of the Spark AR Studio. Facebook still has yet to show its hand when it comes to a lot of this technology. Still, if this work is anything to go by, it seems that Facebook could be using its considerable power for good when it comes to the future of connecting people.

A paper describing the work, titled “Potential of Augmented Reality Platforms to Improve Individual Hearing Aids and to Support More Ecologically Valid Research,” was recently published in the journal Ear & Hearing.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Lenovo Legion Pro 5 gaming laptop with an RTX 4070 is $400 off
Cyberpunk 2077 on the Lenovo Legion Pro 5.

If you're planning to buy a new gaming laptop, we highly recommend looking for offers involving the Lenovo Legion Pro 5. There's one right now from Lenovo itself -- a $400 discount that pulls the machine's price down from $1,900 to $1,500. It's still not cheap, but we assure you that every penny spent on this gaming laptop will be worth it. You're going to have to hurry with your purchase if you don't want to miss out on getting the device at 21% off though, as its price may return to normal at any moment.

Why you should buy the Lenovo Legion Pro 5 gaming laptop
The Lenovo Legion Pro 5 is our top choice in our roundup of the best gaming laptops because of the fantastic value that it provides as a reasonably priced and well-built machine. It's powered by the AMD Ryzen 7 7745HX processor and the Nvidia GeForce RTX 4070 graphics card, plus 32GB of RAM that's enough to run multiple applications like streaming software and web browsers while playing the best PC games at their highest settings, according to our guide on how much RAM do you need. With these specifications, the Lenovo Legion Pro 5 is also prepared to play the best upcoming PC games without any need for further upgrades.

Read more
Best 17-inch laptop deals: Big savings on big screen laptops
Alienware m17 R5 laptop sitting next to a headset.

17-inch laptops sit right at the top of the line for big screens on laptops. Far from being oddballs, however, they are made by the best laptop brands and many are included amongst our lists of the best laptops. Still though, this is not just a list of the best laptop deals, but bigger. Instead, we've found 17-inch laptop deals that show that you can go bigger (and possibly better) without going broke. Here are our favorite 17-inch laptop deals of the season.
ASUS 17.3-Inch Chromebook — $199, was $299

This is probably the cheapest 17-inch laptop (that you'll actually enjoy using) of the moment. It's also one of our lowest price favorites when it comes to Best Buy laptop deals. It's got 4GB of memory and 64GB of eMMC flash storage so it'll be perfect for browsing the net, watching YouTube on (it has a 1080p resolution), and getting a bit of work done without distractions. Bonuses include super-fast Wi-Fi 6 capabilities, an HD webcam with mic, and even a headphone/microphone combo jack.

Read more
Surface Laptop Go 3 vs. Surface Pro 7+: is it an upgrade?
Microsoft Surface Pro 7

The Surface Laptop Go 3 is Microsoft's new affordable Surface laptop, with upgraded hardware and long battery life making it a compelling option for budget laptop buyers in 2023. But for anyone considering an upgrade to a Surface device at around that $800 price, the Surface Pro 7+ still presents an attractive purchase, and it can flip into a tablet when you need it.

That brings to mind the age-old question, of whether newer is automatically better. In this case, it might not be. Let's take a look at the Surface Laptop Go 3 versus Surface Pro 7+, to find out.

Read more