Skip to main content

Alexa can now have a conversation with members of the deaf community

amazon echo in the dark
Greg Mombert/Digital Trends

As convenient as voice assistants have proven themselves to be, these A.I.-powered tools have lagged in their inability to serve the deaf community. But one enterprising computer scientist has developed a solution that will allow those with hearing impediments to experience the highs and lows of Amazon Alexa, Google Assistant, and Siri along with everyone else. Abhishek Singh, who first rose to prominence when he built Super Mario Bros in augmented reality, has created a web application that uses a camera to read sign language, and then translates those words into spoken language for an Amazon Echo. When the Echo speaks its response, the whole process plays out in reverse, resulting in a typed reply for hearing impaired or deaf people to read.

This app goes beyond the pseudo-solution provided by the Amazon Echo Show, which added a screen to the smart home hub in order to allow members of the deaf community to interact with the smart home assistant. However, this still did not allow Alexa and its user to truly have a conversation – Singh’s offering, however, fixes this.

“The project was a thought experiment inspired by observing a trend among companies of pushing voice-based assistants as a way to create instant, seamless interactions,” he told Fast Company. “If these devices are to become a central way we interact with our homes or perform tasks, then some thought needs to be given to those who cannot hear or speak. Seamless design needs to be inclusive in nature.”

In building the system, Singh trained an A.I. system within machine learning platform Tensorflow, signing words through his webcam ad nauseam in order to “teach” the system sign language. He then added Google’s text-to-speech capabilities in order to translate the sign language into spoken words.

While Singh’s solution is an elegant one, the coder still hopes that Amazon will ultimately recognize sign language on its own. “That’s where I hope this heads. And if this project leads to a push in that direction in any small way, then mission accomplished,” he said. “In an ideal world, I would have built this on the Show directly, but the devices aren’t that hackable yet, [I] wasn’t able to find a way to do it.”

Editors' Recommendations

Lulu Chang
Former Digital Trends Contributor
Fascinated by the effects of technology on human interaction, Lulu believes that if her parents can use your new app…
What is Amazon Alexa, and what can it do?
Echo 4th Gen speaker on table.

Amazon Alexa is an interactive voice assistant that can check the weather, launch your favorite playlist, and everything in between. Alexa can be found on most Amazon products, including the new Echo Pop, the iconic Echo Dot, and even a variety of smart thermostats, soundbars, lamps, and more. Aside from taking basic commands from you, Alexa can also dish out commands to the rest of your connected smart home -- making it easy to streamline your life.

Interested in learning more about Amazon Alexa? Then you’re in the right place. Here’s a closer look at where Alexa comes from, how it works, where it got its name, and just about everything else you’d want to know about the popular voice assistant and smart home savant.
Who/what is Alexa?

Read more
Amazon to pay $30M in FTC settlements over Alexa, Ring privacy violations
Amazon logo on the headquarters building.

Amazon has agreed to pay $25 million to the Federal Trade Commission (FTC) to settle charges over privacy violations linked to its digital assistant, Alexa.

In a separate case, Amazon-owned Ring will pay $5.8 million for violations of users’ privacy.

Read more
What to do if your Amazon Alexa app is not working
Alexa app on phone.

Amazon Alexa might be one of the most popular smart assistants, but it’s not without a few flaws. For instance, some users have experienced crashes while using the Amazon Alexa app -- and others have found the app to be entirely unresponsive when trying to control their Amazon Echo devices.

Issues like this are typically picked up on by Amazon and immediately fixed with a patch, but sometimes there’s an issue on the user’s end that needs to be remedied before things start functioning as intended. If your Amazon Alexa app is not working or responding, here are a few ways to fix it.
Check for a software update

Read more