Skip to main content

Amazing new glove can translate sign language into spoken words in real time

Wearable Sign-to-Speech Translation

Researchers at the University of California, Los Angeles, have developed a wearable device, resembling something approaching the Nintendo Power Glove, that’s able to translate American Sign Language into speech in real time using a smartphone app. While it’s still in the prototype phase, it could one day help those who rely on sign language to communicate more easily with non-signers, along with assisting novices who are learning sign language.

Recommended Videos

“Analog triboelectrification and electrostatic induction-based signals generated by sign language components — including hand configurations and motions, and facial expressions — are converted to the digital domain by the wearable sign-to-speech translation system to implement sign-to-speech translation,” Jun Chen, assistant professor of bioengineering at the UCLA Samueli School of Engineering, told Digital Trends. “Our system offers good mechanical and chemical durability, high sensitivity, quick response times, and excellent stretchability.”

The gloves contain thin, stretchable sensors made of electrically conductive yarn which run along the length of all five fingers. They communicate the finger movements of the wearer to a small, coin-sized circuit board that’s worn on the wrist, which in turn transmits the data to a connected smartphone. Because American Sign Language relies on facial expressions in addition to hand movements, the system also involves sensors adhered to users’ eyebrows and the sides of their mouths. Built around machine learning algorithms, the wearable is currently able to recognize 660 signs, including every letter of the alphabet and numbers zero through nine.

ASL reading system 1
University of California, Los Angeles

Chen said that previous sign language translation devices have been based on a wide range of techniques, including electromyography, the piezoresistive effect, ionic conduction, the capacitive effect, and photography and image processing. But the inherent complexity of these tools, in addition to how cumbersome they are, has made them little more than proof-of-concept lab experiments.

“For example, vision-based sign language translation systems have high requirements for optimal lighting,” Chen said. “If the available lighting is poor, this compromises the visual quality of signing motion captured by the camera and consequently affects the recognition results. Alternatively, sign language translation systems based on surface electromyography have strict requirements for the position of the worn sensors, which can impact translation accuracy and reliability.”

The hope is that this wearable sign-to-speech translation system could be more realistically used in real-world settings. In addition to not being affected by external variables like light, the UCLA sign language wearable could be produced inexpensively. “We are still working to polish the system,” Chen said. “It may take three to five years to get it commercialized.”

A paper describing the work was recently published in the journal Nature Electronics.

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
This modular Pebble and Apple Watch underdog just smashed funding goals
UNA Watch

Both the Pebble Watch and Apple Watch are due some fierce competition as a new modular brand, UNA, is gaining some serous backing and excitement.

The UNA Watch is the creation of a Scottish company that wants to give everyone modular control of smartwatch upgrades and repairs.

Read more
Tesla, Warner Bros. dodge some claims in ‘Blade Runner 2049’ lawsuit, copyright battle continues
Tesla Cybercab at night

Tesla and Warner Bros. scored a partial legal victory as a federal judge dismissed several claims in a lawsuit filed by Alcon Entertainment, a production company behind the 2017 sci-fi movie Blade Runner 2049, Reuters reports.
The lawsuit accused the two companies of using imagery from the film to promote Tesla’s autonomous Cybercab vehicle at an event hosted by Tesla CEO Elon Musk at Warner Bros. Discovery (WBD) Studios in Hollywood in October of last year.
U.S. District Judge George Wu indicated he was inclined to dismiss Alcon’s allegations that Tesla and Warner Bros. violated trademark law, according to Reuters. Specifically, the judge said Musk only referenced the original Blade Runner movie at the event, and noted that Tesla and Alcon are not competitors.
"Tesla and Musk are looking to sell cars," Reuters quoted Wu as saying. "Plaintiff is plainly not in that line of business."
Wu also dismissed most of Alcon's claims against Warner Bros., the distributor of the Blade Runner franchise.
However, the judge allowed Alcon to continue its copyright infringement claims against Tesla for its alleged use of AI-generated images mimicking scenes from Blade Runner 2049 without permission.
Alcan says that just hours before the Cybercab event, it had turned down a request from Tesla and WBD to use “an icononic still image” from the movie.
In the lawsuit, Alcon explained its decision by saying that “any prudent brand considering any Tesla partnership has to take Musk’s massively amplified, highly politicized, capricious and arbitrary behavior, which sometimes veers into hate speech, into account.”
Alcon further said it did not want Blade Runner 2049 “to be affiliated with Musk, Tesla, or any Musk company, for all of these reasons.”
But according to Alcon, Tesla went ahead with feeding images from Blade Runner 2049 into an AI image generator to yield a still image that appeared on screen for 10 seconds during the Cybercab event. With the image featured in the background, Musk directly referenced Blade Runner.
Alcon also said that Musk’s reference to Blade Runner 2049 was not a coincidence as the movie features a “strikingly designed, artificially intelligent, fully autonomous car.”

Read more
Apple TV+ just got a price slash that’s tough to resist, and it won’t last long
The Apple TV main screen.

Apple has just quietly announced that it will be slashing the price on its Apple TV+ offering for a limited time deal.

While Apple prices the service at a standard $9.99 per month usually, it has just cut that way down to $2.99 per month.

Read more