Skip to main content

Amazing new glove can translate sign language into spoken words in real time

Wearable Sign-to-Speech Translation

Researchers at the University of California, Los Angeles, have developed a wearable device, resembling something approaching the Nintendo Power Glove, that’s able to translate American Sign Language into speech in real time using a smartphone app. While it’s still in the prototype phase, it could one day help those who rely on sign language to communicate more easily with non-signers, along with assisting novices who are learning sign language.

“Analog triboelectrification and electrostatic induction-based signals generated by sign language components — including hand configurations and motions, and facial expressions — are converted to the digital domain by the wearable sign-to-speech translation system to implement sign-to-speech translation,” Jun Chen, assistant professor of bioengineering at the UCLA Samueli School of Engineering, told Digital Trends. “Our system offers good mechanical and chemical durability, high sensitivity, quick response times, and excellent stretchability.”

The gloves contain thin, stretchable sensors made of electrically conductive yarn which run along the length of all five fingers. They communicate the finger movements of the wearer to a small, coin-sized circuit board that’s worn on the wrist, which in turn transmits the data to a connected smartphone. Because American Sign Language relies on facial expressions in addition to hand movements, the system also involves sensors adhered to users’ eyebrows and the sides of their mouths. Built around machine learning algorithms, the wearable is currently able to recognize 660 signs, including every letter of the alphabet and numbers zero through nine.

ASL reading system 1
University of California, Los Angeles

Chen said that previous sign language translation devices have been based on a wide range of techniques, including electromyography, the piezoresistive effect, ionic conduction, the capacitive effect, and photography and image processing. But the inherent complexity of these tools, in addition to how cumbersome they are, has made them little more than proof-of-concept lab experiments.

“For example, vision-based sign language translation systems have high requirements for optimal lighting,” Chen said. “If the available lighting is poor, this compromises the visual quality of signing motion captured by the camera and consequently affects the recognition results. Alternatively, sign language translation systems based on surface electromyography have strict requirements for the position of the worn sensors, which can impact translation accuracy and reliability.”

The hope is that this wearable sign-to-speech translation system could be more realistically used in real-world settings. In addition to not being affected by external variables like light, the UCLA sign language wearable could be produced inexpensively. “We are still working to polish the system,” Chen said. “It may take three to five years to get it commercialized.”

A paper describing the work was recently published in the journal Nature Electronics.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
A new bracelet can detect if you’re being attacked, automatically call help
smart bracelet fall or assault smartbracelet

Automating personal safety with wearable smart jewelry

Researchers from the University of Alabama at Birmingham have developed a smart wearable they hope could be used to keep people safe in the face of possible physical or sexual assault, or elderly people experiencing a fall. The Smart Jewelry Bracelet uses a combination of machine learning technology and smart sensors to automatically detect when an assault or fall takes place.

Read more
Tinder tests new ‘Feed’ feature, providing real-time updates on matches

Sometimes, sending the first message on a dating app can be tough. In an effort to provide users with more information to pull from when crafting that first message, Tinder is testing a new feature called "Feed."

Located within your Match List, Feed allows you to see what your matches are up to in real time. As you scroll through Feed, you are able to see recent Tinder photos uploaded to their profile, new top artists and anthems they've set, and Instagram posts.

Read more
Like a mechanical shadow, Toyota’s new robot mimics your movements in real time
toyota robot mirrors movement 20171121 03 01

As a company, Toyota may be best known for its cars, but it sure builds some nice robots, too. Its latest, third-generation humanoid robot, unveiled November 21, is capable of exactly mimicking its operator’s movements. According to Toyota, it could one day serve as a robot assistant in homes, hospitals, and disaster zones, on construction sites -- even in outer space.

Called T-HR3, the 5-foot-1-inch robot is controlled via a proprietary Master Maneuvering System that allows it to be operated by a human user kitted out with special sensors that map their hand, arm, and foot movements. A head-mounted display allows them to see through the robot’s eyes. The robot is then able to replicate movements with impressive accuracy via its 29 articulated body parts. A cleverly designed chair, packing a 16 torque servo module, lets the user move the robot either forward or sideways while sitting still.

Read more