Lip reading AI smashes humans at interpreting silent sentences

One of the most memorable parts of Stanley Kubrick’s sci-fi masterpiece 2001: A Space Odyssey is a plotline in which two members of the Discovery One spaceship crew grow increasingly suspicious about the behaviour of the ship’s AI assistant, HAL 9000.

Knowing that HAL is constantly listening to what they are saying, they retreat someplace they know HAL cannot listen and agree to disconnect him. HAL rumbles their plan after the two astronauts fail to take into account the AI’s superior lip-reading capabilities.

Futuristic stuff, eh? Not according to research carried out by investigators at Oxford University. They’ve developed an artificial intelligence program called LipNet, which is able to accurately interpret what people are saying, based purely on the way they move their mouth when speaking.

“LipNet performs lip-reading at the sentence-level using machine learning,” Brendan Shillingford, one of the researchers on the paper, told Digital Trends. “A neural network similar to state-of-the-art speech recognition models processes a sequence of video frames, mapping these to a sentence. Previous approaches worked by predicted individual words rather than sentences.”

The performance of LipNet compares incredibly favorably to human lipreading experts on GRID corpus, the largest publicly-available sentence-level lipreading dataset. In fact, where human experts got just 52 percent, LipNet scored 93 percent. Its sentence-based approach to lip-reading also smashed the best previous attempt by a machine, which managed 79.6 percent accuracy on the same dataset.

However, while the fictitious HAL 9000 uses his lip-reading powers for no good, the team behind LipNet have other goals for their creation. Around 360 million people worldwide have disabling hearing loss. Tools like LipNet could be highly significant for these individuals, by helping to accurately interpret speech in a way that makes their lives easier.

“Other applications that we are interested in include silent dictation in public spaces, covert conversations, speech recognition in noisy environments, biometric identification, and silent-movie processing,” Shillingford continued.

While surveillance is going to be an issue with any technology like this, Nando de Freitas, who also worked on the project, said that it is not an application they have focused on. However, he said that it “would not be surprising” if other labs tried to build on such work for that purpose in the future.

“The public must be aware of this, and rely on our legal democratic institutions to establish appropriate laws that protect our privacy and dignity,” de Freitas continued. “It is our hope that by publishing this work, we help raise awareness, while still emphasizing the usefulness of this tech to help people in need.”

Features

Has Columbus, Ohio raised its IQ yet? A progress report from the mayor

Two years ago, the city of Columbus in Ohio received $40 million to pursue smart city initiatives. So, what’s happened since then? We spoke with its mayor, Andrew Ginther, to discuss progress and what’s ahead.
Emerging Tech

An A.I. cracks the internet’s squiggly letter bot test in 0.5 seconds

How do you prove that you’re a human when communicating on the internet? The answer used to be by solving a CAPTCHA puzzle. But maybe not for too much longer. Here is the reason why.
Home Theater

The best movies on Netflix in December, from 'Buster Scruggs’ to endangered cats

Save yourself from hours wasted scrolling through Netflix's massive library by checking out our picks for the streamer's best movies available right now, whether you're into explosive action, witty humor, or anything else.
Home Theater

From the Roku Ultra to the Fire TV Cube, these are the best streaming devices

There are more options for media streamers than ever, so it’s more difficult to pick the best option. But that’s why we're here. Our curated list of the best streaming devices will get you online in no time.
Emerging Tech

Feast your eyes on the wildest, most elaborate Rube Goldberg machines ever built

Want to see something totally mesmerizing? Check out several of the best Rube Goldberg machines from across the internet, including one that serves cake and other that do ... nothing particularly useful.
Emerging Tech

The 20 best tech toys for kids will make you wish you were 10 again

Looking for the perfect toy or gadget for your child? Thankfully, we've rounded up some of our personal favorite tech toys, including microscopes, computer kits, and a spherical droid from a galaxy far, far away.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Folding canoes and ultra-fast water filters

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Emerging Tech

Scoot your commute! Here are the 9 best electric scooters on the market

Electric scooters are an affordable, convenient way to minimize your carbon footprint and zip around town. Check out 8 of our current favorites, whether you're working with a budget or have some cash to spare.
Emerging Tech

Sick of walking everywhere? Here are the best electric skateboards you can buy

Thanks for Kickstarter and Indiegogo, electric skateboards are carving a bigger niche than you might think. Whether you're into speed, mileage, or something a bit more stylish, here are the best electric skateboards on the market.
Emerging Tech

Hear the sounds of wind on Mars from InSight’s latest audio recording

NASA's InSight craft has captured the sound of the wind blowing on the surface of Mars. The audio file was picked up by the air pressure sensor and the seismometer which detected vibrations from the 10 to 15 mph winds in the area.
Emerging Tech

New experiment casts doubt on claims to have identified dark matter

A South Korean experiment called COSINE-100 has attempted to replicate the claims of dark matter observed by the Italian DAMA/LIBRA experiment, but has failed to replicate the observations.
Emerging Tech

White dwarf star unexpectedly emitting bright ‘supersoft’ X-rays

NASA's Chandra Observatory has discovered a white dwarf star which is emitting supersoft X-rays, calling into question the conventional wisdom about how X-rays are produced by dying stars.
Business

Amazon scouted airport locations for its cashier-free Amazon Go stores

Representatives of Amazon Go checkout-free retail stores connected with officials at Los Angeles and San Jose airports in June to discuss the possibility of cashier-free grab-and-go locations in busy terminals.
Emerging Tech

Full-fledged drone delivery service set to land in remote Canadian community

Some drone delivery operations seem rather crude in their execution, but Drone Delivery Canada is building a comprehensive platform that's aiming to take drone delivery to the next level.