Lip reading AI smashes humans at interpreting silent sentences

One of the most memorable parts of Stanley Kubrick’s sci-fi masterpiece 2001: A Space Odyssey is a plotline in which two members of the Discovery One spaceship crew grow increasingly suspicious about the behaviour of the ship’s AI assistant, HAL 9000.

Knowing that HAL is constantly listening to what they are saying, they retreat someplace they know HAL cannot listen and agree to disconnect him. HAL rumbles their plan after the two astronauts fail to take into account the AI’s superior lip-reading capabilities.

Futuristic stuff, eh? Not according to research carried out by investigators at Oxford University. They’ve developed an artificial intelligence program called LipNet, which is able to accurately interpret what people are saying, based purely on the way they move their mouth when speaking.

“LipNet performs lip-reading at the sentence-level using machine learning,” Brendan Shillingford, one of the researchers on the paper, told Digital Trends. “A neural network similar to state-of-the-art speech recognition models processes a sequence of video frames, mapping these to a sentence. Previous approaches worked by predicted individual words rather than sentences.”

The performance of LipNet compares incredibly favorably to human lipreading experts on GRID corpus, the largest publicly-available sentence-level lipreading dataset. In fact, where human experts got just 52 percent, LipNet scored 93 percent. Its sentence-based approach to lip-reading also smashed the best previous attempt by a machine, which managed 79.6 percent accuracy on the same dataset.

However, while the fictitious HAL 9000 uses his lip-reading powers for no good, the team behind LipNet have other goals for their creation. Around 360 million people worldwide have disabling hearing loss. Tools like LipNet could be highly significant for these individuals, by helping to accurately interpret speech in a way that makes their lives easier.

“Other applications that we are interested in include silent dictation in public spaces, covert conversations, speech recognition in noisy environments, biometric identification, and silent-movie processing,” Shillingford continued.

While surveillance is going to be an issue with any technology like this, Nando de Freitas, who also worked on the project, said that it is not an application they have focused on. However, he said that it “would not be surprising” if other labs tried to build on such work for that purpose in the future.

“The public must be aware of this, and rely on our legal democratic institutions to establish appropriate laws that protect our privacy and dignity,” de Freitas continued. “It is our hope that by publishing this work, we help raise awareness, while still emphasizing the usefulness of this tech to help people in need.”

Emerging Tech

Leafy greens are grown by machines at new, automated Silicon Valley farm

Farming hasn't changed too much for hundreds of years. Now a new startup called Iron Ox has opened its first automated hydroponics farm, producing a variety of leafy greens tended by machines.
Emerging Tech

What the heck is machine learning, and why is it everywhere these days?

Machine learning has been responsible for some of the biggest advances in artificial intelligence over the past decade. But what exactly is it? Check out our handy beginner's guide.
Movies & TV

The best movies on Netflix in October, from 'The Witch’ to ‘Black Panther’

Save yourself from hours wasted scrolling through Netflix's massive library by checking out our picks for the streamer's best movies available right now, whether you're into explosive action, subdued humor, or anything in between.

How General Motors is preparing for an electric, autonomous, connected future

How does an established automaker like General Motors adapt to new tech trends? By looking for people and solutions outside the mainstream, says Ken Kelzer, GM VP of global vehicle components and subsystems.
Emerging Tech

New ‘parkour’ video shows Boston Dynamics robot training to overthrow humanity

Robots doing backflips? That's so 2017! In its latest jaw-dropping YouTube video, Boston Dynamics’ Atlas robot pulls off some frankly astonishing parkour stunts for our viewing pleasure.
Emerging Tech

With VR dinosaurs and ‘Minecraft,’ one hospital is making medicine less scary

From augmented reality rabbits on the wards to a Minecraft recreation of the hospital for kids to explore, one of the world's most renowned children's hospitals just got a major tech overhaul.
Emerging Tech

Will we ever fly supersonic again? Unraveling the concorde’s complex legacy

In a new book, Last Days of the Concorde, journalist and author Samme Chittum delves into the mindset that inspired engineers to design this marvel, the series of events that led to its fatal crash, and the possibility that commercial SSTs…
Emerging Tech

Check out the British Army’s beefy new bomb-disposal robot

The British Army is about to get an impressive new explosive ordnance disposal robot that is able to climb stairs, negotiate slopes, cut wires, and … oh, yes, dispose of bombs, too.
Emerging Tech

Kill it before it lays eggs! Crazy 32-leg robot moves like a cyborg sea urchin

We’ve seen one-legged, two-legged, four-legged and even six-legged robots, but researchers from Japan have gone way, way further with their latest project: A 32-legged robot. Check it out.
Emerging Tech

Awesome Tech You Can’t Buy Yet: DIY smartphones and zip-on bike tires

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!

As deaf gamers speak up, game studios are finally listening to those who can’t

Using social media, personal blogs and Twitch, a small group of deaf and hard-of-hearing players have been working to make their voices heard and improve accessibility in the gaming industry.
Emerging Tech

From flying for fun to pro filmmaking, these are the best drones you can buy

In just the past few years, drones have transformed from a geeky hobbyist affair to a full-on cultural phenomenon. Here's a no-nonsense rundown of the best drones you can buy right now, no matter what kind of flying you plan to do.
Emerging Tech

Get your head in the clouds with the best vaporizers for flower and concentrates

Why combust dead plant matter when you could vaporize the good stuff and leave the leaves behind? Here's a rundown of the best vaporizers money can buy, no matter what your style is.
Emerging Tech

Here’s all the best gear and gadgetry you can snag for $100 or less

A $100 bill can get you further than you might think -- so long as you know where to look. Check out our picks for the best tech under $100, whether you're in the market for headphones or a virtual-reality headset.