Skip to main content

Hiding behind your hands won’t stop next-gen facial recognition software

As evidenced by Apple’s rumored plans to replace Touch ID with facial recognition technology for the iPhone 8, the ability of computers to seamlessly recognize faces is pretty darn impressive these days. The technology is not infallible, however, and there are still things capable of tripping it up. One example? Hands covering faces, which represents a significant challenge, due to how often a particularly animated hand gesture accidentally obscures a speaker’s face. Fortunately, computer science researchers are here to help.

What researchers from the University of Central Florida and Carnegie Mellon University have developed is a method of dealing with the so-called “facial occlusion” problem. Called Hand2Face (which admittedly sounds a little bit like that early 2000s “talk to the hand” meme), they’ve developed technology that can help improve facial recognition technology for a variety of applications — ranging from security to making machines better understand our emotions.

Recommended Videos

“Recognizing and working with facial occlusions are among the challenging problems in computer vision,” Behnaz Nojavanasghari, one of the researchers, told Digital Trends. “Hand-over-face occlusions are particularly challenging as hands and faces have similar colors and textures, and there are a wide variety of hand-over-face occlusions and gestures that can happen. To build accurate and generalizable frameworks, our models need to see large and diverse samples in the training phase. Collecting and annotating large corpus of data is time demanding, and limits many to work with smaller volumes of data, which can result in building models that do not generalize well.”

Arxiv
Arxiv

A big part of the team’s research involves building a bigger archive of hand-obscured face images for machines to learn from. This meant creating a system for identifying hands in images in the same way that present facial recognition systems identify eyes, noses, or mouths. Larger data sets can then be built up by getting the computer to automatically composite new images by taking hands from one picture and pasting them onto another. To make the synthesized images appear genuine, the computer color-corrects, scales, and orients the hands to emulate realistic images.

That’s not all, though: the method for identifying hand gestures could also be used, alongside facial expressions, to identify emotions. “In a majority of frameworks, facial occlusions are treated as noise and are discarded from analysis,” Nojavanasghari said. “However, these occlusions can convey meaningful information regarding a person’s affective state and should be used as an additional cue.”

As the need for machines to be able to read our emotions grows (consider robot caregivers, teachers, or even just smarter AI assistants like Alexa and Siri), solving problems like this is only going to become more important. You can read an academic paper describing the work here.

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
U.S. studies new facial recognition tech built with masked faces in mind
China's coronavirus outbreak

The pandemic has proved facial recognition systems such as Apple’s Face ID are not built to recognize people under masks. Now, the National Institute of Standards and Technology, a government body responsible for accessing such systems, has backed it with more conclusive evidence and says it’s now exploring new models that are designed to handle masked faces.

In its latest study, the NIST has revealed that masks can significantly thwart facial recognition algorithms’ accuracy and raise the error rate to as much as 50% -- even in the case of some of the best and widely used commercial platforms. The report adds that the systems performed worse when the mask was black in color and was worn higher up the nose area.

Read more
Facebook ordered to pay $650 million in facial recognition lawsuit
The Facebook home page on a screen.

 

A federal judge has ordered Facebook to pay $650 million -- $100 million more than originally agreed -- to settle a 2015 facial recognition lawsuit, according to a Wednesday court filing.

Read more
Sebastian Stan lays out Bucky’s future after Thunderbolts
Sebastian Stan in Thunderbolts.

There are some spoilers ahead for the ending of Marvel's Thunderbolts. Stop reading now if you don't want to be spoiled.

Earlier this year, Captain America: Brave New World briefly introduced a new direction for James "Bucky" Barnes, a character Sebastian Stan has been playing since 2011 in Captain America: The First Avenger. In Brave New World, the former Winter Soldier apparently retired from being a reformed hero and went into politics by running for Congress. Thunderbolts reveals that Bucky won his election to the House of Representatives. But his stay in Congress was short.

Read more