Skip to main content

Hiding behind your hands won’t stop next-gen facial recognition software

face palm
Dolgachov/123RF
As evidenced by Apple’s rumored plans to replace Touch ID with facial recognition technology for the iPhone 8, the ability of computers to seamlessly recognize faces is pretty darn impressive these days. The technology is not infallible, however, and there are still things capable of tripping it up. One example? Hands covering faces, which represents a significant challenge, due to how often a particularly animated hand gesture accidentally obscures a speaker’s face. Fortunately, computer science researchers are here to help.

What researchers from the University of Central Florida and Carnegie Mellon University have developed is a method of dealing with the so-called “facial occlusion” problem. Called Hand2Face (which admittedly sounds a little bit like that early 2000s “talk to the hand” meme), they’ve developed technology that can help improve facial recognition technology for a variety of applications — ranging from security to making machines better understand our emotions.

Recommended Videos

“Recognizing and working with facial occlusions are among the challenging problems in computer vision,” Behnaz Nojavanasghari, one of the researchers, told Digital Trends. “Hand-over-face occlusions are particularly challenging as hands and faces have similar colors and textures, and there are a wide variety of hand-over-face occlusions and gestures that can happen. To build accurate and generalizable frameworks, our models need to see large and diverse samples in the training phase. Collecting and annotating large corpus of data is time demanding, and limits many to work with smaller volumes of data, which can result in building models that do not generalize well.”

Arxiv
Arxiv
Please enable Javascript to view this content

A big part of the team’s research involves building a bigger archive of hand-obscured face images for machines to learn from. This meant creating a system for identifying hands in images in the same way that present facial recognition systems identify eyes, noses, or mouths. Larger data sets can then be built up by getting the computer to automatically composite new images by taking hands from one picture and pasting them onto another. To make the synthesized images appear genuine, the computer color-corrects, scales, and orients the hands to emulate realistic images.

That’s not all, though: the method for identifying hand gestures could also be used, alongside facial expressions, to identify emotions. “In a majority of frameworks, facial occlusions are treated as noise and are discarded from analysis,” Nojavanasghari said. “However, these occlusions can convey meaningful information regarding a person’s affective state and should be used as an additional cue.”

As the need for machines to be able to read our emotions grows (consider robot caregivers, teachers, or even just smarter AI assistants like Alexa and Siri), solving problems like this is only going to become more important. You can read an academic paper describing the work here.

Luke Dormehl
Former Digital Trends Contributor
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
U.S. studies new facial recognition tech built with masked faces in mind
China's coronavirus outbreak

The pandemic has proved facial recognition systems such as Apple’s Face ID are not built to recognize people under masks. Now, the National Institute of Standards and Technology, a government body responsible for accessing such systems, has backed it with more conclusive evidence and says it’s now exploring new models that are designed to handle masked faces.

In its latest study, the NIST has revealed that masks can significantly thwart facial recognition algorithms’ accuracy and raise the error rate to as much as 50% -- even in the case of some of the best and widely used commercial platforms. The report adds that the systems performed worse when the mask was black in color and was worn higher up the nose area.

Read more
Facebook ordered to pay $650 million in facial recognition lawsuit
The Facebook home page on a screen.

 

A federal judge has ordered Facebook to pay $650 million -- $100 million more than originally agreed -- to settle a 2015 facial recognition lawsuit, according to a Wednesday court filing.

Read more
Democratic lawmakers propose nationwide facial recognition ban
democrats propose facial recognition ban crowd getty

 

Four Democratic members of Congress unveiled a bill Thursday that would seek to ban facial recognition and other types of surveillance nationwide.

Read more