Hiding behind your hands won’t stop next-gen facial recognition software

face palm
Dolgachov/123RF
As evidenced by Apple’s rumored plans to replace Touch ID with facial recognition technology for the iPhone 8, the ability of computers to seamlessly recognize faces is pretty darn impressive these days. The technology is not infallible, however, and there are still things capable of tripping it up. One example? Hands covering faces, which represents a significant challenge, due to how often a particularly animated hand gesture accidentally obscures a speaker’s face. Fortunately, computer science researchers are here to help.

What researchers from the University of Central Florida and Carnegie Mellon University have developed is a method of dealing with the so-called “facial occlusion” problem. Called Hand2Face (which admittedly sounds a little bit like that early 2000s “talk to the hand” meme), they’ve developed technology that can help improve facial recognition technology for a variety of applications — ranging from security to making machines better understand our emotions.

“Recognizing and working with facial occlusions are among the challenging problems in computer vision,” Behnaz Nojavanasghari, one of the researchers, told Digital Trends. “Hand-over-face occlusions are particularly challenging as hands and faces have similar colors and textures, and there are a wide variety of hand-over-face occlusions and gestures that can happen. To build accurate and generalizable frameworks, our models need to see large and diverse samples in the training phase. Collecting and annotating large corpus of data is time demanding, and limits many to work with smaller volumes of data, which can result in building models that do not generalize well.”

facial recognition occlusion system face hand
Arxiv
Arxiv

A big part of the team’s research involves building a bigger archive of hand-obscured face images for machines to learn from. This meant creating a system for identifying hands in images in the same way that present facial recognition systems identify eyes, noses, or mouths. Larger data sets can then be built up by getting the computer to automatically composite new images by taking hands from one picture and pasting them onto another. To make the synthesized images appear genuine, the computer color-corrects, scales, and orients the hands to emulate realistic images.

That’s not all, though: the method for identifying hand gestures could also be used, alongside facial expressions, to identify emotions. “In a majority of frameworks, facial occlusions are treated as noise and are discarded from analysis,” Nojavanasghari said. “However, these occlusions can convey meaningful information regarding a person’s affective state and should be used as an additional cue.”

As the need for machines to be able to read our emotions grows (consider robot caregivers, teachers, or even just smarter AI assistants like Alexa and Siri), solving problems like this is only going to become more important. You can read an academic paper describing the work here.

Emerging Tech

Doctors could soon ditch stitches and seal skin wounds with lasers

Just like the dermal regenerator in Star Trek, physicians may soon be able to heal skin wounds using smart, laser-based technology. That's thanks to researchers from Arizona State University.
Movies & TV

Fresh footage and Willem Dafoe make the final 'Aquaman' trailer special

Jason Momoa will bring Aquaman back to the big screen for a solo feature in December 2018. Here's everything we know so far about the aquatic superhero's live-action adventure in the DC Extended Universe.
Emerging Tech

Ghostly galaxy discovered lurking on the edge of the Milky Way

A team of astronomers from the University of Cambridge have discovered a strange galaxy next door to the Milky Way. The dwarf galaxy, named Antlia 2, is dark and dim and gives out much less light than expected.
Emerging Tech

Frogs regrow ‘paddle-like’ limbs when placed in a bioreactor

Frogs have partially regrown amputated limbs thanks to a bioreactor at Tufts University. By jump-starting tissue repair, the bioreactor helped the amphibians regenerate a bigger, more complete appendages than they usually do.
Emerging Tech

Ancient crater the size of NYC discovered under the Greenland ice sheet

A huge crater has been discovered beneath the ice of Greenland, and is thought to be the result of a meteorite impact millions of years ago. The crater is one of the largest ever discovered, measuring 19 miles across.
Emerging Tech

Here’s how the InSight mission to Mars will confirm its landing to NASA

NASA's InSight mission has sent a lander to Mars. NASA researchers have now shared details on how they will monitor the touching down of the lander at the end of its 91 million mile journey.
Emerging Tech

Would you swap your keycard for a microchip implant? For many, the answer is yes

Put down your keycard! More people are turning to implanted RFID chips as their choice of workplace identification. Should we be worried about a world in which employees get microchipped?
Outdoors

‘Super magnesium’ may be the next wonder material for outdoor gear

Super Magnesium is a wonder material that is 30 percent lighter than aluminum, as strong as carbon fiber, cheaper to make, and 100-percent recyclable, making it much better for the environment.
Emerging Tech

Forget joysticks — the Guts Game is controlled by a sensor that you swallow

Researchers have created an unusual new game in which players swallow a biosensor and then compete to raise or lower the temperature in their gut. Sound crazy? Here's why it could catch on.
Emerging Tech

Step inside the Nepalese restaurant staffed by robot waiters

A robotics startup from Nepal has created a robot waiter called Ginger. It's capable of delivering food from kitchen to table, and can even engage customers in a bit of friendly banter as it does so.
Emerging Tech

From tornado flushes to remote controls, modern toilets are flush with tech

With the global observance of World Toilet Day on November 19, we take a look at how the modern toilet in our homes and businesses have evolved, and how they are becoming smarter tools in the future.
Emerging Tech

NASA selects the all-important landing site for its Mars 2020 rover mission

NASA said on Monday that the landing site for its much-anticipated Mars 2020 rover mission has the potential to "revolutionize how we think about Mars and its ability to harbor life."
Emerging Tech

NASA’s ‘space wheat’ is helping earthbound farmers grow crops quicker

Could NASA technology for growing plants on other planets help farmers improve crop yield here on Earth? According to researchers in Australia and the U.K., the answer is a resounding yes.
Emerging Tech

Toilet-scrubbing robot takes over one of the world’s crappiest jobs

When it comes to jobs that none of us enjoy, scrubbing the toilet would have to rank pretty highly. So why not hand the job over to robots? Very soon you might be able to do exactly that.