Skip to main content

Facebook wants to literally open your eyes with A.I. that fixes blinks in photos

Previous programs use am image of someone else’s eyes to fix a blink (third column), but Facebook researchers just created a program that uses your own eyes (fourth column). Facebook Research

Artificial intelligence is now smart enough to paint in the missing regions of a face to fix a blink or turn a frown into a smile. The problem? Those A.I. programs use a stranger’s photograph, so while the programs may fix a bad blink, they also create a Frankenstein of you with someone else’s eyes. Facebook researchers, however, may have come up with a solution by training an A.I. that uses your actual eyes from a previous photograph to fix that blink. Facebook published the research on June 18.

Repairing a photograph with A.I. is nothing new — earlier this year, Nvidia researchers created an A.I. healing tool that could replace missing pixels in a portrait, including missing eyes. While those programs can fix something like a blink, the A.I. relied on a database of images of other people, painting in someone else’s eyes to fix that blink. Besides using someone else’s eyes, the technology also tended to be biased toward certain shapes and colors, Facebook suggests.

But Facebook probably already has a record of your face (or a few hundred selfies), so the company’s researchers decided to try to fix that awkward blink without making it look like you got a new pair of eyes. The researchers created two different approaches to the technique, called eye in-painting. The first uses a reference image to replace the eyes. The second stores an actual version of your eyes, similar to how Facebook stores data for facial recognition, to use in those fixes.

While many A.I. programs work with existing databases of photos to train the program, Facebook created its own data set. The set of celebrity photos taken on a runway by professional photographers didn’t match the badly lit selfies the program is more likely to encounter, while other datasets contained images with people wearing sunglasses. Instead, Facebook created a dataset of around 200,000 people, with at least three images of each person.

The program bases the results on the reference eye in order to help preserve identity, while the A.I. helps create a more realistic result. For example, the network used the data photos to learn how to create a realistic result when the pose or lighting in the reference eyes and the edited photos are different.

The resulting program, Facebook says, fixes eye blinks with results that look more accurately like that specific individual and without losing as much data as earlier programs. The results aren’t perfect — for individuals with hair obscuring part of one eye, for example, the program didn’t accurately recreate the shape of the eye, while in other scenarios, the eye color had slight variations. 

Facebook isn’t done researching the possibility of automatically fixing those blinking selfies, but with some additional work, a future version of the social network could do much more with automatic photo fixes before sharing.

Editors' Recommendations

Hillary K. Grigonis
Hillary never planned on becoming a photographer—and then she was handed a camera at her first writing job and she's been…
New A.I. hearing aid learns your listening preferences and makes adjustments
Widex Moment hearing aids.

One of the picks for this year’s CES 2021 Innovation Awards is a smart hearing aid that uses artificial intelligence to improve the audio experience in a couple of crucial ways.

Among the improvements the Widex Moment makes to conventional hearing aids is reducing the standard sound delay experienced by wearers from 7 to 10 milliseconds seconds down to just 0.5 milliseconds. This results in a more natural sound experience for users, rather than the out-of-sync audio experience people have had to settle for up until now.

Read more
Futuristic new appliance uses A.I. to sort and prep your recycling
Lasso

Lasso: The power to change recycling for good

Given the potential planet-ruining stakes involved, you’d expect that everyone on Earth would be brilliant at recycling. But folks are lazy and, no matter how much we might see footage of plastic-clogged oceans on TV, the idea of sorting out the plastic, glass, and paper for the weekly recycling day clearly strikes many as just a little bit too much effort.

Read more
A.I. fail as robot TV camera follows bald head instead of soccer ball
ai fail as robot camera mistakes bald head for soccer ball

CaleyJags : SPFL Championship : Real Highlights: ICTFC 1 v 1 AYR : 24/10/2020

While artificial intelligence (A.I.) has clearly made astonishing strides in recent years, the technology is still susceptible to the occasional fail.

Read more