Facebook wants to literally open your eyes with A.I. that fixes blinks in photos

facebook research fixes blinking photos facebookimageblinks
Previous programs use am image of someone else’s eyes to fix a blink (third column), but Facebook researchers just created a program that uses your own eyes (fourth column). Facebook Research

Artificial intelligence is now smart enough to paint in the missing regions of a face to fix a blink or turn a frown into a smile. The problem? Those A.I. programs use a stranger’s photograph, so while the programs may fix a bad blink, they also create a Frankenstein of you with someone else’s eyes. Facebook researchers, however, may have come up with a solution by training an A.I. that uses your actual eyes from a previous photograph to fix that blink. Facebook published the research on June 18.

Repairing a photograph with A.I. is nothing new — earlier this year, Nvidia researchers created an A.I. healing tool that could replace missing pixels in a portrait, including missing eyes. While those programs can fix something like a blink, the A.I. relied on a database of images of other people, painting in someone else’s eyes to fix that blink. Besides using someone else’s eyes, the technology also tended to be biased toward certain shapes and colors, Facebook suggests.

But Facebook probably already has a record of your face (or a few hundred selfies), so the company’s researchers decided to try to fix that awkward blink without making it look like you got a new pair of eyes. The researchers created two different approaches to the technique, called eye in-painting. The first uses a reference image to replace the eyes. The second stores an actual version of your eyes, similar to how Facebook stores data for facial recognition, to use in those fixes.

While many A.I. programs work with existing databases of photos to train the program, Facebook created its own data set. The set of celebrity photos taken on a runway by professional photographers didn’t match the badly lit selfies the program is more likely to encounter, while other datasets contained images with people wearing sunglasses. Instead, Facebook created a dataset of around 200,000 people, with at least three images of each person.

The program bases the results on the reference eye in order to help preserve identity, while the A.I. helps create a more realistic result. For example, the network used the data photos to learn how to create a realistic result when the pose or lighting in the reference eyes and the edited photos are different.

The resulting program, Facebook says, fixes eye blinks with results that look more accurately like that specific individual and without losing as much data as earlier programs. The results aren’t perfect — for individuals with hair obscuring part of one eye, for example, the program didn’t accurately recreate the shape of the eye, while in other scenarios, the eye color had slight variations. 

Facebook isn’t done researching the possibility of automatically fixing those blinking selfies, but with some additional work, a future version of the social network could do much more with automatic photo fixes before sharing.

Product Review

Canon Pixma Pro-10 review

If you plan to put those digital high-resolution images you’ve shot with a DSLR onto paper, Canon’s Pixma Pro-10 will deliver gallery-quality prints. Just don’t expect it to be speedy.
Mobile

The best weather apps for Android will keep you dry no matter where you go

You may not be able to change the weather, but you can at least be prepared for it. Check out our guide to the best weather apps for Android, so you'll always know what to expect when you step out the front door.
News

Photography News of the Week: Fujifilm creates A.I. for use in designing albums

Spending too much time culling photos and designing albums? Fujifilm is developing artificial intelligence that could help. The A.I., new camera bags, and extreme weather camera protection are in Photography News of the Week.
Mobile

Memoji in iOS 12 lets you create an avatar that looks and moves just like you

Apple's Memoji feature on iOS 12 allows you to send a customize Animoji that looks exactly like you. In comparison to other apps that allow you to make your own custom avatar, Memoji doesn't overcomplicate it.
Social Media

Facebook is paying cash rewards if you find vulnerabilities in third-party apps

As part of efforts to put the Cambridge Analytica scandal and related issues behind it, Facebook said this week it's expanding its bug bounty program to include third-party apps and websites that could potentially misuse its data.
Photography

To post or not to post? Here's when you should put up a picture on the 'Gram

Let's be honest, the majority of us care about the popularity of our Instagram posts. There is a sweet spot, however, if you're looking to boost the number of likes and comments you receive. Here's what you need to know.
Product Review

There's no such thing as the perfect camera, but Nikon's D750 is damn close

The Nikon D750 DSLR is an excellent example of what full-frame cameras can do. It produces beautiful photos and videos. But does it justify its high price? Yes, it does -- and the camera's age means an even better deal.
Social Media

Instagram could separate hashtags for less annoying posts

Just what features will Instagram add next? One enthusiast reverse engineered the app to uncover several potential features the app could be testing, including a dedicated spot for hashtags and geofencing.
Photography

Canon crams more resolution and speed in the 65x zoom PowerShot SX70 HS

The new Canon PowerShot SX70 HS keeps the same notable 65x zoom lens, but adds resolution, speed, and 4K video while lightening the camera body. The bridge camera is a much-needed update to the 4-year-old SX60 HS.
Photography

These point-and-shoot cameras make your smartphone pics look like cave paintings

If your smartphone camera just isn't giving you the results you're looking for, maybe it's time to step up your game. The latest and greatest point-and-shoot cameras offer large sensors, tough bodies, and long lenses - something no phone…
Social Media

Instagram feature that lets you reshare others’ posts may be on its way

Despite constant calls from many of its billion-plus users, Instagram has always refused to offer an option that lets Instagrammers reshare others' posts that appear on their feed. New reports suggest this could be about to change.
Photo Galleries

Leica continues instant film trend with the classically styled Sofort camera

While the name may sound funny -- Sofort is German for immediately -- the Leica Sofort immediately pops out film photos. The company's first instant film camera uses classic Leica styling but adds a mirror for selfies.
Photography

When you're ready to shoot seriously, these are the best DSLRs you can buy

For many photographers the DSLR is the go-to camera. With large selection of lenses, great low-light performance, and battery endurance, these DSLRs deliver terrific image quality for stills and videos.
Photography

The best mirrorless cameras pack all the power of a DSLR, minus the bulk

Mirrorless cameras offer a lot of photography firepower, inside a compact body. Explore the best mirrorless cameras, from the pro-level to the beginner-friendly shooters, in this guide.