Skip to main content

Facebook researchers are developing animated selfies based on emoji reactions

Bringing Portraits to Life
Future versions of Facebook could animate your profile picture as followers click the emoji reactions to your posts. Facebook and Tel Aviv University recently published the results of a study that created short animated selfies by starting with a single photo.

Moving photos aren’t anything new — just look at the millions of GIFs floating around Facebook. But all those animations are generally made from short videos or a sequence of still photos. Facebook and the researchers from Tel Aviv University set out to create an animation using only a single photo. (A Google researcher recently took on a project with a similar objective, creating a scenic video using artificial intelligence and a single photo as a starting point.)

The research team built software that maps a facial expression on a reference video and applies it to the still photo. The system uses several facial markers and watches how the expression changes in the reference video. Using similar facial markers on the image, the program then animates the photo using the same motions.

The result is an emoji-like animation of the still photo. But, a few things were still missing — literally. In a still photo where the expression didn’t include teeth, for example, mapping the photo to a smiling animation creates an odd pink space where the teeth should be. The team then took that coarse animation and created an algorithm to fill in the missing details. Of course, those teeth probably don’t match the teeth of the person in the photo, but the researchers say that since people don’t generally identify someone by their teeth, adding in the missing detail created better results than without the “false” teeth.

The team then further refined the results to add fine details to enhance the expression and remove anomalies, like wrinkles created from the animation that were not in the original photo.

Animating a still photo could create a number of different apps for creating animations with little data, like making the Mona Lisa smile. But as the researchers demonstrate in the project’s video, the tool could also create responsive photos on social media. For example, when a follower clicks the like reaction on Facebook, the image could animate with a smile while other reactions, like sad and angry, could have animations matching those emotions.

The resulting animations aren’t perfect with a few oddities to the animation, but further refinement could bring the feature mainstream. The team plans to continue developing the program by allowing the software to choose from a library of reference videos based on the sample that closely matches the facial features for more accurate animations, as well as expanding the work with 3D methods. Paired with the patent for recognizing emotions through a webcam, tracking facial expressions appears to be an area of interest for the social media company’s research.

The project will be presented at Sigggraph Asia in November.

Editors' Recommendations

Hillary K. Grigonis
Hillary never planned on becoming a photographer—and then she was handed a camera at her first writing job and she's been…
X (formerly Twitter) returns after global outage
A white X on a black background, which could be Twitter's new logo.

X, formerly known as Twitter, went down for about 90 minutes for users worldwide early on Thursday ET.

Anyone opening the social media app across all platforms was met with a blank timeline. On desktop, users saw a message that simply read, "Welcome to X," while on mobile the app showed suggestions for accounts to follow.

Read more
How to create multiple profiles on a Facebook account
A series of social media app icons on a colorful smartphone screen.

Facebook (and, by extension, Meta) are particular in the way that they allow users to create accounts and interact with their platform. Being the opposite of the typical anonymous service, Facebook sticks to the rule of one account per one person. However, Facebook allows its users to create multiple profiles that are all linked to one main Facebook account.

In much the same way as Japanese philosophy tells us we have three faces — one to show the world, one to show family, and one to show no one but ourselves — these profiles allow us to put a different 'face' out to different aspects or hobbies. One profile can keep tabs on your friends, while another goes hardcore into networking and selling tech on Facebook Marketplace.

Read more
How to set your Facebook Feed to show most recent posts
A smartphone with the Facebook app icon on it all on a white marble background.

Facebook's Feed is designed to recommend content you'd most likely want to see, and it's based on your Facebook activity, your connections, and the level of engagement a given post receives.

But sometimes you just want to see the latest Facebook posts. If that's you, it's important to know that you're not just stuck with Facebook's Feed algorithm. Sorting your Facebook Feed to show the most recent posts is a simple process:

Read more