Skip to main content

Facebook researchers are developing animated selfies based on emoji reactions

Bringing Portraits to Life
Future versions of Facebook could animate your profile picture as followers click the emoji reactions to your posts. Facebook and Tel Aviv University recently published the results of a study that created short animated selfies by starting with a single photo.

Moving photos aren’t anything new — just look at the millions of GIFs floating around Facebook. But all those animations are generally made from short videos or a sequence of still photos. Facebook and the researchers from Tel Aviv University set out to create an animation using only a single photo. (A Google researcher recently took on a project with a similar objective, creating a scenic video using artificial intelligence and a single photo as a starting point.)

The research team built software that maps a facial expression on a reference video and applies it to the still photo. The system uses several facial markers and watches how the expression changes in the reference video. Using similar facial markers on the image, the program then animates the photo using the same motions.

The result is an emoji-like animation of the still photo. But, a few things were still missing — literally. In a still photo where the expression didn’t include teeth, for example, mapping the photo to a smiling animation creates an odd pink space where the teeth should be. The team then took that coarse animation and created an algorithm to fill in the missing details. Of course, those teeth probably don’t match the teeth of the person in the photo, but the researchers say that since people don’t generally identify someone by their teeth, adding in the missing detail created better results than without the “false” teeth.

The team then further refined the results to add fine details to enhance the expression and remove anomalies, like wrinkles created from the animation that were not in the original photo.

Animating a still photo could create a number of different apps for creating animations with little data, like making the Mona Lisa smile. But as the researchers demonstrate in the project’s video, the tool could also create responsive photos on social media. For example, when a follower clicks the like reaction on Facebook, the image could animate with a smile while other reactions, like sad and angry, could have animations matching those emotions.

The resulting animations aren’t perfect with a few oddities to the animation, but further refinement could bring the feature mainstream. The team plans to continue developing the program by allowing the software to choose from a library of reference videos based on the sample that closely matches the facial features for more accurate animations, as well as expanding the work with 3D methods. Paired with the patent for recognizing emotions through a webcam, tracking facial expressions appears to be an area of interest for the social media company’s research.

The project will be presented at Sigggraph Asia in November.

Editors' Recommendations

Hillary K. Grigonis
Hillary never planned on becoming a photographer—and then she was handed a camera at her first writing job and she's been…
Facebook wants to spot political misinformation as it happens
what facebook users should know about cambridge analytica and privacy mobile v1

Facebook is hoping to have fewer “whoops” moments by working to identify threats as they happen -- starting with elections. On Thursday, May 17, Facebook announced a partnership with the Atlantic Council to spot potential election threats with real-time insights.

“Experts from their Digital Forensic Research Lab will work closely with our security, policy, and product teams to get Facebook real-time insights and updates on emerging threats and disinformation campaigns from around the world,” wrote Katie Harbath, the global politics and government outreach director for Facebook. “This will help increase the number of ‘eyes and ears’ we have working to spot potential abuse on our service — enabling us to more effectively identify gaps in our systems, preempt obstacles, and ensure that Facebook plays a positive role during elections all around the world.”

Read more
Facebook opens A.I. research labs in Seattle and Pittsburgh
facebook congress privacy laws zuckerberg feature

Facebook has announced that it is opening two new artificial intelligence research labs in Pittsburgh and Seattle. The labs will include professors hired from the University of Washington and Carnegie Mellon University. This has prompted some fears that Facebook is poaching the instructors needed to train the next generation of A.I. researchers.

"It is worrisome that they are eating the seed corn" Dan Weld, a computer science professor at the University of Washington told the New York Times. "If we lose all our faculty, it will be hard to keep preparing the next generation of researchers."

Read more
Facebook investigating more security vulnerabilities with third-party logins
facebook stories on desktop tested sign

After a whistleblower suggested that many personality quiz apps are designed to track user data, Facebook users have yet another reason to avoid logging in with Facebook credentials. Researchers at Princeton University say lax security could allow third-party platforms to use JavaScript trackers to abuse data on some websites using the “login with Facebook" tool. In a report published on the Freedom to Tinker website hosted by the Center for Information Technology Policy at Princeton Unversity, researchers suggest social login APIs can be abused by third-party scripts through two different vulnerabilities.

The researchers found seven third-party companies accessing Facebook user data through a tool allowing users to log into websites using their Facebook ID. The report suggests that signing in with a social account unknowingly allows the user to trust not just that website, but third-party tools on that same website. 

Read more