Skip to main content

The incredible underlying technology of Snapchat’s selfie lenses

snapchat selfie lens computer vision technology daven mathies
Daven Mathies / Digital Trends
Snapchat’s selfie lenses, more commonly referred to as filters, are fun, ridiculous, and good for a cheap laugh. Thanks to facial recognition, selfie lenses can do things like transmogrify your boring ol’ mug into a cute puppy or shoot rainbows out of your mouth. It’s a novelty that wears off quickly, so Snapchat continually introduces new lenses to keep the experience fresh.

But for a feature that exists as little more than a gimmick, there’s a surprising amount of underlying technology powering it.

As reported by PetaPixel, the facial recognition engine that makes Snapchat lenses possible was acquired by the company when it purchased Looksery, a Ukrainian computer vision startup. Computer vision is a buzzy phrase these days, with wide ranging applications from depositing a check with your phone to, well, facial recognition. But, as the video below from Vox explains, Snapchat’s lenses have to do much more than simply recognize a face; they also have to model it in 3D space, taking into account rotation and even occlusion so that overlaid props and effects can animate correctly in video.

How Snapchat's filters work

While the technology itself isn’t new, the ability to apply it in real-time from a mobile device is. As smartphones continue to gain processing power, Snapchat will be able to introduce increasingly complex lenses — and the company has a lot of money riding on them.

Snapchat purchased Looksery for a staggering $150 million in 2015 and just this year bought another computer vision startup, Seene. Snapchat clearly sees an advertising opportunity with its selfie lenses, and began introducing sponsored lenses last year, for which it apparently charges no small fee, according to TechCrunch.

As impressive as the technology is, the Vox video ends on a note of caution when it comes to facial recognition and privacy. Currently, both government agencies and private companies like Facebook are storing huge databases of faces linked to real names, and there’s nothing to legally prevent them from doing so.

Making yourself look like a pirate in a Snapchat selfie is probably harmless, though.

Editors' Recommendations

Daven Mathies
Former Digital Trends Contributor
Daven is a contributing writer to the photography section. He has been with Digital Trends since 2016 and has been writing…
What do the eyes mean on your Snapchat Story?
The Snapchat app store listing on a mobile device with a stylus resting on it.

Snapchat (and Snapchat+ for that matter) offers quite a few features to its users and it's understandable if you're not familiar with all of them. One feature, in particular, seems to have a function that doesn't seem obvious at first glance. If you've ever looked at one of your Stories on Snapchat and noticed a pair of eyes emoji next to it and wondered what it meant, you've come to the right place.

In this guide, we'll go over what that pair of eyes icon means in relation to your Snapchat Stories and take a closer look at the Snapchat feature it represents.
What do the eyes mean on a Snapchat Story?
Essentially, according to Snapchat's Support guide on the matter, the pair of eyes emoji is an indicator that your friends have rewatched a given Story that you've posted. In fact, next to that pair of eyes should be a number that denotes the number of friends that rewatched a given Story.

Read more
Snapchat+ now lets you customize when Snaps on Stories expire
A person using Snapchat on an iPhone.

Snapchat+ subscribers have a few more new features to look forward to this week, including the ability to set different expiration dates for their Snaps on Stories.

On Thursday, Snapchat announced four new features for subscribers of its premium version of Snapchat. The most notable of the newly released features is known as "custom Story expiration." This feature allows Snapchat+ subscribers to choose from a series of expiration date options for Snaps on a Story. The options range from letting Snaps expire after one hour to letting them expire after up to one week.

Read more
Snapchat’s new Dual Camera uses both front and rear cameras simultaneously
The Snapchat app store listing on a mobile device with a stylus resting on it.

Ever wanted to take a Snap using both of your phone's cameras simultaneously? You can do it now with Snapchat's latest feature.

On Monday, Snapchat announced the rollout of Dual Camera, a feature that allows Snapchatters to record content using their devices' front and rear cameras at the same time. And judging by the images Snapchat provided in their announcement, using Dual Camera results in capturing two perspectives of the same moment, often letting the person taking the Snap have more of a starring role in the moment that they're recording.

Read more