The incredible underlying technology of Snapchat’s selfie lenses

snapchat selfie lens computer vision technology daven mathies — Daven Mathies / Digital Trends

Snapchat’s selfie lenses, more commonly referred to as filters, are fun, ridiculous, and good for a cheap laugh. Thanks to facial recognition, selfie lenses can do things like transmogrify your boring ol’ mug into a cute puppy or shoot rainbows out of your mouth. It’s a novelty that wears off quickly, so Snapchat continually introduces new lenses to keep the experience fresh.

But for a feature that exists as little more than a gimmick, there’s a surprising amount of underlying technology powering it.

As reported by PetaPixel, the facial recognition engine that makes Snapchat lenses possible was acquired by the company when it purchased Looksery, a Ukrainian computer vision startup. Computer vision is a buzzy phrase these days, with wide ranging applications from depositing a check with your phone to, well, facial recognition. But, as the video below from Vox explains, Snapchat’s lenses have to do much more than simply recognize a face; they also have to model it in 3D space, taking into account rotation and even occlusion so that overlaid props and effects can animate correctly in video.

How Snapchat's filters work

While the technology itself isn’t new, the ability to apply it in real-time from a mobile device is. As smartphones continue to gain processing power, Snapchat will be able to introduce increasingly complex lenses — and the company has a lot of money riding on them.

Snapchat purchased Looksery for a staggering $150 million in 2015 and just this year bought another computer vision startup, Seene. Snapchat clearly sees an advertising opportunity with its selfie lenses, and began introducing sponsored lenses last year, for which it apparently charges no small fee, according to TechCrunch.

As impressive as the technology is, the Vox video ends on a note of caution when it comes to facial recognition and privacy. Currently, both government agencies and private companies like Facebook are storing huge databases of faces linked to real names, and there’s nothing to legally prevent them from doing so.

Making yourself look like a pirate in a Snapchat selfie is probably harmless, though.

Editors' Recommendations