The incredible underlying technology of Snapchat’s selfie lenses

snapchat selfie lens computer vision technology daven mathies
Daven Mathies / Digital Trends
Snapchat’s selfie lenses, more commonly referred to as filters, are fun, ridiculous, and good for a cheap laugh. Thanks to facial recognition, selfie lenses can do things like transmogrify your boring ol’ mug into a cute puppy or shoot rainbows out of your mouth. It’s a novelty that wears off quickly, so Snapchat continually introduces new lenses to keep the experience fresh.

But for a feature that exists as little more than a gimmick, there’s a surprising amount of underlying technology powering it.

As reported by PetaPixel, the facial recognition engine that makes Snapchat lenses possible was acquired by the company when it purchased Looksery, a Ukrainian computer vision startup. Computer vision is a buzzy phrase these days, with wide ranging applications from depositing a check with your phone to, well, facial recognition. But, as the video below from Vox explains, Snapchat’s lenses have to do much more than simply recognize a face; they also have to model it in 3D space, taking into account rotation and even occlusion so that overlaid props and effects can animate correctly in video.

While the technology itself isn’t new, the ability to apply it in real-time from a mobile device is. As smartphones continue to gain processing power, Snapchat will be able to introduce increasingly complex lenses — and the company has a lot of money riding on them.

Snapchat purchased Looksery for a staggering $150 million in 2015 and just this year bought another computer vision startup, Seene. Snapchat clearly sees an advertising opportunity with its selfie lenses, and began introducing sponsored lenses last year, for which it apparently charges no small fee, according to TechCrunch.

As impressive as the technology is, the Vox video ends on a note of caution when it comes to facial recognition and privacy. Currently, both government agencies and private companies like Facebook are storing huge databases of faces linked to real names, and there’s nothing to legally prevent them from doing so.

Making yourself look like a pirate in a Snapchat selfie is probably harmless, though.

Emerging Tech

Would you swap your keycard for a microchip implant? For many, the answer is yes

Put down your keycard! More people are turning to implanted RFID chips as their choice of workplace identification. Should we be worried about a world in which employees get microchipped?
Product Review

Airselfie 2 may as well be a GoPro stapled to a drunk hummingbird

On paper, the Airselfie 2 is marketed as flying photographer that fits in your pocket and snaps selfies from the sky. Unfortunately it’s more like a HandiCam controlled by a swarm of intoxicated bumblebees
Home Theater

From the Roku Ultra to the Fire TV Cube, these are the best streaming devices

There are more options for media streamers than ever, so it’s more difficult to pick the best option. But that’s why we're here. Our curated list of the best streaming devices will get you online in no time.
Digital Trends Live

Indiegogo’s CEO on how crowdfunding is going beyond fundraising

David Mandelbrot, CEO of Indiegogo, joins us on DT Daily to talk about how Indiegogo is changing its crowdfunding business to offer support and resources, which in turn would lead to more successful campaigns.
Product Review

DJI has always been the king of drones, and the new Mavics are almost perfect

After flying both the Mavic 2 Pro and Mavic 2 Zoom for over a week, we’re convinced that these are two of the best drones that DJI has ever made.
Emerging Tech

Awesome Tech You Can’t Buy Yet: A.I. selfie drones, ‘invisible’ wireless chargers

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Mobile

How to choose the best MicroSD cards for your smartphone or tablet

If you need some extra storage space and your device supports it, then a MicroSD card is an obvious place to start. Here, we look at what to consider and suggest some of the best MicroSD cards for your smartphone.
Photography

Pulitzer Prize-winning photographer trades in his gear for a toy camera

Pulitzer Prize-winning photographer Oden Balilty recently rose to the challenge of shooting for an entire day using nothing but a toy camera, and at the end of it even found a few reasons to praise it over his pricey DSLR.
Social Media

Instagram purges fake followers, likes, and comments generated from other apps

Instagram looking a little more authentic? You can thank machine learning. A new tool is helping Instagram spot followers, likes and comments generated from third-party apps -- and this is just the start.
Product Review

Nikkor 105mm f/1.4 may be a beast in size, but its images are absolute beauty

There’s no such thing as a perfect lens, but the Nikkor 105mm f/1.4E ED comes incredibly close. The original 105mm f/1.4 delivers exceptional quality, albeit in a bulky package.
Deals

Cyber Monday 2018: When it takes place and where to find the best deals

Cyber Monday is still a ways off, but it's never too early to start planning ahead. With so many different deals to choose from during one of the biggest shopping holidays of the year, going in with a little know-how makes all the…
Photography

Deluxe D-Lux: New Leica compact camera adds quality and luxury

Essentially a rehoused Panasonic Lumix LX100 II, the D-Lux 7 adds Leica styling -- and price -- to Panasonic tech. The camera retains much from its predecessor, but brings some needed updates and refinements.
Emerging Tech

The best drone photos from around the world

Most of today's drones come equipped with high-end cameras, which are quickly revolutionizing the world of aerial photography as we know it. Here are some of the best drone photos from around the world.