Skip to main content

Facebook touts groundbreaking app AI that powers artistic Live video filters

facebook prisma filters fb style transfer
Image used with permission by copyright holder
Facebook is betting on video filters (the graphics pioneered by Snapchat that can be overlaid on faces in recorded clips) to get more users hooked on its live-streaming tool. Having already demoed its upcoming style transfer graphics (artistic filters that draw from the style of famous painters to transform videos into works of art), it is now offering more details about the artificial intelligence powering its new mobile tools.

Instead of Snapchat, the inspiration for these Live video filters comes from Prisma — an AI-powered photography app that boasts a range of artistic graphics that can be applied to both images and video. Facebook is currently testing style transfers as part of an update that gives added prominence to your smartphone camera within its flagship app in Ireland.

Unlike Snapchat (the app that pioneered filters or, as it calls them, “lenses”), Facebook wants to apply the software to live broadcasts in real time and not after you’ve captured a clip. It’s already started doing this with some of its traditional, Snapchat-style filters, but style transfers are — according to Facebook — its crowning achievement.

On Tuesday, the company published a new blog post explaining in-depth how the style transfer technique works in real-time. Instead of waiting for the image to be processed on huge computer servers in data centers, Facebook claims it has developed a world-first: a deep-learning platform on mobile that puts the cutting-edge technology in the palm of a user’s hand.

Facebook is essentially embedding its neural network (a machine learning system, loosely modelled on the human brain, that learns by consuming vast tons of data) into its Facebook app for iOS and Android. The company terms this deep learning AI “Caffe2Go,” describing it as a lightweight and modular framework built on top of the open-source “Caffe2” project — meaning it won’t drain your mobile device. Facebook is also optimizing style transfer tech, allowing it to be applied to video running at 20 frames per second on an iPhone 6S or above without a drop in picture quality.

fb_style_transfer
Image used with permission by copyright holder

“The style-transfer tool in the camera is the result of a marriage between two technologies: the Caffe2go runtime and style-transfer models,” write Facebook research scientists Yangqing Jia and Peter Vajda in their blog post. “It took both technologies to make it possible for you to feel like you have Van Gogh’s paintbrush in your hand when you pick up your phone to shoot a video.”

In a second blog post, Facebook CTO Mike Schroepfer details the company’s vision for the mobile AI: “We can create gesture-based controls, where the computer can see where you’re pointing and activate different styles or commands. We can recognize facial expressions and perform related actions, like putting a “yay” filter over your selfie when you smile.”

Facebook CEO Mark Zuckerberg demoed the style transfer feature in October, shortly after which Facebook launched its first Live video filters.

Facebook has not shied away from boasting about its plans for the live-streaming format, and video in general. Last month, the company’s chief product officer Chris CoxCox claimed it now predicts video will generate 70 percent of traffic across Facebook’s properties — such as Instagram, WhatsApp, and its flagship social network — in the next five years. Coincidentally, WhatsApp launched video calling for Android devices in late October — indicating that Facebook is wasting no time in making that future a reality.

Updated on 11-08-2016 by Saqib Shah: Added news of Facebook’s reveal of AI for mobile app

Saqib Shah
Former Digital Trends Contributor
Saqib Shah is a Twitter addict and film fan with an obsessive interest in pop culture trends. In his spare time he can be…
How to make a GIF from a YouTube video
woman sitting and using laptop

Sometimes, whether you're chatting with friends or posting on social media, words just aren't enough -- you need a GIF to fully convey your feelings. If there's a moment from a YouTube video that you want to snip into a GIF, the good news is that you don't need complex software to so it. There are now a bunch of ways to make a GIF from a YouTube video right in your browser.

If you want to use desktop software like Photoshop to make a GIF, then you'll need to download the YouTube video first before you can start making a GIF. However, if you don't want to go through that bother then there are several ways you can make a GIF right in your browser, without the need to download anything. That's ideal if you're working with a low-specced laptop or on a phone, as all the processing to make the GIF is done in the cloud rather than on your machine. With these options you can make quick and fun GIFs from YouTube videos in just a few minutes.
Use GIFs.com for great customization
Step 1: Find the YouTube video that you want to turn into a GIF (perhaps a NASA archive?) and copy its URL.

Read more
I paid Meta to ‘verify’ me — here’s what actually happened
An Instagram profile on an iPhone.

In the fall of 2023 I decided to do a little experiment in the height of the “blue check” hysteria. Twitter had shifted from verifying accounts based (more or less) on merit or importance and instead would let users pay for a blue checkmark. That obviously went (and still goes) badly. Meanwhile, Meta opened its own verification service earlier in the year, called Meta Verified.

Mostly aimed at “creators,” Meta Verified costs $15 a month and helps you “establish your account authenticity and help[s] your community know it’s the real us with a verified badge." It also gives you “proactive account protection” to help fight impersonation by (in part) requiring you to use two-factor authentication. You’ll also get direct account support “from a real person,” and exclusive features like stickers and stars.

Read more
Here’s how to delete your YouTube account on any device
How to delete your YouTube account

Wanting to get out of the YouTube business? If you want to delete your YouTube account, all you need to do is go to your YouTube Studio page, go to the Advanced Settings, and follow the section that will guide you to permanently delete your account. If you need help with these steps, or want to do so on a platform that isn't your computer, you can follow the steps below.

Note that the following steps will delete your YouTube channel, not your associated Google account.

Read more