Art-inspired face blurring can obscure identity without losing humanity

If you want to know how someone feels, there are only a few cues to rely on. You can study their facial expressions, consider the content of what they say, and tune in to the tone of their voice. But that can pose a challenge for video journalists and documentary filmmakers covering sensitive subjects, because the easiest ways to anonymize a source is to scrub out things that make them relatable by pixelating their face or distorting their voice. Their story remains the same but the characters themselves can appear crude.

Steve DiPaola, a computer-based cognitive scientist at Simon Fraser University, thinks there’s a better way. He thinks anonymity can be both beautiful and true to the emotional aspects of the people whose identity it’s concealing.

To that end, DiPaola and his colleagues have developed an A.I.-generated anonymity system that “paints” over video frames, using inspiration from masters like Picasso and Van Gogh to reimagine a person’s appearance. The goal is to minimize outer resemblance but maintain fidelity to a subject’s inner character, allowing their facial expressions and vocal inflections to shine through. If deployed by journalists, the system could support more intimate and relatable stories, particularly in virtual reality, where the power of empathy has proven particularly strong.

With the rise of VR in journalism, the need for more nuanced and affective ways to represent anonymous sources is key.

The project began for DiPaola as a way to make an A.I. system that was capable of creating art on it own. A number of algorithms later, he and his team focused their efforts on a fine art painting, and then, more specifically, one that could paint portraits. But after a small grant from Google News and the Knight Foundation, DiPaola — along with Kate Hennessy, a cultural anthropologist at SFU, and Taylor Owen from the University of British Columbia’s journalism school — reworked their system towards providing anonymity for journalists.

The pivot was apt. With the rise of VR in journalism, the need for more nuanced and effective ways to represent anonymous sources is key. Hearing a first-hand account just isn’t the same when the persecuted person’s face is pixelated and their voice is distorted by a few octaves.

Steve DiPaola and Kate Hennessy, SFU

For DiPaola, fine art portraiture offered the perfect guide. Master painters don’t just depict their subject from the outside. They capture an inner essence as well. From decades of study and practice, and techniques handed down through generations, great portrait artists can show a subject’s personality through a series seasoned brush strokes and blended colors. DiPaola aimed to teach the A.I. to look past the surface layer and reveal what subjects are feeling inside.

“You tell so much with your eyes, eyebrows, and facial movements,” DiPaola tells Digital Trends. “Even the way you jerk your head and look down — so much of that was just lost in the pixelation technique.”

The resulting system is both beauty and beast, relying on five Linux computers and a five step process to anonymize a video.

“We tell so much with our eyes, eyebrows, and facial movements.”

To start, the system identifies a subject’s facial features, placing dots around the eyes, mouth, and nose like standard facial recognition systems do. Users can then use a tool to manipulate the features, for example, raising the subject’s forehead, widening her eyes, and lowering her ears. Depending on how significant the changes, this subject may already look unidentifiable.

“Before the A.I. painter even starts painting, steps one and two help change the look of the sitter image,” DiPaola says.

In the third step, the A.I. cuts the face into geometric planes. DiPaola calls this the “Picasso or Cubist approach.”

And in steps four and five — the impressionistic and Van Gogh-like phases — the AI adds inky edge lines and brush strokes.

ai anonymity progression
Steve DiPaolo and Kate Hennessy, SFU

In DiPaola’s vision for the system, a journalist, producer, or even the subject herself could interact with the platform and adjust how refracted the final product is. The system then applies this anonymity to every individual frame in the video.

The researchers haven’t conducted a large-scale study to test how well their system conceals a source, but in pre-study they found subjects were satisfied with the level of anonymity and participants were more engaged when watching videos painted in this style. And the system gained interest from major news outlets like the Washington Post and Frontline, when the researchers presented the work at a conference in July.

“Can you actually have videos of yourself that are more about your inner and less about your outer?”

But anonymizing sources might just be the beginning. DiPaola is interested in becoming something of a digital cupid, working his summer group at SFU’s School of Interactive Arts and Technology to investigate how the system could be adapted for the dating world.

“Dating sites are using videos more often,” he says. “There’s a lot of data that show decisions are made very quickly based on how the person looks, which is too bad at times. Can we actually refine this process so that you’re looking at how somebody is and not just how they look? By anonymizing the footage, attractiveness is not the first thing you think about. Can you actually have videos of yourself that are more about your inner and less about your outer?”

This is, to be sure, a farfetched idea — and one most dating app users would likely approach with caution. But DiPaola’s ambition is compelling, and just a decade ago few would have foreseen the progress that’s been made by A.I. artists. Who’s to say algorithms can’t someday play matchmaker as well?

Correction: A previous version of this article misspelled Steve DiPaola’s last name.

Web

Firefox Reality wants to bring the ‘whimsical web’ to VR

Mozilla launched a VR-powered web browser today called Firefox Reality. But just what does browsing the web in VR feel like? We went hands-on with this new browser to see how Mozilla imagines the future of virtual reality content.
Emerging Tech

Neural networks? Machine learning? Here's your secret decoder for A.I. buzzwords

Don't know your machine learning from your evolutionary algorithms? Worried every time you sit on a bus in case Google's Larry Page sits down next to you with a pop quiz? Our handy A.I. buzzword guide is here to help.
Movies & TV

The best movies on Netflix in September 2018, from 'Spotlight’ to ‘Black Panther’

Save yourself from hours wasted scrolling through Netflix's massive library by checking out our picks for the streamer's best movies available right now, whether you're into explosive action, subdued humor, or anything in between.
Movies & TV

September brings 'The Dragon Prince,' an animated war series, to Netflix

Looking for a new show to binge? Lucky for you, we've curated a list of the best shows on Netflix, whether you're a fan of outlandish anime, dramatic period pieces, or shows that leave you questioning what lies beyond.
Movies & TV

Laugh, cry, or just relax with the best series streaming on HBO

Whether you prefer the twisted world of 'True Detective' or the lovable techies that have quickly come to define 'Silicon Valley,' here are our picks of the best HBO series now available on HBO Now and HBO Go.
Deals

Black Friday 2018: When it happens and where to find the best deals

Black Friday is the biggest shopping holiday of the year, and it will be here before you know it. If you can't wait until November 23 to start formulating a shopping plan, we've got you covered.
Computing

Nvidia GeForce RTX 2080 vs. GTX 1080

Should you upgrade from your GeForce GTX 1080 to the GeForce RTX 2080? Even if you won't take advantage of ray tracing, there are other benefits to the RTX series. We'll examine how each GPU performs to help you decide.
Computing

A vigilante botnet is taking out crypto-jacking malware

A new botnet is on the rise but it isn't being used to take down websites or hack servers, it's going after crypto-jacking malware. When it discovers the malware on a website, it takes it over before destroying them both.
Computing

New monitors from Philips are sleek, color-accurate, cost less than $200

Phillips' new E-Series line of monitors bring thin-bezel, color accurate 1080p IPS LED displays to the market in three different sizes without breaking the bank, while also packing in AMD FreeSync technology for gamers.
Computing

Facebook appears set on crafting custom silicon for augmented reality devices

Facebook's latest job postings are seeking engineers and developers for custom augmented reality chipsets, and seem to support speculation that the company is looking to produce AR glasses.
Computing

Windows handwriting-recognition tool may pose security risk

A Windows file that is designed to help improve the platform's ability to translate your handwritten notes into readable text may be a security concern. One researcher found it contained passwords and email contents.
Computing

Nvidia GeForce RTX 2080 vs. RTX 2080 Ti

Nvidia's RTX 2080 Ti is a beast of a graphics card, packing in RT, Tensor, and CUDA cores to enhance your gaming experience. We'll compare the Ti variant to the standard RTX 2080, and we'll let you know which card to buy.
Computing

We tested Nvidia’s RTX 2080 and 2080 Ti. Are they a worthy upgrade?

We finally have Nvidia's newest graphics cards, the RTX 2080 and 2080 Ti, and put them through our standard suite of benchmarks and game tests to see how they performed in a standard play of Battlefield 1 and Fornite. How do they compare to…
Photography

Not just for photographers anymore, Loupdeck+ now supports Adobe Premiere Pro

Video editors can now get physical with the Loupdeck+ control board. Originally for Lightroom, a software update allows the controls to adapt to video editing, including color grading, clip trimming, and navigating through the timeline.