Skip to main content

Facebook’s revenge porn prevention tool now includes human review

After testing a tool to fight revenge porn in Australia last year, Facebook is expanding the pilot program and is including human reviewers. On Tuesday, May 22, Facebook shared an expanded test of the tool created in conjunction with a number of organizations that allows users to upload their own sensitive photos privately to prevent someone else from uploading the images publically. The change most notably uses a “specially trained” team member to review the report, while Facebook said the initial test used artificial intelligence. While the initial test included Australia and three unannounced countries, Facebook shared that the tool is now also being tested in the U.S, Canada, and the U.K in addition to Australia.

The social media giant is suggesting that those who consider themselves vulnerable to such tactics pre-emptively upload their images to the social network. While it might seem counter-intuitive, such a solution would let Facebook, and by extension, the uploader, get ahead of the problem by creating a hash of the image. A hash, Facebook explains, is like a fingerprint of the image that allows software to keep other copies off the platform without actually permanently keeping a copy of the image on Facebook servers.

Recommended Videos

Revenge pornography has been a growing problem for years, especially on Facebook, which has tools to remove the image only after it’s been shared. Facebook is looking to do something much more proactive to prevent the practice in the future, and hopes that people who have the potential to be affected will trust its A.I.-driven system to combat it.

Please enable Javascript to view this content

The theory behind the new technique is that someone who knows that there are nude or compromising images of them in the hands of someone who may upload those images can block them from being uploaded. By uploading the image to Facebook privately, the social network can “hash” the media, effectively marking duplicates of that image for immediate takedown should someone else attempt to upload them. Developed in conjunction with a number of non-profits focusing on women’s rights and domestic abuse, the service will apply across all Facebook platforms, including Facebook itself, its Messenger application and Instagram.

Users can go through the organization(s) in their country working with Facebook on the tool to receive a form. In the U.S. those organizations are the Cyber Civil Rights Initiative, the National Network to End Domestic Violence. Other organizations include the Australia e-Safety Commissioner, the U.K. Revenge Porn Helpline, and YWCA Canada.

The tool allows someone fearing potential revenge porn to submit a form. Facebook then sends that user a secure upload link for the image or images. Facebook will then create the hash and delete the actual image from the servers within seven days.

While in the initial tests, Facebook said no human actually viewed the photos and that AI created the hash, Facebook now says one trained staff member will review the report. “One of a handful of specifically trained members of our Community Operations Safety Team will review the report and create a unique fingerprint, or hash, that allows us to identify future uploads of the images without keeping copies of them on our servers,” Facebook’s head of global safety, Antigone Davis, wrote in a post.

The question is whether users trust Facebook enough in the wake of Cambridge Analytica and a bug that saved unpublished videos to upload sensitive images. The tool is still only in testing stages. If the scheme proves successful, it may be rolled out the world over.

Updated on May 23: Added information regarding the expanded test.

Jon Martindale
Jon Martindale is a freelance evergreen writer and occasional section coordinator, covering how to guides, best-of lists, and…
PayPal vs. Venmo vs. Cash App vs. Apple Cash: which app should you use?
PayPal, Venmo, Cash App, and Apple Wallet apps on an iPhone.

We’re getting closer every day to an entirely cashless society. While some folks may still carry around a few bucks for emergencies, electronic payments are accepted nearly everywhere, and as mobile wallets expand, even traditional credit and debit cards are starting to fall by the wayside.

That means many of us are past the days of tossing a few bills onto the table to pay our share of a restaurant tab or slipping our pal a couple of bucks to help them out. Now, even those things are more easily doable from our smartphones than our physical wallets.

Read more
How to change margins in Google Docs
Laptop Working from Home

When you create a document in Google Docs, you may need to adjust the space between the edge of the page and the content --- the margins. For instance, many professors have requirements for the margin sizes you must use for college papers.

You can easily change the left, right, top, and bottom margins in Google Docs and have a few different ways to do it.

Read more
What is Microsoft Teams? How to use the collaboration app
A close-up of someone using Microsoft Teams on a laptop for a videoconference.

Online team collaboration is the new norm as companies spread their workforce across the globe. Gone are the days of primarily relying on group emails, as teams can now work together in real time using an instant chat-style interface, no matter where they are.

Using Microsoft Teams affords video conferencing, real-time discussions, document sharing and editing, and more for companies and corporations. It's one of many collaboration tools designed to bring company workers together in an online space. It’s not designed for communicating with family and friends, but for colleagues and clients.

Read more