Skip to main content

AI learns when people are using hate speech, even when they use code words

hatespeech codeword algorithm
rawpixel/123RF
Anyone who has ever had a safe-for-work website blocked as not safe for work (NSFW) by their work internet filter (or has experienced the potentially embarrassing opposite) knows that programs designed to block out certain pieces of content can often run into problems.

One reason for this is that keyword searches can prove to be overly blunt tools for dealing with something as nuanced, complex, and constantly evolving as language.

This is particularly true when trying to find hateful keywords on social media. For example, last year Alphabet released an algorithm designed to filter out racist words online, only for trolls to start substituting the name of Google products for racial slurs. For a while, it totally outsmarted the software.

A new algorithm, developed by researchers at the University of Rochester, thinks it’s cracked the problem, however. Analyzing Twitter feeds, it can distinguish between phrases like “gas the Skypes” (a substitute for “Jews”) and “I hate Skype” (which hopefully just means Skype) with an impressive 80-percent accuracy.

“We have developed an intelligent data analytics algorithm to track the constantly evolving hate codes which are designed to evade detection,” Professor Jiebo Luo, co-author of the paper, told Digital Trends. “We start with a set of known hate codes, retrieve hate messages containing these codes, [and] build a language model using machine learning techniques to recognize hate messages. On the basis of that, we do two things: [firstly], using the language model to detect ongoing hate messages that may contain new hate codes, and [secondly] using detected hate messages to identify hate spreaders whose new messages are used to discover new hate codes.”

The smart insight is working out which words correlate with others, thereby discovering when a stand-in word is being used for something else by adding context. Sure, these can be changed as well, but there’s only so many words a troll can change before rendering their original statement totally unintelligible.

In all, it’s a very smart use of machine learning. Yes, the boundaries about what it’s OK to say online are still being drawn — and are best worked out by private individuals and companies, not algorithms. But when it comes to stopping people from being confronted with hateful rhetoric online, tools like this go way beyond simple keyword searches.

Next up for the project? “We hope to get more data to make our model more robust and accurate,” Luo continued. “Ultimately, we hope the leading social media platforms such as Twitter, Facebook and so on can adopt our technology, which is described in this paper, and likely will be further developed for higher accuracy to clean up social media. It is our ongoing effort to use data science for social good.”

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Machine learning? Neural networks? Here’s your guide to the many flavors of A.I.
Neural networks? Machine learning? Here's your secret decoder for A.I. buzzwords
machine learning

A.I. is everywhere at the moment, and it’s responsible for everything from the virtual assistants on our smartphones to the self-driving cars soon to be filling our roads to the cutting-edge image recognition systems reported on by yours truly.

Unless you've been living under a rock for the past decade, there's good a chance you've heard of it before -- and probably even used it. Right now, artificial intelligence is to Silicon Valley what One Direction is to 13-year-old girls: an omnipresent source of obsession to throw all your cash at, while daydreaming about getting married whenever Harry Styles is finally ready to settle down. (Okay, so we’re still working on the analogy!)

Read more
Current tech for detecting hate speech is woefully inadequate, researchers find
anti hate speech tech finland gettyimages 113717356

Exactly what constitutes hate speech is one of the most hotly contested topics in 2018. One of the reasons it is so vigorously debated is because of how difficult it can be to define. If humans find hate speech difficult to define, machines find it even more more of a chore, as a new survey of seven different computer systems intended to identify such online speech makes clear. It also touches on just how easy they are to circumvent.

Researchers from Finland’s Aalto University analyzed various anti-hate speech systems, including tools built by Google’s Counter Abuse team. Their findings? Not only can the systems used to flag offensive content online not agree on a solid definition for hate speech, they can also be easily fooled with little more than a typo or letter substitution.

Read more
Social Feed: Facebook asks for hate speech help; YouTuber in jail for fake news
how to backup an iPhone without iTunes

Social media is a fluid technology — nearly every day, the major networks are announcing a big change, coming under fire for the latest controversy or moving forward in smaller ways. Social Feed is a collection of those smaller changes that you may have missed with this week’s biggest news -- like Facebook’s long list of F8 announcements, Twitter’s news update, Snapchat’s slowed growth, group video calls for What’s App and Instagram, and Instagram’s payments test. Find Social Feed every weekend for the latest social news tidbits.
Just how long did Facebook users watch that video? New metrics rolling out for creators
Creatives on Facebook are gaining enhanced tools to see just how long their video held viewer’s attention. On Thursday, May 3, Facebook shared additional metrics added to the existing video retention data inside Pages. The enhanced tool now includes a comparison between users that follow the Page and users that don’t, along with adding gender demographics. The update will also allow for a more detailed look at the charts with a zoom option. Facebook also said they fixed a bug that caused some inaccurate data for videos past two minutes.
YouTuber gets a month in prison for fake news
Malaysia has a new law against fake news -- and now the first person prosecuted under that law is incarcerated, according to The Guardian. YouTuber Salah Salem Saleh Sulaiman pled guilty after posting a video misrepresenting police response time to a death in Malaysia’s capital -- claiming 50 minutes rather than the eight police reported. The YouTuber apologized during the hearing. The judge ordered him to pay a fine of 10,000 ringgit (about $2,537), but the Guardian said he opted for a month in jail “because he could not pay.”
Facebook test asks users for help spotting hate speech

i mean this is hilarious pic.twitter.com/GDtUsrrlzU
— Gene Park (@GenePark) May 1, 2018

Read more