Google has a new plan to fight internet trolls, and it starts and ends with AI

By Lulu Chang Published September 20, 2016

It’s being used to fight ISIS, and now, an app developed by a subsidiary of Google is tackling another kind of vitriol — online trolls.

Jigsaw, an organization that once existed as Google’s think tank, has now taken on a new life of its own and has been tasked with using technology to address a range of geopolitical issues. The latest software to come out of the group is an artificial intelligence tool known as Conversation AI. As Wired reports, “the software is designed to use machine learning to automatically spot the language of abuse and harassment — with, Jigsaw engineers say, an accuracy far better than any keyword filter and far faster than any team of human moderators.

Recommended Videos

Conversation AI learns and automatically flags problematic language, and assigns it an “attack score” that ranges from 0 to 100. A score of 0 suggests that the language in question is not at all abusive, whereas a score of 100 suggests that it is extremely harmful.

And it looks like it’s working. As Wired notes, “Jigsaw has now trained Conversation AI to spot toxic language with impressive accuracy. Feed a string of text into its Wikipedia harassment-detection engine and it can, with what Google describes as more than 92 percent certainty and a 10-percent false-positive rate, come up with a judgment that matches a human test panel as to whether that line represents an attack.”

Currently, the plan is to test Conversation AI first in the New York Times’ comments section (though perhaps YouTube would be a better place to start), and Wikipedia also plans on making use of the software, though it’s unclear how.

“I want to use the best technology we have at our disposal to begin to take on trolling and other nefarious tactics that give hostile voices disproportionate weight,” Jigsaw founder and president Jared Cohen told Wired, “to do everything we can to level the playing field.”

Eventually, Conversation AI will become open source so that any site can make use of its anti-trolling capabilities to protect its users. So advanced is the technology already that it can “automatically flag insults, scold harassers, or even auto-delete toxic language.”

So look out, internet trolls of the world. It looks like your days of abuse may be numbered.

Former Contributor

Fascinated by the effects of technology on human interaction, Lulu believes that if her parents can use your new app…

Topics

Emerging Tech

A new startup wants to fix the military’s working-dog shortage with cloning and gene editing

Promising pitch, plenty of unanswered questions about the dogs.

a new startup wants to fix the militarys working dog shortage with cloning and gene editing warhound featured image

Military working dogs charge into places the rest of us would flee. So, I understand why anyone would want to breed tougher, healthier dogs for the job.

Warhound, a new startup from Colossal Biosciences co-founder Ben Lamm and Anduril co-founder Palmer Luckey, has stepped out of stealth with $200 million to clone and gene-edit working dogs for American troops, police departments, and emergency services (via TechCrunch).

Emerging Tech

AI can describe a picture, but it still struggles to understand blind users

A University of Maryland study finds that Be My AI often buries useful answers in lengthy descriptions, leaving blind users to work harder to get the information they need.

Blind Man using walking stick

AI can describe a picture, identify objects, and answer questions about what a camera sees. But when a blind user needs a quick answer, getting an entire spoken essay about the scene isn't exactly helpful. Researchers at the University of Maryland (UMD) have found that AI-powered visual assistance can leave blind users doing too much work to get the information they actually need.

The study, titled Say It My Way: Exploring Control in Conversational Visual Question Answering with Blind Users, examined how people interact with Be My AI, an AI-powered feature of the Be My Eyes app. The researchers found that users often received lengthy descriptions that didn't adequately address their immediate goals, forcing them to ask follow-up questions or find ways to steer the AI towards a more useful answer.

Emerging Tech

Etsy cracks down on AI slop listings. Everyone else needs to take note.

Now every platform must decide what counts as handmade.

Etsy featured image

Platforms tend to shrug off AI clutter until it threatens what they sell. For Etsy, that thing is trust in the maker.

To protect that, Etsy has banned AI-generated how-to guides and listings made entirely by bots. Most sellers must show at least one unretouched photo of the item (via Etsy Seller Handbook).