Google's Made an API for Identifying Troll Comments

By Holly Brockwell on at

Google's Counter Abuse Technology team and Jigsaw (part of Alphabet) are working on a way to identify and weed out angry manchildren in website comment sections.

Called Perspective, the tool uses machine learning to analyse comments and compare their similarity to ones considered 'toxic'. Each comment gets a score, and publishers and moderators can use this to filter out the ones that are damaging to the conversation. As Google puts it:

Discussing things you care about can be difficult. The threat of abuse and harassment online means that many people stop expressing themselves and give up on seeking different opinions.

Troll comments are often seen as a 'snowflake' problem, an indication that the people being targeted just need to grow a thicker skin. This is almost always said by someone who has never experienced having all their communication channels deluged at the same time with hundreds of insults, threats, horrifying images and disturbing content. The fact is, certain people and certain topics get a disproportionate amount of hate, and that can end up silencing important voices as they decide they can live without the constant threats and abuse. Google and Jigsaw clearly see this as a problem - because it is.

A recent Guardian study showed that of their ten most-trolled writers, eight were women and the only two men were black. Considering how up in arms trolls get about "free speech," it's ironic that their hateful comments end up ensuring that the only people who can speak freely are the trolls themselves. Jigsaw President Jared Cohen puts it thusly:

Imagine trying to have a conversation with your friends about the news you read this morning, but every time you said something, someone shouted in your face, called you a nasty name or accused you of some awful crime. You’d probably leave the conversation.

Google used that measure - leaving a conversation - as the yardstick for the comments deemed 'toxic' in their model. A 'toxic' comment is therefore defined as "a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion."

Perspective isn't quite ready yet - you can request an API key but they're "rolling out through 2017," so it might take a while. It sounds like there'll be more machine learning models available from the Google/Jigsaw collaboration later in the year, too.

In the meantime, the Perspective website lets you scroll through comments and filter them by toxicity. Here are some on Brexit:

It's worth noting that while these score highly by Google's model, they are absolutely nowhere near the level of nastiness comments reach on the internet. But presumably Google didn't want to host those on its website. Which is kind of the point.

If you suddenly have the urge to leave a nasty comment, you can have a go at seeing how it scores for toxicity here. Not in our comment section. Cheers. [Wersm]