The big tech companies may soon be forced to do more to prevent the sharing of extremist content, if Labour MP Stephen Doughty gets his way. The Cardiff South and Penarth MP has proposed an amendment to the Counter Terrorism and Border Security Bill, which is currently shuffling its way through Parliament. Here's what it says:
Monitoring and removal of unlawful content
(1) Internet search engine providers, video and image sharing platforms, and social media platforms shall—
(a) maintain effective procedures for routinely checking whether material posted on their platforms has been posted by or on behalf of a proscribed organisation, or indirectly to encourage support for a proscribed organisation;
(b) ensure that for the purposes of (a), effective manual and automatic procedures are maintained to check for variants of names, slogans, or imagery which would reasonably identify material as relating to a proscribed organisation;
(c) remove or block such content with six hours of its being identified by internal procedures; and
(d) remove or block such content within 24 hours of receiving an external complaint of its existence on the provider’s platforms.
(2) In subsection (1) the reference to material is a reference to a still or moving image, an audio recording, or textual content (produced by any means).
If MPs vote to include it in the Bill, it'll mean that if a proscribed group - like ISIS, Al-Qaeda, or National Action (a Neo-Nazi group) - posts a video on YouTube or Facebook, or tweets an image advocating Bad Stuff it'll oblige the platform to get rid of it sharpish.
Embedded in this amendment are all sorts of nuances of debate: is it right that the state should be able to force content from specific groups to be removed? How do we even define extremist content? What is a "video and image sharing platform" or a "social media platform"? Could such stringent rules make it hard for newer, less well resourced and funded competitors to emerge and comply with the law?
But this isn't my problem. In fact, I think it all sounds fairly reasonable (at least this is my first draft of an opinion on it).
So what's my beef? My problem is incredibly minor, but I think it speaks to a wider issue.
Doughty is quoted in The Guardian as saying the following about his amendment:
If these companies can remove copyrighted video or music content from companies like Disney within a matter of hours, there is no excuse for them to be failing to do so for extremist material.
Great soundbite, sure, but... there is actually a bit of an excuse. This is a terrible analogy. Sorry, Stephen Doughty.
The problem is that there is actually a massive difference between spotting copyright infringement, and spotting less well-defined extremist material. This is because copyrighted material has a reference point - an image or video from the rights holder. The algorithms looking for pirated content know exactly what they are looking for.
What anti-piracy software does is take what are effectively digital fingerprints of its catalogue, which, say, YouTube can then use to scan uploads. By creating a fingerprint of what Avengers Assemble looks like, it can then spot videos with the same fingerprint and flag them as pirated material.
Imagine if I filmed two videos: one of my TV showing Avengers Assemble, and another of some guinea pigs eating a watermelon. Because the piracy spotting tool would have the Avengers fingerprint as a reference, its systems might deduce that my low quality filming of the TV screen looks 75% similar to the reference fingerprint, whereas the guinea pig video might only be 2%. So the algorithm can make a fairly decent guess as to what is copyrighted material and what is not.
This is very different from spotting more general extremist content, as there isn't going to be a discrete set of reference content to compare against. The amendment calls for "effective manual and automatic procedures are maintained to check for variants of names, slogans, or imagery which would reasonably identify material as relating to a proscribed organisation", but this strikes me as a significantly more difficult task.
Sure, such a system could look for the names of proscribed organisations, or images associated with them (such as swastikas), but unlike the piracy this isn't a 1:1 comparison. With content like this, context is important - what if it is a video about how someone doesn't like ISIS? Or a news report containing ISIS flags? With piracy spotting, context doesn't matter - if you're using someone else's content, then it is piracy and therefore bad.
It should be obvious too that flagging extremist content is also more politically contentious, so moderating decisions on edge-cases will create big challenges for moderators. Sure, only a handful of the most mad people might have a problem with removing instructions on how to make explosives, but you only have to glance at the current culture war and the blurred lines between the alt-right and Nazis-who-don't-bother-to-pretend to understand that such moderating decisions are going to be difficult and messy.
I suppose one mitigation technique could be that tech companies could build up a database of the digital fingerprints of extremist content, so that the same banned videos cannot be uploaded twice, for example. Essentially using the same anti-piracy tech as an automatic anti-extremist filter. This might help a little bit (whisper it, but it might even be a good use for the blockchain). But this won't solve a much bigger problem: there's nothing to stop extremists creating and publishing even more new content.
Given that the barrier to entry with online content is simply "own a mobile phone", moderation is going to be much more difficult. And more importantly to my pedantic point, any moderation is going to be vastly more difficult than monitoring pirated material.
Why does this matter? Tech conversations like this are going to continue to dominate politics as I explained when the Cambridge Analytica scandal broke, my worry is that our politicians are just not up to the job of understanding, let alone passing laws in a world where at least some technical knowledge is required to make informed decisions. This isn't meant as a specific attack on Doughty – as I say, his proposal seems broadly sensible to me – but his choice of words does reflect poorly on whether we think he fully understands what he is legislating on.
So really what Stephen Doughty should have said is “If these companies can remove copyrighted video or music content from companies like Disney within a matter of hours, that is an interesting if ultimately flawed comparison with the challenge of what I went them to do for extremist material”.
Update (15:05): The man himself, who I have perhaps slightly unfairly singled out has responded!
Interesting + glad you broadly support - but to be clear I *do* understand the challenges / difficulties in assessing content particularly if it is new. You say that a “database of digital footprints” of such content could be useful - that’s exactly what GIFCT + others doing.
— Stephen Doughty (@SDoughtyMP) June 27, 2018
(The GIFCT he refers to is the Global Internet Forum to Counter Terrorism, which is a joint project that includes the likes of Facebook, Twitter and Google.)