Exclusive: The Most Critically Divisive Films According To Data

By James O Malley on at

Earlier this year a psychological horror film called “Mother!” (yes, with the exclamation mark) hit cinemas. Directed by Darren Aaronofsky and starring Jennifer Lawrence, it is safe to say that the film divided audiences. Some critics raved - The Guardian’s Peter Bradshaw gave the film five stars, whereas others like the New York Observer’s Rex Reed gave the film a big fat zero. Was it genius? Was it hideous? Critics couldn't decide.

Today, Gizmodo UK can reveal that this film didn’t just seem like it had divided audience: We can reveal that it appears to be one of the most divisive films ever, according to our exclusive data analysis.

We can also report that other films that have massively divided critics include other films that you might expect to make such a list: Including Sin City, The Neon Demon, The Passion of the Christ, and The Wolf of Wall Street.

Oh, and Inception for some reason, even though we previously assumed that everyone thought it was really good.

Read on to discover the full list - and how we did it.

Calculating The Most Divisive Films

We were curious to find out which films were the most like Marmite: Which ones get us arguing about whether we love or hate it? But rather than come up with a subjective list, we wanted empirical evidence of divisiveness.

So to find out, we turned to review aggregator Metacritic. The site takes reviews from critics, normalises the scores on a scale of 1-100 (so if the critic scores out of 5, 4 stars becomes 80, 3 stars is 60 and so on - and if they use school-grading-style letters of the alphabet, the site has a way of approximating a score for those too).

We downloaded as much of the Metacritic movies database as we could - 9516 films at the time we scraped, with details over 190,000 reviews by at least named 842 critics. The thinking was that to get a review on Metacritic, a publication has to at least be somewhat decent. And that by focusing on critical reaction, this means that we avoid any "normal" user polling data, which is open to manipulation by coordinated voting and so on.

This also gave us a pool of films to analyse that was of reasonable size: It will definitely have every major release of recent years, and unlike using, say, IMDB’s database, it won’t mix in TV shows, student films, adverts and whatever other detritus IMDB lists alongside films. From glancing through the films we’re analysing, Metacritic’s pool of films is definitely biased towards the post-internet era for obvious reasons - though it does feature some older films (particularly bigger releases and films that people still care about today).

Obviously there could conceivably some issues - Metacritic’s database could have missing films or contain mistakes. Or maybe our data-scraping tool contains flaws (though we’re pretty confident in it). So by and large, we think this is a decent dataset.

And then to work out how controversial each was? We simply applied a common statistical test: Standard Deviation (SD), which provides a “score” of how diverse a given set of scores are. To be clear, this isn’t score about the quality of the film - it's a score showing how much critics agree or disagree. So if a film has reviews all over the place, it gets a high SD - and if critics tend to agree about a film, it will have a low score.

The Most Critically Divisive Films Of Recent Years

So here’s our list of the 50 most divisive films based on critics scores. We’ve limited this to show only films that have received at least 40 reviews on Metacritic so that it reflects the actual critical consensus. If we left in films with fewer reviews, it could skew it: For example, if a film only had two reviews, one scoring 100, and one scoring zero, it would have an enormous SD, despite only being the opinion of two critics.

Yes, Star Wars Episode 3 really is on the list. Presumably there were enough critics still willing it to be good, despite all evidence to the contrary - which will have caused a diverse array of scores. Many of the other entries, as you can see, are films made by auteurs with their own signature styles, so perhaps it is unsurprising that Kill Bill and Birdman made the list.

Interesting too is the other end of the table - films that critics most agreed about. Here’s the 30 films with the lowest standard deviation. What’s most interesting here is perhaps not the prestige films - pretty much every critic thinks that Moonlight and Boyhood are great - but the films that it appears critics broadly agree are mediocre. Yep, Mission Impossible 3 and 22 Jump Street were given Metacritic aggregate scores of 66 and 71 respectively - but according to the standard deviation, clearly no critics thought that they were either utterly brilliant, or or utterly terrible.

Here’s the full list:

So there we have it - now we know the films on which critics agree, and which they disagree. But which do you think is the biggest surprise? Let us know in the comments. We're not expecting there to be much consensus.

James O’Malley is Interim Editor of Gizmodo UK and tweets as @Psythor.