Why are the polls getting it wrong in so many countries?

With the results of the Greek referendum in, the totals are Yes 38.7 No 61.3, a margin of 22.6 points. As I wrote earlier, the final pre-voting polls had all suggested a close result, with those whose fieldwork ended in the last 48 hours averaging a 0.3 point lead for Yes. That’s a huge miss (of 22.9 points). On the same basis, the combined polling errors on the popular vote spread in the UK general elections of 1970, 1992 and 2015 (the three worst since polling became widespread) amounted to 22.2 points.

Here we go again... No lead over Yes on headline referendum voting intention. Data source: Wikipedia

This seems to be part of a trend: recent polling difficulties have included the US midterms and the elections in Brazil, Israel, Denmark, Finland, Estonia, Poland, Croatia, Romania and of course Britain.

There are some mitigating factors. Referenda, as one-off events, are harder to poll than elections – it’s worth noting that the polls for the Greek election in January were pretty accurate. The current crisis situation in Greece may have been a factor. It’s also possible that the complicated question wording – which, as one UK pollster put it, “would never have got past the Electoral Commission” – may have caused some voters to be particularly unsure, or to make their minds up late. And late swing more broadly – usually viewed as an excuse – was in evidence, albeit on a relatively small scale. “On the day” polls (not to be confused with exit polls, of which there were none) suggested a “No” lead of 3 to 4 points.

But there were specific factors elsewhere too. Britain has form on polling error, as does Brazil, while Israel has a five-day blackout period. It seems there are more fundamental factors at work. People are now harder to reach by traditional methods – telephone pollsters are finding response rates in the single digits in many countries, including the UK. Some demographics are becoming dispropotionately difficult to reach, which can raise the twin problems of unrepresentative subsamples combined with large weighting adjustments. And something that is frequently overloked – turnout is falling almost everywhere. This means that estimating relative likelihood to vote is both more important than ever and more difficult.

In most of these cases the polls have been biased against right-of-centre parties, or in the case of the Irish referendum, the conservative option on the ballot. So it’s reasonable to examine whether there’s a global shy Tory effect at work, although that was quite obviously not the case in Greece (though socio-political factors could combine with a broader polling problem). The puzzling thing about Greece is that TV reporters have had trouble finding people admiting to voting “Yes”, despite predominantly being in situated in Athens, where “Yes” actually did slightly better than nationally. That would be consistent with shy “Yes” voters, yet the polls were 23 points wrong in the opposite direction.

On the question of UK polling, I’ve been busy behind the scenes with the NCP investigation and should be able to post some more relatively soon.

Update (6th July): A couple of readers pointed out that Romania also saw a case of “shy Tory” during their 2014 presidential runoff. I’ve also updated the Greek results to the final tallies.

About The Author

Related Posts