Finally we have some new polls! Two phone, two online, including a parallel test from ICM. Interestingly, the online polls showed a move towards Leave, the phone polls showed a move towards Remain. We should be getting at least one more phone poll this week.
— NCP EU Referendum (@NCPoliticsEU) May 16, 2016
— NCP EU Referendum (@NCPoliticsEU) May 17, 2016
The ORB figures that I tweeted showed the “all voter” numbers, rather than the headline “certain voters” figure, which shows a much smaller shift. And the TNS poll showed a big swing towards Leave compared with the previous TNS survey, driven mainly by huge swing among 2015 non-voters that said they would vote this time. Since samples usually have too few non-voters and pollsters have to upweight them, this type of swing is amplified:
Even so, it looks like the phone-online wedge might be widening again. Quite a few observers have suggested that, by fair means or foul, the two modes ought to converge further. But ICM’s Martin Boon thinks otherwise.
What hard evidence do we have in terms of which (if either) is right? To repeat what I’ve said many times, I don’t think that either method is inherently “better” than the other – each has its strengths and weaknesses. The “real votes” evidence seems to favour phone polls at the moment – note the overestimation of UKIP‘s vote share at the midterm elections (and by more than online polls did at the general election).
And today we got something else – from another country, which makes it less useful than the above, but the similarities are uncanny. Nate Cohn at the New York Times has done some analysis of the accuracy of polling on Donald Trump. As many readers will know, Trump has tended to do quite a bit better in online polls than those with what our American friends call “live interviewers”, which has been attributed by some to interviewer effects. This is relevant because Trump’s support has similarities with support populist parties in other countries and part (though by no means all) of the Leave based in the UK. Cohn’s modelled national equivalent vote shares (based on actual state-by-state results) is closer to the phone polls than online:
He then compares general election polls where, again, Trump underperforms Hillary Clinton by a bigger margin on the phone than online. Just as James Kanagasooriam and I found in Polls apart, the difference ways that the two modes deal with undecided voters has a substantial effect on the number saying they’ll vote but don’t know which way. As is very often the case in elections (as opposed to referendums) they seem to break relatively evenly between the two candidates. That’s consistent with the discrepancy being sample-related, though relatively small at about 2.5 points. Again, this is what you’d expect when a lot of people are voting along party lines.
Further analysis, this time on turnout, came from Anthony Wells, who reminded everyone that turnout isn’t just about age – it’s also important to consider socioeconomic status. So while the focus has been very much (too much) on the greater likelihood of (Leave-leaning) older people to vote, it’s important to remember that (Remain-leaning) middle-class voters are also more likely to turn out.
What about the NCP forecast? This is the first update for a week, as there simply haven’t been any new polls in that time. Now we have four news ones, but the net of all of them is a swing towards Leave. There are quite a few caveats to this – because there haven’t been many polls recently, newer polls will have a disproportionate impact, and sharp moves always have some impact even though the model mitigates them. When those two things happen together, relatively big moves can happen. And of course, we have more polls coming up in the next few days, but for the time being this is how we stand:
So this move should be viewed with caution, but for now the probability of Brexit is back up to 24%. And partly as a result, the gap between the forecast and then betting markets has dropped below 4 points, with bookmakers having reported heavy betting on Remain for the last month.