Making sense of recent polls and the divergence between phone and online polling

Given the amount of movement in some opinion polls recently, you’d be forgiven for thinking they’re all over the place. Complicating the picture are the run-up to the election, technical factors such as methodology changes by some houses, and the fact that this is a particularly difficult environment in which to poll, given the large and complex flows of voters between parties.

Differences between pollsters can be random, or due to timing, but can also be systematic (see Anthony Wells’ analysis of house effects). An important difference is phone versus online polling – phone polls are an attempt to approximate a random sample, but are limited by landline coverage, poor response rates and “shy” responses. Online polls are statistical models based on survey panel responses. They avoid some of the pitfalls of phone polling, but as they aren’t random, they will be biased if the model is inadequate. Phone polls typically weight by recalled past vote, online polls by party ID, the targets for which are derived from past voting. Online polls are cheaper to conduct and are therefore more frequent and with larger sample sizes, an important consideration when assessing variability.

One area where the two methods were producing substantially different results a few months ago was in the combined vote share of the two largest parties. In the chart below, the blue lines (right-hand scale) show a consistent share of about 65% in online polling, whereas phone polls had shown a combined share 4-5 points lower than online polls in at the end of 2014, but were two points higher this month. The rise was was fairly constant across the individual pollsters.

Online and phone polls both converge and diverge

The single most important number is the difference in popular vote shares of Labour and the Conservatives (or equivalently, the swing, which is half of the change in the spread since the last election). On the chart the spread is shown in red, on the left-hand axis. The Conservatives are consistently performing better relative to Labour in phone polls than online polls, taking a one-point lead with the former. This may be exaggerated by ICM’s suspected outlier last week, but even among the other phone pollsters, Labour’s lead has gone. It’s still there in online polls, but yesterday’s rather roguish-looking six-point Labour lead from Survation is responsible for the widening – the other online pollsters show a similar picture from last month. Thus the widening between the two red lines might not be real.

To sum up, the race for the lead is obscured by some very noisy numbers, amplified by the tendency of partisans to spin favourable polls for all they’re worth (and then some). But in relative terms, phone polls continue to look a bit better for the Conservatives, online a bit better for Labour, but the underlying picture has continued to move the Tories’ way, such that the race is now literally all-square:

As the larger parties have risen, the smaller parties (in aggregate) have fallen, but far from uniformly. UKIP, to which I’ll return in a moment, have dropped with all but one of the pollsters.

Phone pollsters had shown higher levels of Lib Dem support than their online counterparts at the end of last year, as I wrote at the time. This was largely for technical reasons – A disproportionate amount of their 2010 voters had been saying “don’t know” for much of this parliament, which meant they were excluded in most online polls, but reallocated (albeit at a reduced rate) in many phone polls. But as these undecided voters have made their minds up, the gap has disappeared. The bad news for the Lib Dems is that they’ve mostly decided to vote Green, which explains the symmetry of the two parties’ phone poll lines below:

The picture is just as mixed with the smaller parties

The Greens are probably the hardest party to do polling on, simply because their support is as much as eight times larger than at the last election, making past-vote weighting a rather blunt tool. As such, sample quality becomes extremely important – the above chart illustrates consistently higher Green support in phone polls, but while phone pollsters are fairly consistent (all between 7% and 8% in February) the online polls this month have ranged from 4% with Survation and ComRes to 7% with Opinium and YouGov. Overall the pattern is one of continued Green growth at the Lib Dems’ expense, but at a slower pace than in January.

UKIP divides pollsters, too

Turning specifically to UKIP, the differences between pollsters have remained very wide, but all of them have shown a decline in support for the party, with the exception of Populus, whose methodology changes have reduced the combined share of Labour and the Tories and increased that of UKIP. Two of the biggest drops have been from ICM and Ipsos-MORI, who were already reporting the lowest UKIP VI among phone pollsters and now show the party in single-figures. Survation, which has found by far the strongest UKIP support of any pollster, has also shown a substantial decline, though their latest poll was the highest of the month once again. The rest of the pack have converged, however, in the mid-teens.

Why is UKIP support falling? Some have pointed to polling showing record support for staying in the EU. But given that Europe seems to be a secondary concern to UKIP voters, it seems more likely that a first-past-the-post squeeze is taking place. Who benefits is less clear – ICM and Opinium showed swings from UKIP to the Conservatives, while Ashcroft, Survation and Ipsos-MORI showed swings from UKIP to Labour.

About The Author

Related Posts