New NCP analysis – where the polls went wrong

New analysis published today by Number Cruncher Politics finds that the polls were wrong at May’s general election because the flows of voters between parties, following the Lib Dem collapse and the UKIP surge, were substantially different from what voting intention polling had measured.
By modelling electoral data and using the cross sectional (face-to-face) wave of the British Election Study (published on Friday), NCP is now able to explain what happened.

  • Polls had almost without exception shown that 2010 Lib Dem voters that had switched to one of the two larger parties had overwhelmingly opted for Labour. However, the analysis suggests a different picture – Lib Dem to Labour switchers did outnumber Lib Dem to Conservative switchers, but only narrowly.
  • Polls also consistently underestimated the UKIP threat to Labour. On average, they got the UKIP vote share right, but badly mismeasured where it came from. Almost all had shown proportions of 2010 Tories to 2010 Labour voters to be two- or three-to-one – but the new data suggests that the breakdown was significantly more even.
  • It is likely that over the last two election cycles, UKIP has taken more voters from Labour than from the Conservatives (or any other party).
  • Polls had also suggested that the direct flows between Labour and the Conservatives were very small and essentially cancelled each other out. There was in fact a very small net flow of 2010 Labour voters to the Tories.
Where the polls were wrong in one chart 2010 to 2015 (L-R) voter flows, double other/non-voters excluded. Data sources: NCP using BES waves 6 and cross-section

Click to enlarge

  • These errors were reflected elsewhere in polls besides voting intention. 2015 Tory voters appear to be much more socially liberal – with more progressive views on gender and racial equality – than was previously thought.
  • The new BES data also suggests that the Tories had both a larger vote share and a larger lead among women than among men.
  • These numbers support the theory that Conservative modernisation helped – not hurt – the Tory performance in 2015, as Daniel Finkelstein argues in today’s Times, citing the new NCP analysis.
  • Methodologically, the primary root of the problem is very likely to have been a sample imbalance combined with problems weighting the data.
  • It is widely noted that opinion poll respondents tend to be more politically engaged than voters as a whole. However, this bias also arises unevenly within opinion poll respondents. Further analysis of the sample and its imbalance has identified a significantly undersampled section of the electorate, which was Conservative-learning in both 2010 and 2015.
  • Those that had voted for one of the coalition parties in 2010 and subsequently changed their vote to an opposition party were overrepresented in polls. The risks from this “electoral flux” were highlighted in NCP’s pre-election analysis on May 6th, which predicted the polling failure and Conservative victory.
  • NCP has under development a model based on this new analysis, to attempt to solve the underlying problems. The adjustments from the initial version of the model correct around 4 points of the error (or around 5 points taking into account differential likelihood to vote). The average of published polls at the election understated the Conservative lead by 6.5 points.

See the full analysis.
Notes: Number Cruncher Politics (NCP) is the non-partisan UK polling and psephology site that predicted the 2015 opinion polling disaster. For more information please see the ‘about’ page or connect via social media. Matt Singh is a polling analyst and NCP founder. He is based in London and tweets at @MattSingh_.

About The Author

Related Posts