FAQ: The Number Cruncher Politics polling average

See also the main NCP EU referendum page.

What is the Number Cruncher Politics polling average?

The polling average is an aggregation of published opinion polls for the UK’s EU membership referendum. Each poll is adjusted for the house effect of the polling company, that is, the fact that polls produced with different methodologies often have systematically different results. Because these differences are usually predictable, they are taken into account so that the average doesn’t jump around depending on what type of poll was last published.

Why does the polling average show slightly different numbers to polls of polls on other sites?

Most polls of polls take a simple average of the last few surveys. This is just about OK for general election polling, but very unstable in a situation like this, when different pollsters are getting such different numbers. Simple averages also put disproportionate weight on online polls, because there are more online pollsters than telephone pollsters and they tend to poll more frequently.

The NCP average follows a more scientific approach. It adjusts each poll based on how that company’s polls have tended to deviate from the average (the house effect). It also puts more weight on more recent data, with equal weight given to telephone and online polls.

A consequence of giving equal weighting to online and phone polls is that the average is more in favour of Remain than other polling averages. However, not doing this would implicitly assume that the true levels of support for each side are closer to what online online polls are saying, simply because they are conducted more often. This doesn’t make much sense.

Where pollsters use more than mode, polls in each mode are considered separately, although polls with the same methodology but for different clients are not.

Is this your forecast of the result?

No. As Lord Ashcroft would say, it is a snapshot not a forecast. The NCP forecast is published separately. The polling average is simply a scientific method of aggregating what the polls say. It does not adjusted for expected error or predict how public opinion is likely to evolve as the referendum approaches.

Are the polls right?

This is the inevitable question following the disaster at last year’s general election. Many changes have been made but underlying challenges remain. Given the particular difficulties involved in polling a one-off event like a referendum, it wouldn’t be surprising if the polls are a bit wide of the mark.

The average isn’t adjusted for expected polling error, but the forecast is. See Polls apart for more on this.

Doesn’t it all come down to turnout?

Turnout clearly matters, but it’s not the only thing that matters. In fact turnout isn’t particularly well understood in the context of the EU referendum. Both sides have high- and low probability voters – Leave has more support among older people, Remain has more support among more affluent people and those with higher levels of education. Analysis of polling and election data suggests that these two demographic drivers broadly cancel each other out.

Does the average weight all pollsters equally?

In addition to the equal weighting of phone and online polls, pollsters are weighted by their historical performance and other criteria, and more recent data is given greater weight.

I am technical – what are the details?

NCP may publish a detailed paper at some point. Or not. To summarise, the house effects are calculated using a weighted dispersion from a LOESS regression of regular polls and then normalised using house weights. Each poll is then adjusted for its house effect, subject to a proportional buffer. The adjusted polls are then combined into a linearly-weighted moving average.

See also the main NCP EU referendum page.





About The Author

Related Posts