The investigation into the polling failure at May’s general election isn’t yet complete, but that hasn’t stopped a fresh controversy breaking out in the meantime. Today’s Times reports that Vote Leave has written to the BPC with regard to a 2013 survey on the EU by YouGov for the CBI.
At this point, the letter doesn’t appear to be in the public domain, so I can only go on what as been reported. However, given that Vote Leave has tweeted the Times article, it seems reasonable to assume they consider it accurate.
There are a couple of issues here, one relating to disclosure and one relating to methodology. Per the Times:
“Vote Leave also alleges that the CBI and YouGov have breached best practice by failing to disclose information about the poll on their websites”
This is not the case – the YouGov tables are online here and the CBI press release (which links to the tables) is here. And although both were published several weeks after the fieldwork was ended, this is allowed and happens from time to time – BPC rule 2.6 stipulates publication of tables 2 working days from when the data was published, rather than when the survey was fielded.
The complaint asks the BPC to investigate “serious violations” of the council’s rules by YouGov. Rule 1.2 requires that BPC members “aim to use sampling methods and/or weighting procedures designed to broadly represent the opinions of all people in designated groups.” This is important because the designated group (stated at the top of the tables and in the first sentance of the CBI’s press release) is CBI members, not British business more broadly. So whether or not the poll was “wholly unrepresentative” of the latter isn’t a polling issue – the poll was of the former.
“The CBI is understood to have selected the sample for YouGov from its membership list. In total, 451 of the members selected responded”
Normally, YouGov would use its extensive in-house panel for polling. But for non-standard research, client-provided lists are sometimes used – there is nothing in the rules to prevent them from doing so, although if this is how the sampling was done then YouGov might have been clearer about it.
Then the crucial part:
“These facts are capable of giving rise to the inference that the CBI was allowed to select which of its members were surveyed in order to further the CBI leadership’s longstanding pro-EU stance”
In other words, there doesn’t appear to be any evidence of wrongdoing, just a potential inference.
Amusingly, BPC secretary Nick Moon, who is also MD of exit pollster GfK NOP, appears to have hit “reply to all” on the following email:
Don't think the the British Polling Council meant to hit reply all following complaint about the YouGov/CBI poll pic.twitter.com/CTf76oeram
— Richard Fletcher (@fletcherr) November 2, 2015
Looking “pretty dodgy” sounds bad, but note that it’s “at first glance” and – critically – he says “we don’t need to rule on that”. In other words, no suspicion that YouGov deliberately conducted an unpresentative poll.
Update (17:50): The British Polling Council has published a response, including an apology for the language used in its email. Separately, YouGov CEO Stephan Shakespeare has issued the following statement: