To create the YouGov election model we conducted more than 300,000 interviews, and then got two great data science brains — Ben Lauderdale of the LSE and Doug Rivers of Stanford — to apply our results to the Office for National Statistics map of Britain.
For two weeks our model was describing one reality, and everyone else — technical experts, gurus, field operatives, media pundits — were describing a totally different reality. We pushed and pulled the data in every direction to find the flaw, but we couldn’t find one. All the while we were being attacked from every direction.
The Conservative campaign strategist Jim Messina tweeted: “Spent the day laughing at yet another stupid poll from @yougov.” He mistakenly used the word “poll” because that’s the simplistic methodology we’ve all got by with for decades. You get a sample of 1,000, for example by random phone dialling, ask a few questions, apply some weights to correct any demographic skews, apply some guesstimates on turnout, and that’s it, you’ve got yourself a poll. In simpler times it worked quite well. But now the skews are too serious. For example those who are more educated and more engaged tend to be more willing to respond to pollsters. The telephone polls greatly over-represented them in the Brexit referendum and painted a picture of an EU-friendly nation.
Internet polls drawing more representative samples make the otherwise traditional polling method more robust, which is why over the years YouGov has managed to be more consistently accurate than most of the industry. But even with this advantage, we got the 2015 election wrong.
So we turned to a new methodology based on the most advanced data science called MRP (multilevel regression with post-stratification).
There were hoots of derision across Twitter and the airwaves. On Thursday, for the first time in history, Canterbury voted Labour.
We will continue to run polls but when it comes to complex and volatile situations where detail really matters, we will use MRP. Stephan Shakespeare is chief executive and founder of YouGov Essentially, we look at everything we know about the people who live in a constituency using a variety of sources including the ONS, and we match people in our panel with the types of people in that constituency. Imagine 1 per cent of Canterbury residents are female graduates in their 30s on lower quartile income. We would look at the voting intention of all such people in our panel, apply shifts based on what we know about regional differences, then add further tweaks based on what we’ve learnt from Cantabrians in our panel. Our model showed the constituency leaning to Labour.