The missing million goes missing

On Thursday, I spent the afternoon in Craigmillar, a poor area of Edinburgh. I watched a bagpiper lead a crowd of working class Scots from estate to polling booth, flames firing from his pipes. It was a moving yet ramshackle scene. Morale was high. Hope was in the air, along with the smell of petroleum. And yet when the group arrived at the polling station very few of the marchers actually went in to vote.*

Craigmillar. https://t.co/j4tVzvweaV

— John McDermott (@johnpmcdermott) September 18, 2014

Earlier, I spent the morning in Blackhall, a quiet middle class area of north Edinburgh. Voters, many of them elderly, quietly shuffled in and out. Voters.

I won’t stretch the point too far but in this contrast lies illumination for why No won by about 10 percentage points – and why pollsters underestimated the margin.

Consider Glasgow and Dundee: two cities that Yes needed to win – and win big. Their turnouts were 75 and 79 per cent respectively. Compare these turnouts with those in other large areas such as Aberdeenshire and Edinburgh, which registered turnouts of 87 and 84 per cent respectively. Crudely, Glasgow and Dundee have more Yes-type voters than Aberdeenshire and Edinburgh. Yes didn’t need a high turnout; it needed a differentially high turnout. The opposite happened.

Ben Page, chief executive of IpsosMori, suggested as much early on Thursday:

… most experts suspect that if anything “No” could do better than the polls predict. Why? The main reason is differential turnout. Yes supporters tend to be younger and more working class — groups that normally are much less likely to actually vote than older more middle-class people. We [IpsosMori] have 13 per cent of our Yes voters and nine per cent of No voters who say they are absolutely certain to vote but have never voted before. If they are overstating their likelihood of voting we will have underestimated the No vote. While 95 per cent of Scots say they are certain to vote, it will be amazing if this actually happens today.

This is the context for the pollsters’ under-estimation of the No vote by 2 to 4 percentage points. They reported between 51-53 per cent in their final polls; the final total was 55 per cent. They all called the result correctly but they seem to have been caught out slightly by the number and demographics of Scots who voted.

The data will be parsed for weeks but here are three initial additional reflections.

First, as regular readers may remember, I wondered in June whether the polls were over-estimating the Yes tally. (It was just a thought, not a prediction.) This was because all of the polling companies, when preparing their final results, have to convert their raw samples into “representative” samples. In the independence referendum, all the raw samples had a lower number of Yes-leaning voters; younger, poorer, and male, this group is less likely to participate in polls. Therefore the final “weighted” tallies had to be adjusted upwards. What seems to have happened is that too much weight was applied to these Yes-type voters, and to their participation.

Second, the companies that used phone polling, rather than internet panels, achieved marginally more accurate final polls. Experience with the 2010 UK general election suggests that internet polls might be more likely to pick up ephemeral fits of enthusiasm, such as “Cleggmania”, or perhaps, the “surge” in the Yes vote.

Third, remember the undecided voter. The poll numbers were usually reported as excluding “don’t know” voters, which accounted for about 10 per cent on the eve of the poll. The (sparse) data we had suggested that undecideds would split equally. However, historically, referendums have seen last minute swings to the status quo, especially among undecideds. Something had to give. What saved Quebec for Canada may have secured a larger than expected margin for the No side in Scotland.

Pollsters would have been confounded by a Yes, instead they were mildly surprised by differential turnout.

The “shy Nos “were not so shy. The “missing million” went missing.

—-

*I assume they voted earlier. There were three “short walks to freedom” (yup) but it looked to me that the same people were going on every march.