UK referendum puts pollsters to test

In a sandstone office building in suburban Edinburgh, 400 people are at work compiling research that can shift billions of pounds in the currency markets and may reveal Britain’s political future.

This is the nerve centre where Ipsos Mori, the world’s third-largest pollster, carries out telephone surveys for business, the public sector and media clients wanting to gauge the result of Britain’s June 23 referendum on EU membership.

Such pollsters’ work has become more influential than ever ahead of the vote. Recent polls showing the Leave campaign ahead sent the pound plunging, while hedge funds and investment banks are commissioning private exit polls to profit from early information about the result.

And yet many polling groups are battling to save their reputations after a string of well-publicised failures last year and in the face of big challenges in this month’s vote.

“We don’t know whether people behave differently in a referendum,” acknowledges Gideon Skinner, Ipsos Mori’s head of political research. “We have a lot less data to work from.”

Investors are basing their decisions on figures generated by untested methodologies still being tweaked with less than two weeks to go.

John Curtice, a polling expert at the University of Strathclyde, warns that almost all of those changes favour the Remain campaign. “Is there a risk they’re chasing what they think the figures should be?” he asks.

“They all individually have very good reasons for making these changes but it is intriguing that they are all in the same direction.”

The pollsters have been here before. After they failed to forecast the Conservative absolute majority in the 2015 election they were widely suspected of “herding” — tweaking findings to minimise differences with other surveys. An industry inquiry into their performance was unable to rule out such a possibility.

But there is also a big divergence between online pollsters, which tend to show a finely poised campaign, and those who use phone calls to survey the general public — and who generally show the Remain campaign ahead.

Matt Singh, a polling analyst who correctly called the general election result, has argued that phone polls are more likely to get the referendum result right.

He is not alone: the betting markets are considerably more influenced by phone polls than their online counterparts, according to Mike Smithson, a betting expert.

“If you take part in a phone poll you get a call out of the blue at home or on your mobile and theoretically that is much more random than someone choosing to sign up to an internet panel, where you usually get paid for doing so,” says Mr Smithson.

Ipsos Mori’s Edinburgh call-centre is largely staffed by students on flexible contracts who randomly dial landline and mobile numbers, reading from a prepared script.

The betting markets’ reliance on phone polls has a pedigree: during the final weeks of the general election, 56 per cent of the phone polls showed a Tory lead while only 10 per cent of online polls did, according to Mr Smithson.

This aggregate picture is what investors and the media should look at, Mr Curtice argues. While individual polls can err, the overall weight of evidence is generally a good guide to what to expect, he says. “The exceptional poll is the one that gets the most publicity and triggers an overreaction by the markets but that is the one that is most likely to be wrong.”

————————-
UK’s EU Referendum: How people would vote

For a more detailed summary of opinion polling visit the FT’s Brexit poll tracker page
————————-

The main challenge for political pollsters is the very limited timescale and expense their media clients can tolerate, according to Mark Diffley, Scotland research director for Ipsos Mori.

Most political polls, commissioned by newspapers and broadcasters, involve just a few days’ fieldwork. By contrast, research for private companies and governments can stretch over months and use far more statistically reliable methodologies.

“It’s not as though we don’t know how to get the best results,” says Mr Diffley. “A great deal of the work we do is very high-quality and costs a lot more than a media client will pay. The issue is that we don’t have the luxury of that time and expense when we are two or three weeks out from a big political event.”

Although political polls only constitute a small proportion of most polling companies’ work, their high profile makes them a useful marketing tool — as long as they get it right.

Given the poor performance last year, their performance in the referendum campaign is particularly important, Mr Curtice says. “Their business model involves taking an enormous risk for the publicity [that political polls bring]. There are lots of things hanging on this referendum and one of them is the future of the polling industry.”

————————-
UK’s EU referendum: full coverage and analysis

View the FT’s comprehensive guide to the vote on whether Britain should stay in Europe, with all the latest news, analysis and commentary from both sides of the debate. See more 
————————-