Why looking at one poll won’t give you the full picture
With the election campaign now well underway, everyone will have one eye on the polls. Parties will be closely monitoring them to determine whether or not their announcements have landed with their audiences, and where they might need to change tack to drum up support. But polls also play a big part in the spin machine – with one solitary poll often being referenced as though it was the only one that mattered, whilst ignoring those that don’t back their argument.
Tuesday (28 May) was a case in point – several Conservative spokespeople and pundits jumped on the JL Partners poll that gave Labour a lead of just 12 percentage points (pp) as proof that the polls were narrowing and the big National Service and pensions announcements were turning the tide. Except that this poll was conducted before both announcements, and polls published since show the Labour lead solidly in the 20s.
So, with so much attention on the polls and so much variety, I wanted to set out some things to think about when looking at the polls as we get to Thursday 4 July, but my key message is not to focus on the individual polls – look at the broader picture. Our poll of polls on the Cavendish election hub pulls in a wide range of results and shows you the trend to help you with that.
Don’t compare day-to-day
It’s extremely tempting, in a period where everything points to one giant poll, to compare the polls day-to-day. However, in reality you should only compare the polls done by that pollster with their previous one. JL Partners’ 12pp Labour lead poll came out last night but today we’ve seen a YouGov poll with Labour 27pp ahead. This swing didn’t happen overnight.
A more accurate comparison would be to look at the change between JL Partners’ previous poll in early May, which had a 15pp Labour lead. The change here is put down to the Tory over-65s lead going from 8% to 20%, but again this happened before the big Tory announcements and it’s likely that older undecideds have now simply decided.
Different models
Many pollsters use the same online panel of people to get their samples for election polling. This essentially means that each 2,000ish sample of voters are taken from a larger pool, often responding to polls from different pollsters (usually with some kind of incentive for doing a certain number of polls). The really big ones (YouGov, for example) use their own and Ipsos are one of the few that continue to do telephone polling – which often produces interesting results because it doesn’t exclude those who are less tech savvy.
Beyond that, the key difference is what they do with the “Don’t Knows”. Most polls you’ll see exclude the ‘Don’t Knows’ completely and only show the voting intention of those who have actually decided who they want to vote for. In recent years this has generally meant bigger Labour leads because the Labour vote is stronger: a significant portion of the 2019 Tory vote is now undecided.
The leads are different for a reason…
The polls you’ll see with smaller Labour leads – notably JL Partners and Opinium – try to work out, if pushed, how undecided voters would vote if they made up their minds. JL Partners do this using machine learning – using a variety of predictors like previous voting habits and age, as well as perception of party leaders and policies to essentially predict who the undecideds will vote for. Because more undecideds tend to be 2019 Tory voters than any other group, this means the Tory vote tends to increase and the Labour vote decreases – closing the gap a bit.
This is not without controversy – JL Partners’ method in particular has been scrutinised, to the point that they’ve recently had to start putting out articles to explain what they do. If you want to see the arguments for and against , there are some great X threads in the last few days from Number Cruncher Politics’ Matt Singh (for) and University of Southampton’s Dr Will Jennings in the opposition camp.
The importance of Don’t Knows
I’ve written previously about my fascination with Don’t Knows – they’re the ones to watch as we approach election day. Unless something huge happens, those who have decided they’re voting Labour or Conservative or SNP today will probably still tell the pollsters they’ll do that until election day (that doesn’t mean they will actually vote that way, what with tactical voting likely to be a bigger player this year than ever before).
As we get closer to election day, the number of undecideds should reduce, which will be where the bulk of any ‘narrowing’ comes from. Take Tuesday’s 27pp Labour lead from YouGov.15% of respondents said ‘Don’t Know’ or ‘Will not vote’. Keep them in the data and the results are quite different:
Of the 15% who are ‘Don’t Know’, 20% are 2019 Tory voters, compared to just 9% being 2019 Labour voters. If the polls narrow (see below), much of this will come from this group – generally pollsters find that if you’ve made the decision to switch from one party to another (rather than switch to ‘Don’t Know’) you’re less likely to go back.
The polls always tighten. But…
The core polling narrative at every election is that the polls always narrow. Pollsters are divided on this – as demonstrated in CityAM today – because the sector significantly changed their methods after the 2015 Election when pretty much no one accurately predicted a Conservative majority government. This means that, looking at the broader trends, you should get an accurate picture of how things are shifting as the election gets closer. But the pool of potential switchers is small now, so whilst their may be some narrowing there aren’t that many left to win over.
by David Button, Associate Director and Head of Insight