Why the Standard shouldn’t get too hung up on individual polls – it’s the rolling averages that should be worrying them

The release of the Len Brown poll on Thursday raised some eyebrows, not least because the sample size was only 248 Aucklanders. Of those 248 Aucklanders, only 22.7% said they would vote for Mr Brown in the 2016 elections, while 57.7% said nay.

The issues with the poll were many and varied:

1. Sample size: With only 248 respondents, that’s one hell of a margin of error. The Herald didn’t report the margin of error, but the standard margin of error for that sort of sample size would be just over 6%.

2. Possible issues with weighting: Of those surveyed, 85% said they’d voted in the Auckland Council election, which is interesting, as voter turnout was only 36%. Further, only 37% of those who said they voted, said they voted for Len Brown. That’s despite him getting 47.8% on election day.

Now I don’t believe that responses regarding whether people previously voted or who they previously voted for are really worth much more than a grain of salt. People lie, even when it’s to an anonymous pollster on the phone. We know that voting is a Good Thing, so we don’t like to admit we were slack and never quite got round to posting the damn ballot paper. Likewise, we don’t like admitting we voted for the guy who’s obviously not the flavour of the month. No one likes admitting they voted for someone who now seems to be a bit of a loser.

Nonetheless, the weighting discrepancies in the Len Brown poll seem to be simply too large to just write off as being the result inaccurate provision of data by the respondents.

3. Len Brown v The Imaginary Candidate : It doesn’t often mean much when people are asked to choose between a real person and a generic opposition. In America, for instance, poll results tended to change dramatically in the run-up to the last Presidential election, when the line of questioning went from comparing President Obama to a generic un-named Republican opponent, to comparing Obama to each of the specific likely Republican candidates. Faced with a Republican candidate who was real and flawed – no longer imaginary and perfect – Obama’s stats tended to rocket up.

It’s all very well to ask, “Would you vote for Mr Brown in 2016?” However, those results may change dramatically once the other candidates are known. After all, if either of Mr Palino or Mr Banks threw their names in again as the centre-right candidate, Mr Brown might just scrape back in…

(For the record, I’m not a Len Brown supporter, and I think he should have resigned.)

The Len Brown poll was of course a part of the larger Herald-Digipoll which had delivered such doom and gloom to Labour supporters everywhere. Thus, over at the Standard, they seized upon the notion that the poll was a rogue – if the Auckland figures were so badly skewed, the poll could be safely discounted.

That’s all very well, and it’s undoubtedly nice to be able to sweep the worst poll of the year under the carpet, but Mickey Savage (the author of the Standard post) keeps banging away with the statement, “David Cunliffe has been quoted as saying that Labour’s internal polling is in the mid 30s”. Repetition does not necessarily create truth.

The thing about individual polls which the Standard is quite right about is that they can be wrong. That’s why they have margins of error. Every so often, one comes along that is just plain out-of-whack with everything else. Individual polls bounce up and down. They have statistical noise.

Which is why people should focus on the bigger picture – if we bundle up all of the polls that are coming out, and we create rolling averages and trend lines, what does it show? Polls of Polls, such as are done at Rob Salmond’s Polity, David Farrar’s Curiablog and here, try and minimise the rogue polls and statistical noise.

And that’s where things still don’t look rosy for the Labour party. Regardless of whether the Herald-Digipoll is swept under the carpet,  the average poll data this year has not showed Labour looking even close to being in the mid-30s. Of the 10 major polls released thus far this year, 5 have had Labour under 32%. Only once have they hit 34%, and not once have they crossed that mark. Rob Salmond’s Poll of Polls has them on 32.6% (last updated 24 February 2014), David Farrar has them on 32.7% (last updated 15 February 2014), while I’ve currently got them on 31.7% (last updated yesterday).

Individual polls come and go. But when the polls of polls all put your party at between 31.7% and 32.7%, it’s probably time to worry.

Advertisements

2 comments

  1. Kia ora.

    I am not disputing that Labour’s polling is not stellar and I agree that a poll of polls can provide the most accurate results. I personally follow Danyl McLaughlan’s poll of polls and his approach I believe has some merit.

    I was initially quiet about the Herald poll but it was only after I saw the rather extreme conclusions reached about Len Brown that I started to question the Auckland part of the sample and when I saw Labour’s support crashing in Auckland as opposed to the rest of the country that I questioned its accuracy. My post was designed to question the accuracy of the poll rather than state it was wrong.

    The only people seizing on the poll are a number of Herald commentators claiming that Labour is in crisis. For some reason the Roy Morgan poll released a few days later which showed Labour + Green as being neck and neck with National has not received the same sort of attention.

    Of course Labour wishes that its polling was higher. But in an MMP environment it is the sum of the whole that is important.

    1. Kia ora, Greg. Thanks for taking the time to respond, and your reasons for your Standard post are certainly fair enough.

      I’ll be interested to see how Danyl McLaughlan’s poll of polls fares, come election day, as his corrections for poll bias seem somewhat extreme to me. But statistical models are always able to be adjusted, and it might be me having to rethink my in-house polling bias corrections after 20 September…

      Certainly, it’s the sum of the whole that matters, rather than Labour’s individual polling, and because of that I’ve been waiting to no avail to see just how the media would cover the latest Roy Morgan poll. One would have thought the left and right level-pegging would have been worth covering, given the previous coverage along the lines of ‘the left is finished’. I guess the Roy Morgan results don’t fit their current story arc.

      Of course, if Labour continues to poll weakly, it makes it harder for them to persuade the softer National voters to abandon the seemingly-more-successful ship (self-fulfilling prophecies of defeat, and the like). 45.5% puts both National and Labour/Greens a good few percentage points away from getting over the line, so it’s vital for the left that Labour can strip away a few more per cent from National, and that the undecideds plump for Labour or the Greens in larger numbers than National. I’m not convinced that the Greens will be able to grow their vote much higher than the 14% of the latest Roy Morgan, which leaves Labour with a bit of work to do.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s