Will National drop 6% in three months? A less than serious analysis…

Rob Salmond at Polity notes that in the 2008 and 2011 election campaigns, National has dropped 6% in the three month campaign window. His analysis is fairly simple:

All I did was find any published poll where the field dates included the day three months before election day, then compared that to the final election result.

On this analysis, Mr Salmond finds that in 2008, National went from an average of 51% on 8 August to an election result of 44.9% (a drop of 6.1%), while in 2011 National dropped from an average of 53.1% on 26 August to an election result of 47.3% (a 5.8% decrease).

So what would we get if we use Mr Salmond’s analysis on this current election cycle? The election will be held on 20 September 2014, so we take all polls where the field dates include 20 June 2014. That’s just two polls – the 3News Reid Research poll that put National at 49.7%, and the last Roy Morgan poll that had National on 48%. That’s an average of 48.9%, meaning Mr Salmond would expect to see National drop to about 43%.

But wait! Mr Salmond notes that for his 2008 election analysis, he made “one one-day exception to get the 2008 N up” from two polls to three. That meant he included a Colmar Brunton poll that spanned 9 to 14 August, one day outside the three month date of 8 August. Including that poll made no difference to the result, as the Colmar Brunton result was 51%, exactly in the middle of the other two polls.

If we apply the 2008 “one-day exception” to 2014, we also get to include the last Colmar Brunton poll, which put National on 50%. The average of the three polls therefore increases slightly to 49.2%.

Of course, the issue with this sort of analysis is the tiny data set used. In each election year, the analysis is based on just three polls. If there’s an outlier poll in the data set, it will skew the average.

So does Mr Salmond’s analysis include any outliers? Well, in both the 2008 and 2011 data sets there’s a poll from Fairfax. Fairfax’s final poll prior to the 2005, 2008 and 2011 put National, in every one of those three elections, well higher than any other polling company’s final poll. And in between elections, Fairfax tends to have National polling almost 2% higher than the industry average.

So how high did Fairfax put National three months out from the 2008 and 2011 elections? In 2008, they had National at 54% – that’s 3% higher than the Colmar Brunton poll and 6% higher than the Roy Morgan. In 2011, they had National at a staggering 57.1% – over 5% higher than the other two polls which both had National on 52%.

Eradicate the Fairfax outliers from Mr Salmond’s analysis and the 2008 average drops to 49.5%, while the 2011 average drops to 52%. That’s a three month fall of 4.6% in 2008 and 4.7% in 2011.

Eradicate the outliers and the “one-day exception” and there’s only one poll left in the 2008 data set, the Roy Morgan showing National on 48% – 3.1% higher than National’s 2008 election result. The average drop over both elections? 3.9% – rather lower than 6%.

If National’s average polling as at 20 June 2014 (not including the “one-day exception”) is 48.9%, we’d expect to see National therefore drop to 45%.

Of course, my tongue is lodged firmly in my cheek with this “analysis”. My point is merely that statistics can be made to say whatever you wish. Hell, if we include the 2005 election in the analysis, applying the “one-day exception” and excluding Fairfax, there’s only one poll in the data set and it shows National 3.1% below it’s election result of 39.1%. Average everything out on that basis and National drops only 2.1% on average! If their average polling as at 20 June 2014 (including the “one-day exception”) is 49.2%, we can then expect National to get 47.1%, just 0.2% below what they got last election! That should make John Key and Stephen Joyce sleep better tonight…


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s