Can we Finally Retire Useless Poll-Based Seat Projections?

Another election, another problematic night for Canada's pollsters.  Although national polling picked up the late election Liberal surge (some say the polls themselves were responsible for that surge - but that's another blogpost), they failed to predict the magnitude of upset.  The Liberals finished with 39.5% of the popular vote - fractionally lower than the Conservatives received in 2011.  At the end of the night, the Liberals finished with 184 (54.4%) of the seats, giving Canada yet another false majority government (in 2011, the Conservatives finished with 166 of 308 seats, or 53.9% of available seats).

The extent of the Liberal surge wasn't the biggest surprise for pollsters - the collapse of the New Democrats, and to a lesser extent, the Conservative parties proved to be the biggest story of the night for pollsters - or more specifically, how those collapses played out on the ground.

I'll use one pollster as an example here, as all of the pollsters either under-predicted the Liberal surge, or over-predicted the Conservative/NDP vote to a degree.

EKOS released its final poll of the election on October 18, 2015.  That poll had the Liberals at 35.8% - almost 4 points lower than where they actually finished.  The Conservatives were pegged at 31.9% (which was actually bang on what the Cons ended up with), and the NDP were at 20.4% (less than 1 point of where the ended up at 19.7%). EKOS once again over-estimated the Greens, giving them 5.6% on the 18th, compared to our election-night finish of 3.5%,  To a lesser extent, EKOS also over-estimated the Bloc, awarding them 4.9% on the 18th versus their actual finish of 4.7%.

Those numbers aren't out in left field when it comes to final tallies - at least not percentage-wise.  Election night surges appear to be more commonplace now that we find ourselves in a 24-hour polling cycle, so it stands to reason that there may be big shifts happening on that last day of voting.  Certainly the trends for all three parties over the final week showed that the Liberals were heading for a victory of some sort (although few were predicting a majority - shoutout to Mainstreet Technologies here, who predicted the majority two weeks ago), and that the NDP was in free-fall.

The real big polling failures, however, happened when the polling numbers were used by pollsters and others to forecast the number of seats each party was likely to have on election night.  Most seat-based projection methodologies start with a snapshot from the last election, and use national and/or regional polling data to predict outcomes in various ridings.  A lot of the criticism leveled at this
approach has to do with the fact that local personalities, issues and other factors are left out of the equation.  However, for those watching Canada's elections, it is increasingly becoming apparent that the "local" aspects of electoral success have little to do with election outcomes - save maybe for

having a strong ground game in place in various ridings to pull the vote on election days.  Local issues, candidate's personalities and credentials, are having less and less of an impact on which way the vote goes.  The national campaign is everything.

Given the lack of importance of the "local" in electoral outcomes at the riding level, it should be that the job of the pollster's is made easier for seat projection forecasts.   And yet, instead of accuracy, what we've seen is a failure.

The Liberals ended up with 184 seats; EKOS projected just 151.  Poll aggregator ThreeHundredEight dot com predicted 146, with a range between a low of 124 and high of 161.  The Toronto Star's The Signal had the Liberals at 160, ranging between 140 and 177.

The NDP finished with just 44 seats, yet EKOS projected 54, ThreeHundredEight 66 (range: 51 to 90), and The Signal 50 (range: 29 to 71).  

Even for the Conservatives, the seat projections were off the mark, despite the polls coming closest to predicting that party's popular vote count.  The Conservatives finished with 99 seats.  EKOS had the Cons pegged for 116.  ThreeHundredEight - 118 (ranging between 100 and 139).  The Signal - 120 (ranging between 109 and 133).

Although the popular vote polls all experienced some problems predicting outcomes, the seat projections for the parties proved to be quite out of synch with where the parties ended up on election night.  How can it be that projections based on data that was only somewhat off led to such wildly problematic seat forecasts?

Frankly, the seat projectors starting points are in the wrong place.  Just as pundits were quick to point out that when the writ is dropped, it's a whole new ball game (made in response to the NDP's on-going claim that they just needed 35 more seats to form government - a claim that failed to recognize the fact that some or many of their seats at dissolution might have been at risk of loss - and perhaps the NDP believed it's own hype, and mistakenly took those 2011 electoral successes for granted during the 2015 campaign), so too should pollsters come to recognize this fact: Past results are a poor predictor of future outcomes.

That's something that PunditsGuide's Alice Funke has been on about for years (see: "Pundits' Guide weighs in with a caution against strategic voting," Scott Piatowski, Rabble.ca, October 1, 2008).  PunditsGuide, of course, is storehouse for actual election data.  Funke has done a lot of analysis to come to this conclusion - and yet, election after election (federal, provincial, municipal), pundits fail to take into consideration the wisdom she's gleaned from crunching the numbers.  

As a result, a great disservice is done to Canadian voters - voters who are likely far more influenced by the polls than the parties - or the pollsters - want to give them credit for.

(opinions expressed in this blog are my own and should not be interpreted as being consistent with the Green Parties of Ontario and Canada)