Electoral Prediction and Cognitive Bias

Over the past couple of weeks, a startling number of pundits and commentators have been relentlessly attacking political statistician Nate Silver and his blog FiveThirtyEight. Why? Because his electoral prediction model, which uses a mix of national polls, state polls, demographic information and economic data, calculates that Obama's chances of winning the election are slightly better than the convention wisdom suggests. At present, they're hovering at a little below 75 percent.

Nate has a great track record when it comes to predictions: In 2008, he called all 35 Senate races right, as well as 49 of 50 states for president. In 2010, he correctly called 34 of 37 Senate races and 36 of 37 governors' races. When he was wrong, the outcome was usually decided by a razor-thin margin. And his reasoning for this year's prediction is simple: Obama holds small leads in enough crucial swing states (e.g. Ohio, Wisconsin, Nevada, Iowa) to get him to the needed 270 electoral votes.

But the critics dismiss all that, declaring in a textbook case of 20–20 hindsight that those other predictions were a cakewalk, and this time is different. They whine that Nate's biased because he's rooting for Obama (which never shows in his incredibly calm and even-handed commentary). They complain that his poll weighting system is subjective and introduces bias (even though it's actually based on objective measurements of poll recency, methodology and track record). And when all else fails, they mock him as puny and effeminate.

It would be one thing if Nate was alone in making the forecast that he does... but he's not. The various prediction markets, which despite their flaws are usually pretty accurate, tend to mirror his probability estimate very closely. And it turns out that his model is actually quite generous to Romney compared to other competing models of the same type.

So why have Nate's projections been subjected to such merciless criticism? For several reasons, none of which have anything to do with the merits of his model.

One is that the media has an incentive to portray elections as close—in this case, a virtual dead heat—so that people get excited and tune in for more coverage. So when someone comes along claiming that one candidate actually has a small but substantial lead, the public and the less-savvy pundits are naturally skeptical.

Another reason comes down to the fact that people do a very poor job of grasping probabilities. Commentators hear Nate estimate a 75% chance of Obama winning and think, "Wow, he must be really sure of himself." That's certainly the impression Joe Scarborough gave when he insisted that Obama's chances were at 50.1%. But Nate's prediction isn't all that dramatic. What many fail to understand is that if you assign Event X a 75% probability of occurring, it means you expect it to not happen 25% of the time. In fact, if such events occur more often than three out of four times in the long run, you've made a very real error.

The third and most glaring reason is a combination of wishful thinking and confirmation bias. Conservatives want very badly for Romney to win this election (or more to the point, for Obama to lose), so some will do anything to interpret the data as favorably as possible. Their most common defense is an allegation that the pollsters are (intentionally or not) oversampling Democrats—a claim based on the faulty assumption that party identification is static, rather than fluid and subject to change in response to current events. Another is to hold polls favoring Obama to a higher methodological standard, while clinging uncritically to those favoring Romney, such as the overly volatile Gallup tracking poll. Still another is to ignore polls altogether and point to less direct indicators, like an alleged closing of the gap between male and female voters or the candidates' favorability ratings. Yet one more is to claim, baselessly, that undecided voters break dramatically against the incumbent. Anything to keep the dream alive.

Meet the man who thought Bush's response to Katrina
would be his crowning achievement.

Finally, the more cynical conservative pundits may be consciously biasing their predictions. They have an incentive to tell their audience what they want to hear—Republicans who want to be reassured will look to them for certainty. Dick Morris, for instance, has such a catastrophic track record of predicting GOP victories that it's hard to imagine he's anything but an opportunist looking to get more attention and sell more books.

Now, before any staunch liberals out there get too cocky about the follies of their counterparts across the aisle, I should point out that this mindset is by no means limited to one party or ideology. In 2004, Democrats were guilty of groundlessly criticizing poll oversampling just as Republicans are today. And a quick perusal of the comments on Nate's blog posts will reveal many left-leaning readers expressing far more confidence in Obama's chances than is warranted by the data. Many of his acolytes also seem to follow the blog just to pacify their anxieties rather than to follow the data wherever it leads. So no matter what your politics, beware of how your biases influence your views and expectations.

Now I leave you with one final prediction. If Romney wins, you can bet that all the critics will be crowing with triumph and declaring the demise of FiveThirtyEight. But if Nate turns out to be right, you can bet those same critics will brush it off as a fluke, blaming voter fraud or Hurricane Sandy or anything else they can think of to resolve their cognitive dissonance, in much the same way that a cult will rationalize its failed doomsday predictions.