TV viewers watching news of the approach of Sandy last week may not have thought much of President Obama’s urgent appeal to supporters to vote early. While no one who has worked turnout operations and statistic models ever accepted Nate Silver’s prediction that Obama had an 87% chance of winning (NY Times, October 4, 2012), we do understand the urgency in President Obama’s appeal in hopes of keeping his re-election prospects from slipping further below 50%.
As someone who was run hundreds of grassroots turnout efforts including a bipartisan one featured in Campaigns & Elections for a massive grassroots effort by a bipartisan team, but also constantly runs sports and political statistical models, I understand exactly why the President’s appeal was so urgent and why Silver’s percentage was so far off.
I feel pretty confident that Romney wins 267 electoral votes including Arizona, Colorado, Florida, Iowa, New Hampshire, North Carolina, and Virginia. In addition, Romney will easily win Nevada, Ohio, and Wisconsin ON ELECTION DAY. If Romney wins any of those three with the other 267, he wins the election.
John Nolte pointed out that Gallup is reporting that Romney has shockingly overtaken Obama in early voting, in which case Romney wins easily. But for argument’s sake, let’s assume the much more liberal Time poll from last week is correct and Obama is beating Romney in early voting but Romney is running 30% better on election day than among early voters.
Ohio started early voting October 2, but Nevada and Wisconsin not until October 20 and 22, respectively – so even though Ohio is the closest to the Atlantic Ocean, storms that could hold down the efforts to pick people up and get them to the polls costs the Obama campaign many campaign many votes they will not get back on Election Day. Even if we are generous and say Obama is a 3-to-1 favorite in each of those states, his chance of winning all three is only 45%. Romney is like a football team that needs to win one of their last three games to make the playoffs and are 3-to-1 underdogs in each game. They still have a 55% chance of making the playoffs, and Obama knows that if he loses early votes then his chances in each state get closer to 50% — at which point his chance of winning the overall vote drops to 20%.
However, there is more good news for Romney for those of us who know turnout and number-crunching. Romney’s surge gives him an outside chance in four other states – Pennsylvania, Minnesota, Michigan and Connecticut, and Obama does not get a head start in any of those four states – the voting is done on Election Day except for absentee balloting.
If we give Romney only a 25% chance in the four early voting states, and give him only a 10% chance in Minnesota, Michigan, Connecticut and Pennsylvania, then Romney has more than a 70% chance of winning at least one of those states to elect him President with the 267 mentioned earlier. In fact, he clearly has a much better chance than the 25% and 10% chance we’ve given for these states – give him just a 40%/20% chance in each of these states and Romney’s chances of getting at least one of them increases to over 88%.
Now, I won’t make the same mistake Nate made in giving Obama an 87% chance of winning. In fact, while I believe Romney gets the 267 I list, it is certainly possible Obama could take one or two back. If New Hampshire or Iowa went Obama, Romney would still be President unless Nevada was his only breakthrough of the seven, etc. And, in fact, Romney’s chances of becoming President actually rest at about 58% right now – within a few points of where I have had it all year, even when he was down almost double digits in the pre-debate polls a couple of times.
So how was Silver so far off when he gave Romney a 13% chance on October 4? First off, that percentage was published by him in the New York Times and taken by him to CNN appearances – so it would appear the intended affect may have been to depress Romney’s chance of winning rather than reporting on it.
Of course, if Obama wins in a nail-biter, you could argue Silver was correct in the same way the same way one of the two guys arguing in a bar and giving Romney or Obama an 100% chance will be correct. But it’s really hard to argue that in a race coming down this close was an 87% to 13% probability a month ago.
Silver’s approach was flawed in three ways:
1. Nate does not acknowledge the margin of error.
The first problem with Silver’s 87% certainty is the number of presidential elections used to arrive at a formula to predict future elections. Elections have changed completely since television, so you are talking about 13 elections of any real relevance. The margin of error basing any study on 13 Presidential elections is 27%. Can you imagine releasing a poll you had run of 13 likely voters and announcing that Romney led Obama 62% to 31% with 7% undecided? Because of the 27% margin of error, Romney might be as high as 89% or as low as 35% if based on only 13 respondents. While I am oversimplifying the comparison, the fact is there are not nearly enough presidential elections on which one can base an 87% prediction. When we ran very good models on how likely college prospects were to produce in the NBA, we had hundreds and hundreds of past performances, and on college projections, we analyzed thousands of player seasons. Presidential elections are unique, and 13 since the first TV debates are not nearly enough.
2. What percent of factors can you calculate?
Silver has to first acknowledge what percentage of factors he can and cannot measure – but there is no way he is measuring factors that constitute 87% of why an election is won, so to put a figure that high simply can’t be done with credibility.
When I publish basketball calculations I acknowledge that there are certain things that cannot be calculated – picks, the pass before the assist, etc., things for which subjective adjustments must be made, but most things can be tracked through stats. However, when I wrote a book calculating how good players were from the 1920s, there were far fewer stats, so I had to put a greater percentage of the valuation on subjective accounts, such as what was written about players.
When I crunch numbers for the hundreds of candidate and referenda campaigns in which I’ve been involved, I likewise balance polling and hard numbers with what we are hearing in door-to-door knocks (including records on knocks on the doors of 170,000 voters a couple of years ago) and at events and focus groups. In a presidential election, Silver can calculate a lot of things, but I’ve knocked on doors in eight states, I’ve been to hundreds of political meetings and organized turnout, and these subjective factors must be given weight. You can’t claim an 87% certainty on a presidential result based on statistical analysis alone.
Likewise, anyone involved in campaigns knows that the most dramatic shift for a challenger is often the first debate when, for the first time, voters perceive the challenger as an “equal” to the current President/Congressman, etc. While we certainly did not know that Obama would have such a bad debate, anyone who knew politics knew that the first debate would likely help Romney substantially, even if the two were equal. Thus, to put an 87% certainty on a win before what was very likely to be one of the most dramatic events of the campaign again shows naïveté on how political campaigns play out.
3. In light of the first two, Silver needs to adjust his certainty for how much of his prediction should be statistical/objective vs. experience/subjective.
On www.valueaddbasketball.com I projected months ago that the top three teams next year would be Indiana, UCLA, and Louisville. If I had gone one step further and said, “there is a 50% chance those three teams will be the top three teams,” I would have been making the same mistake Silver is making (though to a lesser degree for Silver). In fact, there is less than a 1% chance the top three will line up in that order.
The problem isn’t with a statistical model is bad – it’s a way of sorting through thousands of players on more than 300 teams. If you don’t know anything else, I would believe that those three teams had a better chance than any three teams you would pick – but I built that model months ahead of time before knowing who would be injured, late transfers, etc. Jay Bilas or some other expert will have much more subjective knowledge and factors on who was playing hurt last year, etc., and weigh additional factors.
Likewise, Silver just can’t put an 87% certainty on a prediction when he doesn’t have the knowledge of going through campaigns and understanding things firsthand, such as how Democrats overcome transportation issues with lower-income voters by putting much more effort into a ground game, or how Republicans work rural events.
John Pudner wrote his first statistical analysis for the New York Post more than 20 years ago, ranking pitchers on opposite Sundays from when Bill James ranked batters. Since then he has run more than 200 political campaigns for candidates or ballot issues while continuing to write statistical analysis for sports and developing a database of thousands of basketball players at www.valueaddbasketball.com.