Grading 2008 election prognostications on a curve

Mark Schmitt at The American Prospect grades various theories of how the election turned out. Less numerical than our evaluation.

I think there are some verbal miscues in my previous posting, but I think I'm just gonna leave 'em alone.

2008 State by State Electoral Predictions Compared

The 2008 election is over and Obama solidly trounced McCain. Several sites kept a running total of polling results by state, including pollster.com and RealClearPolitics. Nate Silver at fivethirtyeight.com went a step further, analyzing the polling data and explicitly making predictions of the voting totals by state. I also found RJ Elliot who gives a state by state prediction of the voting percentages for Obama and McCain. There are no doubt other sites with predictions. I found one, but they wanted money to view their predictions, so they are omitted from this comparison.

I took the final estimates from these four sites to compare to the actual vote totals. A caveat should be kept in mind: Silver was making an explicit prediction about the election, while RealClearPolitics and pollster.com are aggregating and summarizing polling data, though presumably with the purpose of predicting the election. RJ Elliot also was making predictions, though I didn't see any explanation of his methodology. For actual results, I took data from Dave Leip's Atlas of U.S. Presidential Elections. I didn't try other sources, this was the first that I found.

Final cavil: I collected fivethirtyeight.com and RealClearPolitics' predictions on the day after the election. The pollster.com and RJ Elliot predictions were collected later, on Thanksgiving or the day after. While I don't think have any suggestion that any of these predictions were changed after the election, there certainly is the possibility of that happening.

Results
Herewith the results. I took the predictions for Obama and McCain's state by state percentage, subtracted from the actual percentage, squared the difference, averaged across states, and took the square root. This gives a root mean square error (RMSE) of prediction difference from truth for both the Obama side and the McCain side of the estimates. The results are in the image. Hopefully the image shows up in the right place.


For each site there are two results, for Obama and for McCain. For each row there are two numbers. It turns out that RealClearPolitics (RCP) only predicted results for 38 states out of the 50 states plus DC. The others predicted for all 51. The first column (starts 3.0, 2.9) includes all 51 states plus DC plus the national vote treated as a 52nd state. The second column is the predictions only on the 38 states where RCP made predictions. So the first column you can compare 538, pollster and Elliot. In the second column you can compare all 4 sites.

The states where RCP did not make predictions were the very partisan and small states where the winner was very clear throughout the entire election (think Utah and DC) and where there were very few polls. Both 538 and pollster do much better in the accuracy of their predictions for the subset where RCP was making predictions. On the 38 states where RCP made predictions, we see that 538 had the smallest RMSE followed by pollster then Elliot with RCP having the worst predictions.

Across the 52 predictions (50 states + DC + national), Elliot edges 538 perhaps barely, both of which definitely beats pollster. The RCP result in the first column is not comparable to the other numbers. I inspected the individual residuals (differences prediction minus reality) in the states where RCP did not make predictions. These residuals were often very large. The largest residuals for 538 were from the states (Alaska, Arkansas, DC, Hawaii, Louisiana, Nevada, Vermont, Wyoming), defined as being in error by more than 4% on either Obama's or McCain's vote total. DC, Hawaii, Louisiana Vermont and Wyoming were not predicted by RCP.

Election 2008: Tie Ball Game

At pollster.com, they have their own calling of the presidential race based on current state polling. Their count has 208 electoral votes for McCain, 202 for Obama and 128 electoral votes in toss-up states. If you go through the toss-up states, there are 61 votes for McCain and 67 for Obama, leading to an exact electoral college tie.

Here are who is ahead in each of the 128-electoral-vote-totaling toss-up states:
JM
MT - 3
NV -5
IN - 11
OH-20
WV - 5
VA - 13
NH - 4

BO
CO - 9
MN - 10
WI - 10
MI - 17
PA - 21

Total: 67 Barack Obama, 61 John McCain
Added to 202 BO and 208 JM
gives us a 269-269 tie.

Update: Was still tied yesterday using the same mechanism, though the strong and leaning Obama and McCain states had shifted quite a bit. And I did a little editing above.

Red Meat and How Obama Needs to Run Against McCain

I've been feeling that Obama's campaign has been missing the red meat of attacks at Bush. Tonight Tuesday at the Democratic convention, we've been getting some well placed licks in, but there needs to be more.

When people complain that they don't know what Obama stands for, I think they really mean, lay out what Bush stands for and why you are different. Not just who you are for, but what you are for, what you will do and most importantly how that is different from who Bush is for, what Bush has done and what Bush wants for the country.

TalkingPointsMemo has an excellent further discussion that they need to really explain that McCain is worse than Bush. Not the same as, but much much worse. As a marketing gimmick, 'the same as' is bland and boring. The Democrats need to explain what Bush is for, what McCain is for and why that will be even worse for the country than Bush. Not just as bad as, but much much worse.