A Word of Caution: Electoral-Vote.com

I have been told by several different people lately, all of whom seem to be extremely confident in their sources, that this state or that will go this way or that regardless of whatever analysis I've been reading. That usually sounds like the sort of arrogant crap I'd be saying to people, not the other way around, so I went looking at just what sources provided such good information. In four of four people queried I received the same answer: electoral-vote.com. Well I've had it, let's take a look at just why the votemaster is running a bad site for electoral projections, shall we?

Let me start by saying that I may already be off on the wrong foot here. I enjoy e-v quite a bit, even have a little widget over on the sidebar there, but my browsing it is usually for quick access to the Political Wire as well as occasional analysis. That is to say decidedly not the projection itself. Why? A few reasons immediately jump to mind, but let's start with 2004. This was the projection on election day. Guess what, a lot is wrong with it. Here's a side-by-side of what was projected, and what actually happened:


Electoral-vote.com projection on November 2nd 2004:
Actual election outcome later that day:Just to point out the obvious first: Hawaii was projected red. I'm going to repeat that. Ahem. Hawaii was projected red. This is the beginning of my opening argument. Hawaii was supposed to go red, according to the votemaster (that's the alias of the guy who runs the site in case you haven't gathered as much yet), because there was a poll done on October 20th, 2004 by SMS Research which showed Bush at +1 over Kerry. The most recent poll prior to this SMS poll was done on September, 11th, 2004 by more-well-known American Research Group (ARG). We'll hear more about ARG in a minute, but for the time being just know that their poll had Kerry +10. That means, according to Electoral-vote.com, that there was a shift of 11 points towards Bush in the closing months of the election without anyone even having to campaign there. There's more. The reason that this massive swing shows up on the site is because every poll is given equal weight; thus, if only one guy does a poll of your state this week, that guy's numbers are gospel to the votemaster.

Here's some more evidence of that being a flawed system. Let's go back to those two electoral maps, and take a look at some other states called incorrectly. Wisconsin and Florida are both wrong, with Florida being extra-wrong compared to Wisconsin. Guess what, both of those polls were done by Opinion Dynamics on October 31, 2004. Nice track record they built for themselves over at Opinion Dynamics. Let's put a little icing on the cake here, take a look at New Mexico, see how it's blue? Who called that? John Zogby, master of getting it wrong. Granted, the SUSA poll in Iowa was wrong, too, but they've gone and made up for that greatly this time around. Let's take a peek at how these others are doing thus far, shall we?

This is a link to Nate Silver's pollster ratings (when it comes to way-over-the-top analysis, like the kind needed to get an idea of a who's-who in polling, Nate's the guy to go to). The Pollster-Introduced Error score works like golf, the less Nate sees your numbers as needing adjustment the better you are at your job. Naturally, Ann Selzer (+.75) is the last word in polling, but by the time you get down into the PPP range you're really looking at blips on the radar as they are bound to be outliers most of the time. Let's see, who is ranked lower than PPP (+1.86).... hmm, oh! Why, look. It's John Zogby (+2.16) in the bottom half of pollsters, and in the bottom quarter are ARG (+2.32) and Opinion Dynamics (+2.36). If only there was some way of not weighting these guys the same as the better pollsters... oh that's right Nate does that.

This process has been going on this cycle as well, of course, and it's really starting to get pople misinformed. ARG put out a poll in West Virginia claiming Obama +8. Since this represented a sixteen point swing from Rasmussen not two weeks earlier, both campaigns as well as seven pollsters flocked to the state only to find out that it is not actually a toss-up. Electoral-vote, of course, kept the ARG numbers up on their projection for a full week.

Last example, it's what actually got me to finally sit down and write this, is today the votemaster's decided to label Florida as exactly tied. His reasoning is that four polls have been out in the past four days (he is slow to update his projections, which also annoys me) that all cancel each other out. In other words, two of the polls had Obama winning, and two had McCain winning, so the state is an exact tie. He doesn't mention that the polls having McCain up both only have him up +1, and that one of the polls is from FOX. He also doesn't mention that only one of the pro-Obama polls has a +1 for Obama (the PPP poll) while the other (the much-better rated Quinnipiac) has it as +5. Florida may go either way, who knows, but those four polls by no means give any sensible person reason to call things precisely dead-even. It's clearly leaning Obama if only by a couple of points.

To review: Electoral-vote.com is a great place to get some quick analysis, very prompt political wire updates (faster than fivethirtyeight for some reason), and last week's polls presented as a legitimate electoral projection. Those first two are great reasons to stop by, but leave that third one be - or at least keep it to yourself when we're talking about politics.