As the campaign season gets into full swing, voters can expect a deluge of polls. Every major media outfit and several independent polling organizations will provide almost real-time information on every twist and turn in the political landscape. The polls will not only cheer or frighten partisans on all sides, they will likely have a gravitational affect on individual campaigns themselves, as candidates adjust their campaigns to polling results. But, voters should beware. Even modern day polling is more art than science. 

All polls reflect certain biases–not necessarily in the political sense–of pollsters. Taking a small sample and extrapolating it to the overall electorate involves lots of judgement calls that may not provide an accurate picture of the political landscape. While voters should look to sites like RealClearPolitics, which average a basket of recent polls to smooth out aberrations, the occasional “outlier” poll, showing results wildly different than other polls, is occasionally correct. It mostly comes down to the choices pollsters make in conducting their poll.

If you are reading this, you’re likely fairly politically aware and understand some basic differences between many polls. You understand that the first step in accessing a poll is looking at what’s called the “voter screen.” In other words, is the poll of adults, registered voters or likely voters. The difference matters a lot:

Both Pew Research and Nate Silver have each looked at the differences for different elections from 2004, 2008 and 2010; and they both came to essentially the same answer:

– Polling “adults” generally favors Democrats by a net of 7%.
– Polling “registered voters” generally favors Democrats by a net of 4%.
– Polling “likely voters” is always the most accurate.

So if you have one poll of “adults” which says D53.5-R46.5, another of “registered voters” which shows D52-R48, and another of “likely voters” which shows D50-R50, they’re all saying the same thing. When you factor in the relevant adjustments for each screen, they’re all showing a tie at somewhere around an exact 50/50 split of those who will actually wind up choosing between Democrats and Republicans.

For the life of me, I don’t understand why media outlets like The Associated Press continue to poll “adults” on political issues. Around 20% of adults aren’t registered to vote. Putting aside the rather large inherent bias toward Democrats, why do we even care to know the political views of those who won’t be voting? Its about as useful as polling Canadians on their preference of U.S. politicians. 

With the exception of Rasmussen Reports, however, most media and polling organizations use the registered voter screen until late in the campaign. This is due to the not unreasonable belief that, early in the campaign season, it is difficult to estimate who is most likely to show up at the polls. It won’t come as a shock to learn that people often lie in polls, claiming they will definitely vote but then, for a variety of reasons, fail to do so. So, as you see polls of registered voters, keep in mind that there is a general bias of +4% for Democrat candidates. 

But, even polls using a likely voter screen can be misleading. At this point, we need to discuss one of the less talked about and least understood aspects of polling: weighting. 

When you start from a random sample of voters and begin conducting the actual interviews, it is very likely that the total universe of voters you actually speak with aren’t representative of the overall populace. You may have too many male, white, low-income, high education or Midwestern voters. Polling firms deal with this by “weighting” the sample, essentially tossing certain interviews so that the final results reflect responses from a representative sample that matches the nation’s demographics. 

Most of this is fairly technical and, with the exception of the occasional disreputable firm, fairly straightforward. Where it gets very tricky is where polling firms “weight” their sample based on their estimate of the partisan breakdown of the electorate. In other words, how many democrats, republicans and independents they include in their sample. This judgement call can throw off even the more accurate likely voter screen.

In 2008, an obviously big year for Democrats, the partisan breakdown of the actual electorate was:

By ideology, the breakdown was:

In 2010, an obviously big year for the GOP, the partisan breakdown of the actual was:

By ideology, the breakdown was:

So, any poll in 2010 that used 2008 as their baseline, i.e. weighting their polling sample to reflect the partisan breakdown of 2008, would have been wildly off. Remember, the pollster would have “tossed” certain interviews to get to the D-39, R-32 and I-29 sample. 

So, is the electorate in 2012 going to be more like 2008 or 2010? Personally, with an energized GOP and conservative base, I don’t think the 2012 electorate is going to come remotely close to the partisan breakdown we saw in 2008. But, most pollsters seem to disagree and are weighting their polls for just such an outcome.

Organizations like Gallup and The Associated Press make it almost impossible to find out their partisan screen. Newer organizations, though, like Politico, DailyKos and Fox News do make this information available. 

A recent poll by DailyKos/PPP, which had Obama up by 3 points, had the following partisan screen:

So, the DailyKos poll expects a bigger Democrat and liberal turnout than in 2008. Somehow, I don’t think that’s likely.

Politico‘s recent poll, which found Romney with a 1-point lead had the following partisan screen:

(Note: I’ve done my own “weighting” and assigned “leans GOP” and “leans Democrat” to “Independents.”) 

A recent FoxNews poll, which showed Obama with a 7-point lead had this partisan breakdown:

What color is the sky in FoxNews’ world if they think the Democrats, in 2012, are going to increase their share of the electorate from 2008? When was it, exactly, that a bunch of independents suddenly switched to the Democrat party? 

I think all of these polls are oversampling Democrats and undersampling Republicans. The nadir for the GOP was 2008, when they only made up 32% of the electorate. In the wake of ObamaCare and a stalled economy, there is no way the GOP is going to sit home like they did when faced with a McCain candidacy. Also, the Democrats were at the high-water mark of the “hope and change” promise of Obama in 2008, when they made up 39% of the electorate. There is no way they reach that level again. 

So, every poll you see, dig deep into the partisan breakdown. Your mileage may vary, but you’d be right to adjust the numbers accordingly.