Making sense of all these different polls

A lot of mystery is surrounding the polls for the last few days. Some of them show a big bump (bounce?) for Kerry. Others show no change whatsoever.

There is a pattern to the discrepancies, however. Whether this pattern explains why the polls have deviated in the last few days is a question I cannot answer. I believe it worth nothing this pattern however. Perhaps one of you has some expertise beyond my own and can comment.

Let's look at the major polls from the last two weeks. Gallup Last Week: Bush +8; Gallup This Week: Tie CBS News / NY Times Last Week: Bush +8; This Week: Bush +1 Zogby Two Weeks Ago: Bush +2; This Week: Bush +1 Fox News Last Week: Bush +2; This Week: Bush +3 ABC News / Washington Post Last Week: Bush +6; This Week: Bush+5 Pew Last Week: Bush +8; This Week: Bush+5 ARG Last Week: Bush +1: This Week: Tie

What is funky about this? Two of the polls that had a big, big, big Bush lead last week (unreasonably big) are now showing a tie. The polls that showed a moderate Bush lead last weeks ago show the same Bush lead. The third poll with a big Bush lead is now back with the moderate Bush lead polls.

One way to view the change in the polls might be that Pew, Gallup and CBS have undergone some kind of correction. CBS and Gallup are the only polls that have seen a statistically significant shift in the polls this week, and they were the ones who were the outlying polls last week.

A week ago, I noted (not on this site) that the discrepancy between the Gallup poll and the ARG poll among registered voters was outside the margin of error. In other words, it was not random statistical variation that was producing the different results. The result was caused by some methodological difference between them.

Now this is no longer the case. All of the polls are reconcilable with all of the other polls. (Bush +2 to +3) My advice to the folks out there is to just relax about these polls. The Kerry surge might actually be these formerly outlying polls correcting themselves to more accurately reflect the electorate.

And, if the real world is actually Bush +3, and things remain constant, that would be something like Bush 51, Kerry 48, Nader 1 on election day. Were we expecting anything except something this close?

I am interested in talking about this with some savvy Freepers. One thing I do not want to discuss, though, is whether these polls are biased in the political sense. I do not believe that to be the case. It is not in the interest of any of these polls to skew their results. While the pollsters/journalists have a political bias, they also have an accuracy bias with these polls, since their polling will be tested on November 2nd. We should presume that the companies doing these polls do not have a political agenda that translates into these polls (the bias comes when the journalists interpret the polls).

Greetings, fellow Freepers, lurkers, Patriots, and all USA-loving folks!

This is the new media speaking, and we are growing faster than a pond full of guppies. The old media is dead—they just don’t know it yet. But they are starting to sense it, and are panicking.

The genie is out of the bottle. The internet now has hundreds of millions of fact-checkers, lie detectors, and cool brainiacs in their pajamas, laughing our butts off at the lame attempts of the “journalism majors” on TV to spin things their way.

The media wing of the democrat party is dying. CBS, ABC, NBC, all except FOX are tanking in their ratings, losing money, and dying a slow, welcome death.

Who broke the Dan Rather phony document story? One of our braniacs here, in pajamas, Buckhead. The “th” heard round-the-world ignited blew Dan out of the water in a couple of hours.

For those of you new here, this is not uncommon. Free Republic has over 100,000 registered users, and a whole lot more than that come here every day. We have scientists, brilliant PhD’s, gun owners, good-ole-small business owners, Capitalists, meat-eaters, and some of the most brilliant in the media. Yes, they lurk here, too. They know that Free Republic breaks stories faster, and more accurately, than any site, or any media, in the world.

Thanks to Jim Robinson, the founder. He was sick of the Clinton lies and crimes, and did something about it. He started this site. At first it barely survived, and Jim put his money where his mouth is. He almost lost his house. Some of us early folks here remember mailing checks in, to keep it running. We knew how important this place is, and now the world knows.

Feel like throwing a brick through your TV every time you watch CNN? Do something about it. Don’t just lurk, jump in with us. Become part of history, and tell your grandkids you were one of the early supporters. Yes, this is early. Very early. Believe it or not, this site is only 6 years old. When you look back in 20 years, you will be proud to have been a part of the founding.

Please join us, and contribute. Do it by credit card. Become a monthly donor—just send $3 a month. That’s all it takes. Think your $3 a month won’t make a difference? You are wrong. It makes all the difference in the world. Just like this site. And when you see your name on the monthly donor list, you will feel so proud! It’s amazing how it lifts you up, to know that you are making a difference.

I know that for some of you, even that small amount hurts. I’ve been there, believe me. But when you see that line item in your credit card bill every month, you will smile from ear-to-ear, and know that you are actually making a difference in this screwed-up world.

Come on in, and join us, and let’s change the world!

IMHO, the only polls that count are the "money polls," like TradeSports.com. Using TS.com odds as of 3:00 P.M yesterday, I did a total EV calculation by state using the latest closing odds the way we were taught to do it in B-School, and the way I’ve done projections for the banks and investment bankers ever since. Multiply the probability times the objective, and out comes the answer. Using this approach, it’s a fairly close EV race, but as of 3:00 p.m. yesterday, GW’s probability weighted EV count is 281.98, based on TS.com odds.

I also ran the numbers with the EVs for the slam-dunk states, which I defined as states with >=90% or <=10%, and allocating 100% of the EVs one way or the other. It didn’t make any difference to speak of; GW’s EV count using this calculation rendered 281.42 EVs.

This wasn’t satisfactory to me from a sensitivity perspective, so I started narrowing the range to shove full vote counts into one candidate’s lap based on the best odds available.

80-20 = 282.87
70-30 = 280.64
60-40 = 285.83
55-45 = 293.08
50-50 = 295.00

Kerry’s EV count actually worsens as close states are assigned one way or the other, because GW has a number of states in which he is narrowly leading, and Kerry has no states in which he is narrowly leading. His leads are all substantial to enormous. This has implications on the popular vote count; I think we could quite likely see GW re-elected and lose the popular vote.

Using the 50-50 results, I then arbitrarily assigned to Kerry the states where Bush has a relatively small odds advantage:

Bush odds less than or equal to 55: IA, NH, revised EV count = 284

Bush odds less than or equal to 60: Add WI, revised EV count = 274

Bush odds less than or equal to 65: Add FL, revised EV count = 247, GW loses.

So, the key states, in order of risk, are presently Iowa, New Hampshire, Wisconsin, and Florida. Based on the odds, that’s where I would be spending my time and effort. The rest of the states are so far out of the running one way or the other, it’s just not worth the effort to influence them. Even though it is less risky of a state versus IA, NH, and WI, Florida, as was the case last time, is the decisive state. Starting from any of the probability weighted scenarios set forth above Florida can take GW down all by itself.

I assume the market is picking up the possibility of vote fraud; but, who knows what effect that might have. Bush’s Ohio contract is trading at 66, which make it, Florida, trading at 61, and Wisconsin, trading at 64, the greatest opportunities for Democrat vote fraud. It would take a lot of fraud to swing these two states, but if I were a ‘rat fraudster, those are the two states where I would be concentrating.

It’s interesting that you can’t really do the analysis in reverse, because Bush has only one contract trading in the 35-50 range, New Mexico, which is trading at 35. PA is trading at 34, and Minnesota at 33. Oregon and Washington both appear hopeless. I don’t see any real possibilities of pleasant surprises for GW based on the present odds.

Over all, however, GW is the odds on favorite to win a majority of the electoral votes. As far as the "market" is concerned, Florida is safe, and with Florida GW wins.

Polls rely on random sampling, which is difficult to achieve due to "confounding factors" such as what kind of people own phones, what kind of people answer a phones at what time of the day or week, what kind of bother to take a poll, how different kinds of people are distributed in different area codes. If you make a lot of calls to Berkley California Kerry wins in a landslide, Nadar comes in second, the communist party comes in third etc.

So the pollsters try to minimize these confounding factors by making a weighting adjustment for party affiliation. The problem is they have to guess what ratio of Democrats, Independents, and Republicans to use. Usually they base this on exit polling from previous elections. Some pollsters do not make such an adjustment. Still others decide whether or not they think they need the adjustment after the poll is taken.

Polls, such as Newsweek, that do not weight tend to be rather erratic in these ratios, and in the case of most of the polls that I have looked at that had a debate bounce for Kerry, they have shifted from a high Republican mix to a high Democrat mix. This suggest either they have fallen prey to confounding factors, or there has been a sudden national shift in party affiliation.

Polls, such as Fox/Opinion Dynamics that decide whether to weight after seeing their results tend to show very little change in general, and don't tend to be helpful. Essentially they end up being tweaked to show what the pollster already thinks.

Polls that weight by party affiliation are probably the best for tracking trends, but they are still of questionable use in determining the real results. I have not seen much of a Kerry debate bounce in these polls.

I think that all of these post-debate polls, taken immediately after the debate, and particularly on a weekend are less-than random. The proximity to the debate poisons the "randomness" of the sample. For instance, it may well likely change the way people decide to cooperate with the caller, or if the question "Who won the debate?" is asked just prior to "Who will you vote for?" it could act as a "push" poll because it gets the respondent thinking positively about Kerry just prior to going for the big "Who will you vote for?" question.

Normally, outside the influence of the debate results, all of the pollsters take and report their polls at varying times during the week and do not have the immediate biasing influence of their perceptions of the debate. What we have here is a confluence of all pollsters taking their polls at the same time, and producing in actuallity more of a referendum of who won the debate than who people plan to vote for in November. I think once the pollsters get back to polling during normal days and away from the debate, Kerry's temporary bounce will fade.

Those are all good points, but you must remember that there are still good reasons to let partisan identification vary in the polls.

The presumption that partisan identification is an independent variable, i.e. that it is a unchanging cause rather than a variable effect, freezes the poll from picking up potential changes in it, which would make it a dependent variable. That partisan identification can change, that it can act as a dependent variable, would mean that polls that treat it as an independent variable have a poor methodology.

These polls with varying numbers are not just commenting on the horserace. They are also commenting on how many partisans they pick up in the real world. These numbers can vary depending upon events. Thus, Bush had a poor night on Thursday, so the next day when Newsweek or Gallup calls, weak Republicans identify themselves as Independents, Democrat-leaning Independents identify themselves as Democrats. The change in party ID is caused by events on the ground; thus, it is best understood (in this case) as a dependent variable. Meanwhile, Zogby, Fox News or the others that freeze party ID fail to pick up that change because they treat it as independent.

I am not saying here that Gallup and the other polls that let party ID vary are right, but I am also not saying that Fox News is right, either. Both sides are taking a guess on this issue. Both guesses have some justification and neither is a decisively clear choice. You have made a strong case for considering party ID as an independent variable, but there are strong reasons to consider it a dependent variable. What is the reality? We shall not know the answer until 11-2, where we will see if partisan ID is different than it has been in years past.

More than that, thoughI think you are making a mountain out of what is a statistical molehill. I have a lot of problems with the Newsweek poll, but both that poll and the Gallup poll have partisan samples that are within the margin of error from their past partisan sample. Statistically speaking, you cannot say that anything is really happening in that poll. The change in partisan ID is entirely explicable by the kind of random statistical variation that occurs anytime you take a sample of a population to draw inferences about the latter.

This is a point that people (not you in this post) usually fail to take from polling. When, for instance, the CBS News poll show the race tied at 47-47, what they are saying is that they are 95% confidant that Bush has somewhere between 44% and 50%, and that they are 95% confidant that Kerry has somewhere between 44% and 50%. So, all they are saying is that the race is somewhere between Bush +6 and Kerry +6.

Statistical sampling of any population is, at best, a rough estimate. Usually, as in the case of most social science questions, that roughness is not a problem. But in a presidential contest, one where the difference in 2000 was 537 votes, there is a precision required that polling simply cannot provide. Pollsters call their work an "art" in recognition of this fact. They are being a tad disingenuous when they say that. It is not an "art." Rather, it is a science that is poorly suited for presidential politics.

The other thing to remember is that the polls using likely voters, hence the most accurate, are the ones still showing a Bush lead. Newsweek's polling used registered voters only which most often skews towards Democrats. I worked in the polling industry for 8 years and Newsweek is looked on as a joke. They do not balance by party I.D. and often over-sample members of one or the other party. Media polls are quick and dirty, more about being on time than being right. Additionally, though they did not do so in their latest poll, Newsweek often polls on Friday night which most in the industry know you must avoid. More affluent voters are less likely to be at home and thus the poll skews towards Democrats when done on Friday night.

Gallup's latest may even be a bigger mess. Apparently they polled a combination of registered AND likely voters, then culled out the likely voter sample to come up with a separate result. That is faulty methodology at best since pulling out a sub-sample like that could well mean you've pulled out a sub-sample without any real geographical balance. For instance, those likely voters may very well come from regions and states that are more heavily Democratic such as the West Coast. Frankly I think in that event you're almost better off just doing an all-registed voter poll than a mix of both likely and registered voters.

Anyway, that's my two cents. I worked for Time/CNN's polling outfit, Yankelovich Partners, (it's now someone else) and I saw how these media polls get done. It's likely watching sausage getting made. There's not much in the way of demographic quotas or weighting, and don't even get me started on the type of people they put on those phones. Probably half of them are punching in Kerry when the respondent says Bush.

I agree with you about the problem of the closeness to the debate skewing the polls. I agree particularly because I think Kerry's victory was purely stylistic, and therefore likely ephemeral.

However, there is no peculiar confluence of polls going on here. Gallup's poll was released Sunday night, which is the night Gallup always releases its poll. ABC News started its daily tracking Monday as planned. The only poll that was fishy, to my mind, was the Newsweek poll.

Gallup's poll for CNN and USA Today has been a weekend poll this campaign season. Weekends in the summer are one thing, weekends in the fall are another. Things get complicated because of football. I wonder whether or not we shall continue to see Gallup as a "pro-Kerry" outlier for the rest of the campaign season.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.