Are political polls accurate? CA recall study of 20 polls says...NEVER TRUST POLLS AGAIN!

RealClearPolitics.com & original material ^ | 10/21/03 | Wolfstar

[Wolfstar]

This study is original material created for the information and use of FreeRepublic.com readers and active participants. Please feel free to share the information with others.

[Wolfstar]

Pinging as promised.

[Wolfstar]

Ping, as discussed.

[ambrose]

ping

[Wolfstar]

Pinging you because of your strong interest in the recall.

[Timesink]

Media Shenanigans / Schadenfreude ping

[Timesink]

One question: People's opinions on things do change over time. Am I misinterpreting that you are, to some extent, arguing that polls taken two months before the election are somehow less valid because they did not end up matching with the final election results?

[KayEyeDoubleDee]

What is/are your background/credentials, in terms of statistcs, etc?

[goldstategop]

Nope. Voter surveys conducted AFTER the election showed most voters made up their minds a month ago. How come NONE of the polls caught this? This is another area where they didn't see voter behavior coming.

[Timesink]

Nope. Voter surveys conducted AFTER the election showed most voters made up their minds a month ago. How come NONE of the polls caught this?

Not to defend the pollsters, but if they didn't get the pre-election surveys right, why would the get the post-election surveys right?

(Actually, I guess that's not exactly a defense of the pollsters' abilities at all, is it? Hee hee.)

I'm also a total believer in the average person's inability to remember how they felt on an issue one week in the past compared to today, much less a month ago or more. Lots of people rationalize outcomes in their own minds after the fact.

[Wolfstar]

The study focuses on the accuracy of polls, not on their validity. My underlying premise is that the accuracy of most polls can't be proven. The value of this study is that these 20 polls were taken during a compressed period of time much closer to the election they were predicting than, say, polls that compare President Bush to a generic, unnamed Democrat more than a year out from the 2004 election. It's easier to evaluate these 20 polls for accuracy against the election as compared to presidential "approval ratings," for example. How can anyone prove their accuracy? Against what benchmark? Can't be done.

For a more in-depth answer to your query, look at the question: Do polls become more accurate closer to an election? The answer is that, in this study, they trended toward more accuracy closer to the election, but not strongly. Even at the end, only one poll got all five questions right within it's MOE.

[Wolfstar]

None. I am a consumer of news and information who wants some means to evaluate ephemeral things like polls. However, one does not need a background in statistics to do the simple math required for this study. Either a result was within its poll's MOE or it was not as compared to the actual election results. Not rocket science. In fact, not science at all, but plain old-fashioned athritmatic and observation.

[Wolfstar]

Ooops, that should be "arithmetic" not the misspelled "arithmatic." I've been up waaay too late trying to get this posted. Time to hit the sack.

[onyx]

What a magnificent post. Bookmarked with appreciation and gratitude. Thank you, Wolfstar.

[onyx]

POSSE ping TO FABULOUS AND INFORMATIVE POST.

[KayEyeDoubleDee]

Either a result was within its poll's MOE or it was not as compared to the actual election results

I'm not trying to be a critic here, and I totally appreciate your efforts. However, I am almost positive that, for example, a MOE of 5% means something like "there is a 90% chance that the actual results will lie within +/-5% of the poll results." Anyway, something to think about.

[heleny]

Wow, the LAT got 60%? I know 20% (the Schwarzenegger vote predictions) was always wrong, and many FReepers recognized that, but I didn't know their results on the other four questions were so (relatively) accurate.

If someone analyzed the actual error (instead of categorizing right/wrong), the LAT probably would not be in the top two, since their Schwarzenegger predictions were very far from the actual results.

Thanks for summarizing the results and sharing with us all!

[heleny]

a MOE of 5% means something like "there is a 90% chance that the actual results will lie within +/-5% of the poll results."

Some of the polls stated something like that, but the newspapers or TV news charts often simply say the MOE was 5%, and it's often left out entirely in oral TV/radio news.

Maybe they couldn't predict the dynamics of the actual voters this time, because voters were motivated in ways not typical of the usual elections. If they had realized this, the polls should have claimed an 80% or lower accuracy.

[heleny]

Some polls allowed for an "undecided" category, while some may have pushed the respondents to choose yes/no or to tell their preference in the candidate question. I don't know what effect on the accuracy removing the undecideds might have.

[RatSlayer]

you said
"Lots of people rationalize outcomes in their own minds after the fact."

Truer words were never spoken.