Polling: The Good, the Bad, and the Ugly: Part III

Self - Vanity | 10/31/12 | Self

[GOPFlack]

Polling – The Good, The Bad, And The Ugly – Part III Robo Polls and Modern Methodology Issues

Sorry I was away for several days. Softball ran deep into Sunday as my daughter’s team stubbornly played their way out of the losers bracket. Then Sandy whacked my power until last night. But, maybe today is a good day for another installment as I’m sure the last week poll jitters are setting in on everyone.

Knowing that Scott Rasmussen is the pollster of choice for many here, at least in determining the national popular vote, I thought I would chat about the advantages and limitations of robo polling. As I discussed previously, campaigns generally won’t rely on automated polling unless they lack the resources for quality live interview polls. I attribute that mainly to a deeply ingrained attitude by the traditional polling community that too much can go wrong with automated polling. What do they believe are the limitations?

First, there is a sense that the amount of data you can collect from a robo poll is limited. It’s hard to get someone to hang in with an automated pollster if they start going into deep crosstab type questions. For simple preference and a few toplines you might be ok. But the sense is a live interviewer has a better ability to get a caller invested in a lengthy survey. It’s simply harder to hang up on a person than a machine once you have started.

Second, there is angst about whether or not people are actually capable of pushing those response buttons correctly, even though the polls will ask you to confirm your answer to a given question. I don’t think the instances of people playing with pollsters in robo polls are any greater than what people do in live interview polls. So, I’m not so sure about this one.

Third, there is a belief that his likely voter screen isn’t as tight as a live interviewer can achieve. Of course, this could be a problem with many pollsters of all stripes.

Fourth, and most importantly, these polls can significantly under sample certain populations skewing the results. Taking a giant sample can’t overcome those deficiencies. Let me expand on this point further. In 2000, Rasmussen was just really beginning to deploy and refine a novel technology. As with any new technology, it has to be tweaked and adjusted over time against real life experience in the field. Rasmussen missed the late movement to Gore over the weekend before the election. Whether that movement was related to the DUI or a massive urban ground game Donna Brazille put together that kicked in is open to question – probably both. But what was not in dispute was that Rasmussen significantly under sampled minority votes.

Why? To his credit, he went back to find out and adjust the technology. What he found was that for whatever reason, minorities tend to not answer automated polls. That will obviously skew your results to the high side for the Republican no matter what percentage of the minority population votes. To his credit, he adjusted to account for this factor. So when folks ask if he reweights his results, the answer is “yes.” I don’t know how he determines the changes he makes to his raw data or if it stays fixed over time simply because it is baked into the technology. The bottom line is he basically got it right the last two elections as far as the popular vote because whatever he did worked. That’s one reason why I get steamed when I hear Dems claim he is a GOP operative. He is nothing of the sort, or he wouldn’t feel the need to actually do this to ensure accuracy.

Here is my personal take on what his polling does well. First, I think he is pretty good at spotting trend lines in his POTUS track because he does a three - day roll with a very large sample. I don’t like Gallup because it is slower to pick things up because it is a seven - day roll. Most campaign tracking will use a three - day. Second, he can produce quick polling that can again pick up on trends in states that might not show up in more traditional polls until later.

I am not a huge fan of his state polling. Not because he isn’t often right, but I don’t like one night samples no matter the size (see Part I on what can happen to skew a one night poll).

As far as automated polling, he is state of the art. Whether that art is approaching perfection is a larger question that will have to be looked at as time goes on. To his credit, though, he doesn’t sit by and simply stay with what he does, but keeps striving to improve it - which speaks to a level of integrity that so many other outfits, including many non - automated ones, simply lack.

Which brings me to PPP. PPP works for the SEIU. I promise you as much money as they have to spend, the SEIU does not rely on PPP’s work to make campaign decisions. If they did, I would turn in my union card immediately (if I had one) because they are flushing money down the toilet. PPP is a campaign influence tool, not a scientific polling outfit designed to tell the truth. How else do you explain that a day before the deadline for Akin to drop out they magically show him ahead then right after the deadline he is getting trounced? Why did they even do those polls? Because they wanted him to stay in the race knowing what would happen if he did and then show his situation was hopeless right after to dry up his money. I could go on and on here, but I wouldn’t put a plug nickel on anything they say. The better way to look at what they do is to ask whom are they trying to motivate or demoralize for their left wing clients?

Finally, a few notes on problems everyone has in the modern polling world. First, low response rate. The response rate for polling is consistently below ten percent and getting lower. That is most likely a reflection of people being exhausted with being constantly barraged on their landlines and increasingly cel phones by endless solicitations and having caller ID. Caller ID makes people loathe to answer calls from “private caller,” odd area codes, or “unknown caller” in order to avoid having to be annoyed with some silly solicitation. That doesn’t mean it can’t be overcome by calling more numbers, but it begs the question as to who is actually answering these things in the first place? Hyper-partisans, lonely weirdos who want to talk to someone, etc?

Second, almost everyone except the very elderly are moving away from landlines to cel phones. To no one’s surprise, the only reason most people, unless they live in a rural area, still have a landline is for internet service. Pollsters are trying to adjust by calling cel phone exchanges now. But people are very picky about who they will talk to on their mobile line and really resent anyone they don’t know or need to talk to calling them. I had a WP/ABC pollster call me in August on my cel and I hung up because I was so ticked that I had been called on my cel. If it wasn’t from a 202 (DC) exchange I would never have answered in the first place. I refuse to believe that this problem has been overcome by random digit dialing because the problem is personal and attitudinal, not just technological.

Some pollsters have gone to the other extreme by doing online polling. To say that this stuff is just short of alchemy would be an insult to alchemists. There has to be by its very nature absurd problems with likely voter screens, socio-economic differences etc. of who and how these things measure. The amount of voodoo you would have to do fix one of these things is akin, again, to prognosticating not polling.

As always, I hope this helps. I will conclude tomorrow or Friday about why I hate exit polls and how they are used as campaign weapon to direct or depress turnout – at least in recent elections and have a few comments on late tracking polls.

[GOPFlack]

Ping

[MdPoke]

Thanks for posting.

[Hero]

Thanks again. Keep ‘em comin’.

[Jet Jaguar]

Poll ping

[Jet Jaguar]

Poll Ping.

[dfergu7477]

Thanks very much. Look forward to your final installment!

[ApplegateRanch]

Thanks.

[Andy'smom]

Thank you. Very informative.