Asking the experts: Which polls are, or aren’t, legitimate?

Hotair ^ | 09/27/2012 | AllahPundit

[SeekAndFind]

After yesterday's post on poll trustworthiness, I started wondering whether there's any poll or model that's been consistently accurate over time and therefore worth watching down the stretch as a weathervane of where the race really stands. I e-mailed two experts whom I trust and put that question to them. Is there any steady signal they trust amid the cacophony of statistical noise? Anyone we can look to as a beacon in the darkness when the NYT drops its next D+10 sample of Utah or whatever on us?

Short answer: No, there's no one whom they count on to get it more or less right every time. Polling averages did well in 2008 and 2004 but not so well in 2000 and 1996. The first person I spoke to told me flatly that it’s not worth paying much attention to the numbers now because the assumptions being made about the composition of the electorate on November 6 differ too widely among individual pollsters to distill a truly useful average. That uncertainty is compounded by the fact that, with six weeks left until America votes, there's still an ocean full of potential "black swans" --- wonderful/terrible jobs reports, war with Iran, a new eurozone spasm, etc --- that could send the trendlines fluttering. (Team Romney told Rich Lowry they think their dip in Gallup’s tracker lately is due to one such black-swan moment whose effects are already fading.) Once we get to within a week or two of election day and pollsters’ assumptions finally start to coalesce, the polling averages will become more reliable as an indicator of where the race really stands. As my own addendum to that, I think we’re close enough to the first debate that there’s no point picking through polls until late next week at the earliest. Why worry about this week’s data when there’s a hugely important event that’s bound to affect the race right around the corner?

My other source had less to say about the reliability of polling averages generally than their reliability with respect to specific candidates. He told me that if you look at historical averages, you find that they underestimated Gore in 2000, Dole in 1996, and Bush 41 in 1992 — all of them dull, somewhat stiff candidates whom their respective bases weren’t thrilled about. Why would polls miss the mark on people like that? His theory is that pollsters pay lots of attention to voter enthusiasm but less attention to whether voters say they’re “certain” to vote, and in the case of candidates who aren’t beloved by their base, those two variables don’t match up especially well. There were plenty of Republicans who weren’t enthusiastic about Bush and Dole but who were nonetheless certain to vote for them in hopes of defeating the Democrat. Ditto for Gore vis-a-vis the GOP. (Kerry and McCain were also dim lights to their bases and the polls gauged their support pretty well, but in McCain’s case he had a huge shot of enthusiasm late from adding Palin to the ticket.) He thinks the same thing could be happening this year — essentially, pollsters are keying off of the Dems’ slight edge in “enthusiasm” and missing the fact that plenty of unenthusiastic Republicans will be at the polls anyway to vote for a guy who’s taken to citing RomneyCare lately as proof of his empathy. If that’s the case, then they’re lowballing Romney’s support. And in a tight race, that’s potentially a decisive error.

See? I am capable of writing a poll post that’s not hopelessly eeyorish. Although I’d be lying if I said this wasn’t how I felt when I saw those Gallup numbers yesterday. Oof.

[RedStateRocker]

Well, you are most welcome to your opinion, and far be it from me to give self proclaimed experts any more credence than they deserve...

I don’t claim to be an ‘expert’, but I did get formal training and have worked professionally in the field, and thus at least as qualified as most people to comment, with all due respect :-)

My point is only that the reality is more akin to marketing research than the polls you and I see... the polls most likley to be accurate those that are used to guide the allocation of assets (as opposed to predicting who will win), are usually very propriatary and are considered to be accurate by the people who matter (the candidates and their staffs) by dint of the fact that the people who create them are paid to do so. You can disagree or disparage all day, but it’s the same combination of “scientific” measuring and ‘mojo’(interpetation) that every company from Ford to Johnson and Johnson do.

Unless you consider every company that pays for market research to be stupid for doing so.

And sampling theory is not really ‘complex’, 8th grade math at most; my point was only that it is sort of counter intuitive but verifiable that it CAN be accurate.

[ROCKLOBSTER]

How in the hell did they judge accuracy?

I guess by comparing their polling percentages with the actual election results.