WHICH POLLSTER WAS THE MOST ACCURATE? (ranks both national and state pollsters)

RealClearPolitics ^ | 11/8/04 | T. Bevan & J. McIntyre

[Cableguy]

By and large the pollsters did an excellent job this year, especially when you consider the significant variables they were grappling with (massive registration increases, historic levels of intensity, cell phones, etc) and also that they were working under intense public scrutiny and scorching partisan attacks from both sides.

At the national level the answer to the question is pretty straightforward. Ed Goeas's GW-Battleground Vote Projection and Pew Research got it exactly right. Goeas's final Battleground projection was Bush 51.2, Kerry 47.8, Nader 0.5 and Pew's final allocation was 51-48-1.

CBS News/NY Times also nailed the final spread in the race (Bush +3), though they didn't allocate undecideds which makes their final less impressive than Battleground's and Pew's.

Raghavan Mayur at TIPP also deserves a mention for outperforming almost all of the big media pollsters and coming in just a tick off the final results with their final Bush 50.1, Kerry 48.0, Nader 1.1 projection. Scott Rasmussen of Rasmussen Reports also had a solid final result with their Bush 50.2, Kerry 48.5 final projection.

USA Today/CNN/Gallup finished poorly this year. Ironically, Gallup would have finished at or near the top of the list had they allocated undecideds 50/50 like Pew, as opposed to giving what appears to be 100% of the undecideds to Kerry.

By the way, just as a point for future reference Bush's Job Approval rating among likely voters in Gallup's final poll was 51%. President Bush's percentage of the national vote total this election? 51.0%.

Of the entire group of pollsters this year, only three got it wrong. Celinda Lake of Lake, Snell, Perry & Associates (the Democratic polling firm for the GW-Battleground 2004 Poll), Marist College and FOX News/Opinion Dynamics all projected a John Kerry popular vote victory. (The Democratic polling firm Democracy Corps also called for a one-point Kerry victory.)

Poll	Bush	Kerry	Nader	Error
Final Results	51.0%	48.0%	0.4%	-
Battleground/Tarrance	51.2	47.8	0.5	Dead On
Pew Research	51	48	1	Dead On
CBS/NY Times	49	46	1	0%
TIPP	50.1	48.0	1.1	1%
CBS News	49	47	1	1%
Rasmussen	50.2	48.5	-	1%
ABC/Wash Post	49	48	0	2%
Harris	49	48	1	2%
NBC/WSJ	48	47	1	2%
Reuters/Zogby	48	47	1	2%
Newsweek	50	44	1	3%
CNN/USAT/Gallup	49	49	1	3%
ARG	48	48	1	3%
Marist College	49	50	0	4%
Battleground/Lake	49	51	0	5%
FOX/Opn Dyn	46	48	1	5%

Now to the battleground state polls. Let's stipulate up front that both the national poll and the battleground state analyses are not meant to be comprehensive or quantitative, but rather quick snapshots using a simple, commonsense grading system. We looked at two basic factors to determine accuracy rankings: the percentage of states where the pollster correctly picked the winner and how much the pollster's projections varied from the final result in each state.

One last thing. In addition to calculating the difference between the projections from each polling firm and the final vote totals, we also tried to determine whether we could detect any trends that might show favorability toward one party or another by consistently underestimating or overestimating support for one candidate or another.

Here is what we found from all of the final polling conducted in the 2004 battleground states at the Presidential level (Click Here to View the Full Tabulated Results):

1) Mason-Dixon
Failed to Project Winner: 6.2% | Average Error = 1.8

A final Minnesota poll showing a one-point Bush win is the only blemish on Mason-Dixon's otherwise perfect scorecard this year. Not only did Brad Coker project the correct winner in 15 out of the 16 battleground states we looked at, he did so with amazing accuracy. Four states were dead on the final number and the overall difference between Mason-Dixon's final polls and the actual election results was a minuscule 1.8 points. Furthermore, if you look down through the list of Mason-Dixon's projections it's impossible to detect any consistent leanings toward one candidate or another.

2) Rasmussen
Failed to Project Winner: 6.2% | Average Error = 2.3

Rasmussen's battleground state polling this year was extremely solid and a close second to Mason-Dixon. He was dead on in PA and just missed in IA. Average error was a very good 2.3 points, with NJ and AZ the only states where he wasn't within three points of the final spread. No partisan trends either way.

3) SurveyUSA
Failed to Project Winner: 7.1% | Average Error = 2.8

Some people have questioned methodology and reliability of SurveyUSA's polls. Their performance in the battleground states this year should answer a good number of those questions: 13 out of 14 states called correctly, including dead-on numbers in Maine, Michigan, and Ohio. The difference between projections and actuals in a few of the states (CO, FL, NV & NJ) was on the high side, even if it was within the margin of error. No identifiable leanings toward either candidate.

4) Research 2000
Failed to Project Winner: 14.3% | Average Error = 3.1

Del Ali's firm only conducted polling in seven battleground states this year. They got six of those states right, including nailing a Bush one-point victory in Iowa. The big miss came in Florida, where Research 2000's final poll called for a one-point Kerry win. Average error from the final results was 3.1 percent - which is respectable - though in every instance except one (Iowa) they overestimated support for Kerry and underestimated support for Bush.

5) Quinnipiac
Failed to Project Winner: 33.3% | Average Error = 2.3

Many people dismissed Quinnipiac's final poll in Florida (Bush +8) as an outlier. Wrong. Quinnipiac was closer than most in FL, and they also did a nice job in the only two other states where they polled, NJ and PA. In fact, Quinnipiac would have finished pretty high in our rankings except they called for a tie in Pennsylvania. All three of their projections overestimated the spread for President Bush.

6) Zogby
Failed to Project Winner: 27.3% | Average Error = 3.8

As we all know, Zogby had been on record for months saying that Kerry was going to win this race. Despite his final tracking poll that put Bush ahead by one point nationally, Zogby's polling at the state level reflected his belief that Kerry was going to be the beneficiary of huge turnout - especially among the youth vote. The result is that Zogby missed three of the eleven states he polled in (FL, IA, and NM), had a relatively high error rate across the board (3.8%), and his numbers generally skewed in favor of John Kerry.

Adding insult to injury, Zogby's bizarre election day antics calling for "surprises" in Colorado and Virginia and a decisive 311 electoral vote victory for Kerry suggest he was relying on (not to mention taken in by) the badly skewed early exit poll data.

Let's be honest: Zogby's conduct this year bordered on outrageous. No other independent pollster was out making public predictions of a John Kerry or George W. Bush victory months before hand. And no other pollster decided to wait until 5:30pm Eastern time on election day to post their final numbers.

7) American Research Group
Failed to Project Winner: 50% | Average Error = 2.0

ARG got a bit unlucky this year. They called for a 1-point Bush win in NH and the result was Kerry +1. They also projected a 1-point Kerry win in New Mexico and the result was Bush +1. The big miss, however, came in Florida where ARG's last poll had Kerry up two. ARG did offset these misses by nailing tight outcomes in IA and WI giving them an excellent score in overall average error.

8) FOX News/Opinion Dynamics
Failed to Project Winner: 50% | Average Error = 4.5

It stands to reason that if your national numbers are way off then at least some of your state numbers are going to be bad as well. This is certainly true of FOX's final poll in FL where they showed Kerry ahead by five points. In fact, that 10-point miss stands out as the worst among the final battleground polls we looked at. Another miss in Wisconsin put FOX News/Opinion Dynamics at 50/50 in battlegrounds, with an average error rate of 4.5%

9) Strategic Vision
Failed to Project Winner: 44% | Average Error = 2.4

We were assaulted by some people for labeling Strategic Vision a Republican polling firm. For the record, the reason we did this was twofold: 1) they had a history of polling for Republican clients and 2) their polls were not sponsored by any independent media outlets like newspapers and television stations.

Strategic Vision's projections for a Bush win in MI and a tie in NJ, seems to provide evidence that, at least in these two states, they were skewing toward President Bush. Their polling in the other seven battleground states was reasonable, though except for Florida and Ohio, they consistently underestimated support for Senator Kerry.

10) CNN/USA Today/Gallup
Failed to Project Winner: 67% | Average Error = 5.8

To find Gallup's name at the bottom of the list is nothing short of shocking. In four of the biggest, most important states in the election this year (FL, OH, PA, and WI) CNN/USA Today/Gallup wasn't even close. In fact, they got it exactly backwards calling for Kerry wins in Florida and Ohio by 3 and 4 points, and Bush wins in PA and WI by 4 and 8.

 

[JLAGRAYFOX]

Zogby and his brother are just low life Arab terrorist scum that support the "traitor" Democrat Party!!! End of story!!!

[sevry]

Newsweek's was probably the closest, at 6%, had it not been for the Dem vote drives, and ballot stuffing. I thought Bush would EASILY win by 8%. He was running against a joke of a candidate, a man who might fear he'll be asked even to resign his Senate seat in the coming months. But the Dem did get out the vote. What they did is spoil the margin of victory. They had no way to know that in sheer numbers, the Bush turnout would be what it was.

The ultimate numbers might be interesting, broken down by party affiliation. All I've seen are the totals for each candidate, even by county. But I haven't seen those county votes broken down by GOP/Dem. And I wonder how many actually registered . . as Dem voted for Bush?

[GoMonster]

The pollster in charge of Dynamics is a bigtime Demorat.

[Future Useless Eater]

Nationally, Gallup's final poll wasn't so bad, but he allocated almost ALL the undecided to Kerry.

If he had instead honored historical averages (per Pat Caddell) that 60% of the last-minute undecided breaks for the INCUMBENT in presidential elections, then Gallup's final numbers would have been VERY close

[okstate]

"Mason Dixon being top dog is no suprise, but the performance of the phonebots -- Rasmussen and SurveyUSA -- certainly was."

I'm not surprised by SurveyUSA's results. They were the 2nd-best (behind M-D) in 2002 and in 2000.

[kesg]

I'm not surprised by SurveyUSA's results. They were the 2nd-best (behind M-D) in 2002 and in 2000.

Maybe, but in 2004 they seemed to get some key battleground states wrong in most of their polling, most notably Ohio and Florida (where at least until the end they consistently tilted three or four more points to the French Poodle guy than other major polls).

[finnman69]

bump for reference

[AHerald]

Reference Bump ... For the manic poll-addicted Freepers who are driving us more level-headed types bonkers. ;)

[An American!]

Good reference on last election results. Here is another image

from 2004 in mid October

[AHerald]

Thanks for that graphic.