Free Republic Browse · Search News/Activism Topics · Post Article

Poll Math (Fun with Zogby!)
10/21/2008 | PhilosopherStones

Posted on 10/21/2008 11:58:01 AM PDT by PhilosopherStones

Thought I'd have a little fun with poll math. Zogby was kind enough to provide some data which I have compiled into two tables below.

The original Zogby data (which they use to promote their business, so they must be proud of it) can be found here.

Explanation of Table 1:
Table one shows how accurate Zogby was when compared to actual election results (I've omitted races in which their was a significant third party vote or where Zogby had no polling data posted. Hey, you don't like it? Make yer own DANG spreadsheet!)

Col 1 is just the line number. Col 2 is the race. Col 3 is Zogby's pre-election number for Republicans. Col 4, ditto for Dems. Col 5 is the (absolute) spread between the two candidates according to Zogby. Col 6, 7 and 8 are the ACTUAL election results. Col 9 (E) is the difference between the Zogby results and the election results, ordered by greatest to least. Finally, Col 10 (C) shows whether Zogby called the race correctly (R) or incorrectly (W).

 # RACE R D Dif R D Dif E C 1 SEN, WI 38.0 47.2 9.2 67 30 37.0 27.8 R 2 GOV, TN 40.5 54.3 13.8 30 69 39.0 25.2 R 3 GOV, AZ 43.7 51.3 7.6 35 63 28.0 20.4 R 4 SEN, NM 37.5 58.2 20.7 30 70 40.0 19.3 R 5 GOV, OH 43.3 50.8 7.5 37 60 23.0 15.5 R 6 SEN, WA 46.1 50 3.9 39 58 19.0 15.1 R 7 GOV, CO 46.6 46.4 0.2 41 56 15.0 14.8 W 8 SEN, MN 42.8 50.5 7.7 38 58 20.0 12.3 R 9 HOUSE, IL-6 40.0 54.2 14.2 49 51 2.0 12.2 R 10 SEN, NY 33.0 57.3 24.3 31 67 36.0 11.7 R 11 GOV, AR 48.2 45.3 2.9 41 55 14.0 11.1 W 12 SEN, OH 47.3 48.7 1.4 44 56 12.0 10.6 R 13 HOUSE, VA-5 46.0 36.4 9.6 59 40 19.0 9.4 R 14 HOUSE, NM-1 43.9 53.2 9.3 50 50 0.0 9.3 W 15 SEN, PA 43.8 52.7 8.9 41 59 18.0 9.1 R 16 SEN, MI 41.9 49.3 7.4 41 57 16.0 8.6 R 17 SEN, MA 32.4 61.8 29.4 31 69 38.0 8.6 R 18 SEN, RI 38.8 53.4 14.6 47 53 6.0 8.6 R 19 GOV, CA 47.6 38.8 8.8 56 39 17.0 8.2 R 20 HOUSE, NY-20 51.2 37.1 14.1 47 53 6.0 8.1 W 21 GOV, GA 48.3 36.3 12.0 58 38 20.0 8.0 R 22 GOV, IA 46.4 47.6 1.2 45 54 9.0 7.8 R 23 GOV, WI 47.2 46.7 0.5 45 53 8.0 7.5 W 24 SEN, NV 58.3 36.8 21.5 55 41 14.0 7.5 R 25 GOV, OR 46.6 47.2 0.6 43 51 8.0 7.4 R 26 SEN, TX 54.8 35.9 18.9 62 36 26.0 7.1 R 27 GOV, PA 42.0 55.2 13.2 40 60 20.0 6.8 R 28 HOUSE, FL-22 39.6 49.2 9.6 51 47 4.0 5.6 R 29 HOUSE, VA-2 51.0 43.4 7.6 51 49 2.0 5.6 R 30 SEN, MD 44.3 48.8 4.5 44 54 10.0 5.5 R 31 GOV, CT 53.1 40.5 12.6 63 45 18.0 5.4 R 32 HOUSE, CO-7 39.8 54.1 14.3 42 51 9.0 5.3 R 33 HOUSE, CT-4 43.5 50.7 7.2 50 48 2.0 5.2 W 34 GOV, MI 42.7 51.6 8.9 42 56 14.0 5.1 R 35 HOUSE, KY-4 41.9 44.9 3.0 52 44 8.0 5.0 W 36 HOUSE, IN-2 39.0 51.9 12.9 46 54 8.0 4.9 R 37 GOV, MA 32.7 58.1 25.4 35 56 21.0 4.4 R 38 HOUSE, CT-2 46.6 42.3 4.3 50 50 0.0 4.3 R 39 GOV, NY 27.3 63.2 35.9 29 69 40.0 4.1 R 40 HOUSE, IA-1 41.5 48.6 7.1 43 54 11.0 3.9 R 41 HOUSE, OH-18 33.1 53.2 20.1 38 62 24.0 3.9 R 42 HOUSE, NC-11 43.3 47.6 4.3 46 54 8.0 3.7 R 43 SEN, FL 49.9 45.1 4.8 53 45 8.0 3.2 R 44 GOV, NV 46.6 47.5 0.9 48 44 4.0 3.1 W 45 HOUSE, PA-6 43.5 48.6 5.1 51 49 2.0 3.1 W 46 HOUSE, NY-26 47.0 46 1.0 52 48 4.0 3.0 R 47 HOUSE, NY-24 39.3 45.6 6.3 45 54 9.0 2.7 R 48 GOV, IL 33.0 39.6 6.6 40 49 9.0 2.4 R 49 HOUSE, MN-6 52.2 42.1 10.1 50 42 8.0 2.1 R 50 SEN, NJ 42.8 48.7 5.9 45 53 8.0 2.1 R 51 SEN, MT 45.5 47.4 1.9 49 49 0.0 1.9 R 52 SEN, TN 49.3 48.1 1.2 51 48 3.0 1.8 R 53 HOUSE, NY-25 45.0 48.6 3.6 51 49 2.0 1.6 W 54 GOV, MD 43.9 49.3 5.4 46 53 7.0 1.6 R 55 SEN, AZ 51.8 44.4 7.4 53 44 9.0 1.6 R 56 HOUSE, IN-9 46.0 47.5 1.5 46 49 3.0 1.5 R 57 HOUSE, KS-2 44.2 46.7 2.5 47 51 4.0 1.5 R 58 HOUSE, AZ-8 41.3 52.5 11.2 42 54 12.0 0.8 R 59 SEN, MO 47.3 48.6 1.3 47 49 2.0 0.7 R 60 GOV, MN 44.7 46.1 1.4 47 46 1.0 0.4 W 61 SEN, VA 43.7 44.8 1.1 49 50 1.0 0.1 R

You'll notice that we have to get all the way down to line 39 before we see that the actual results and the Zogby predictions are within 4 points of each other. In other words, more than 60 percent of the time, Zogby's predictions were outside of their own given margins of error.

The other thing to note is that even when Zogby was within the margin of error, they called the race wrong 18% of the time (4 out of 22).

Table 2 is the same as table 1 except ordered by Zogby spread:

 # RACE R D Dif R D Dif E C 1 GOV, CO 46.6 46.4 0.2 41 56 15.0 14.8 W 2 GOV, WI 47.2 46.7 0.5 45 53 8.0 7.5 W 3 GOV, OR 46.6 47.2 0.6 43 51 8.0 7.4 R 4 GOV, NV 46.6 47.5 0.9 48 44 4.0 3.1 W 5 HOUSE, NY-26 47.0 46 1.0 52 48 4.0 3.0 R 6 SEN, VA 43.7 44.8 1.1 49 50 1.0 0.1 R 7 SEN, TN 49.3 48.1 1.2 51 48 3.0 1.8 R 8 GOV, IA 46.4 47.6 1.2 45 54 9.0 7.8 R 9 SEN, MO 47.3 48.6 1.3 47 49 2.0 0.7 R 10 GOV, MN 44.7 46.1 1.4 47 46 1.0 0.4 W 11 SEN, OH 47.3 48.7 1.4 44 56 12.0 10.6 R 12 HOUSE, IN-9 46.0 47.5 1.5 46 49 3.0 1.5 R 13 SEN, MT 45.5 47.4 1.9 49 49 0.0 1.9 R 14 HOUSE, KS-2 44.2 46.7 2.5 47 51 4.0 1.5 R 15 GOV, AR 48.2 45.3 2.9 41 55 14.0 11.1 W 16 HOUSE, KY-4 41.9 44.9 3.0 52 44 8.0 5.0 W 17 HOUSE, NY-25 45.0 48.6 3.6 51 49 2.0 1.6 W 18 SEN, WA 46.1 50 3.9 39 58 19.0 15.1 R 19 HOUSE, CT-2 46.6 42.3 4.3 50 50 0.0 4.3 R 20 HOUSE, NC-11 43.3 47.6 4.3 46 54 8.0 3.7 R 21 SEN, MD 44.3 48.8 4.5 44 54 10.0 5.5 R 22 SEN, FL 49.9 45.1 4.8 53 45 8.0 3.2 R 23 HOUSE, PA-6 43.5 48.6 5.1 51 49 2.0 3.1 W 24 GOV, MD 43.9 49.3 5.4 46 53 7.0 1.6 R 25 SEN, NJ 42.8 48.7 5.9 45 53 8.0 2.1 R 26 HOUSE, NY-24 39.3 45.6 6.3 45 54 9.0 2.7 R 27 GOV, IL 33.0 39.6 6.6 40 49 9.0 2.4 R 28 HOUSE, IA-1 41.5 48.6 7.1 43 54 11.0 3.9 R 29 HOUSE, CT-4 43.5 50.7 7.2 50 48 2.0 5.2 W 30 SEN, MI 41.9 49.3 7.4 41 57 16.0 8.6 R 31 SEN, AZ 51.8 44.4 7.4 53 44 9.0 1.6 R 32 GOV, OH 43.3 50.8 7.5 37 60 23.0 15.5 R 33 GOV, AZ 43.7 51.3 7.6 35 63 28.0 20.4 R 34 HOUSE, VA-2 51.0 43.4 7.6 51 49 2.0 5.6 R 35 SEN, MN 42.8 50.5 7.7 38 58 20.0 12.3 R 36 GOV, CA 47.6 38.8 8.8 56 39 17.0 8.2 R 37 GOV, MI 42.7 51.6 8.9 42 56 14.0 5.1 R 38 SEN, PA 43.8 52.7 8.9 41 59 18.0 9.1 R 39 SEN, WI 38.0 47.2 9.2 67 30 37.0 27.8 R 40 HOUSE, NM-1 43.9 53.2 9.3 50 50 0.0 9.3 W 41 HOUSE, FL-22 39.6 49.2 9.6 51 47 4.0 5.6 R 42 HOUSE, VA-5 46.0 36.4 9.6 59 40 19.0 9.4 R 43 HOUSE, MN-6 52.2 42.1 10.1 50 42 8.0 2.1 R 44 HOUSE, AZ-8 41.3 52.5 11.2 42 54 12.0 0.8 R 45 GOV, GA 48.3 36.3 12.0 58 38 20.0 8.0 R 46 GOV, CT 53.1 40.5 12.6 63 45 18.0 5.4 R 47 HOUSE, IN-2 39.0 51.9 12.9 46 54 8.0 4.9 R 48 GOV, PA 42.0 55.2 13.2 40 60 20.0 6.8 R 49 GOV, TN 40.5 54.3 13.8 30 69 39.0 25.2 R 50 HOUSE, NY-20 51.2 37.1 14.1 47 53 6.0 8.1 W 51 HOUSE, IL-6 40.0 54.2 14.2 49 51 2.0 12.2 R 52 HOUSE, CO-7 39.8 54.1 14.3 42 51 9.0 5.3 R 53 SEN, RI 38.8 53.4 14.6 47 53 6.0 8.6 R 54 SEN, TX 54.8 35.9 18.9 62 36 26.0 7.1 R 55 HOUSE, OH-18 33.1 53.2 20.1 38 62 24.0 3.9 R 56 SEN, NM 37.5 58.2 20.7 30 70 40.0 19.3 R 57 SEN, NV 58.3 36.8 21.5 55 41 14.0 7.5 R 58 SEN, NY 33.0 57.3 24.3 31 67 36.0 11.7 R 59 GOV, MA 32.7 58.1 25.4 35 56 21.0 4.4 R 60 SEN, MA 32.4 61.8 29.4 31 69 38.0 8.6 R 61 GOV, NY 27.3 63.2 35.9 29 69 40.0 4.1 R

Here, look at the races that Zogby found close. Particularly, look at where Zogby had the race closer than 4 points. You'll find that they called the wrong winner a whopping 39% of the time (7/18 races).

Don't let the pollsters get you down. This thing is winnable!

TOPICS: Politics/Elections
KEYWORDS: 2008polls; mccain; obama
first 1-2021-22 next last
I still have the spreadsheet if any stats wienies want to make a point...

Otherwise, as always, feel free to revise and extend my remarks.

1 posted on 10/21/2008 11:58:01 AM PDT by PhilosopherStones

To: PhilosopherStones

Great work! Thanks for taking the time to sort out the data. This is very helpful for the poll junkies.

2 posted on 10/21/2008 12:01:10 PM PDT by TonyInOhio (This is no time to go wobbly.)

To: PhilosopherStones

election poles ping

3 posted on 10/21/2008 12:05:08 PM PDT by gleeaikin

To: PhilosopherStones

election poles ping

4 posted on 10/21/2008 12:05:16 PM PDT by gleeaikin

To: PhilosopherStones
Very interesting analysis. I just have a couple of formatting suggestions. It would probably be clearer if you didn't use "R" for "correct" -- perhaps use a "C".

Also, if you used subscripts to distinguish between the "predicted" and "actual" results. E.g. Rp and Ra.

5 posted on 10/21/2008 12:05:46 PM PDT by USFRIENDINVICTORIA

To: PhilosopherStones

Just scanning the numbers, it seems the overriding trend with Z is that he gives more of an an advantage to the Republicans than the actual vote reflected—which is not encouraging.

6 posted on 10/21/2008 12:09:53 PM PDT by TruthHound (You can keep the "change"!)

To: USFRIENDINVICTORIA

Too late now. Next time I’ll FReepmail you for suggestions before I post any formatted text.

7 posted on 10/21/2008 12:10:15 PM PDT by PhilosopherStones

To: PhilosopherStones

And Zogby under-estimates Republicans by an average of 4 pts. Surprise! Surprise!

8 posted on 10/21/2008 12:10:20 PM PDT by Thane_Banquo (I'm not against 0bama because he's black. I'm against 0bama because he's red.)

To: PhilosopherStones
Over at "http://www.electoral-vote.com/" they have the EV at Obama 364, McCain 171 and Ties 3.

Anyone know just how accurate this site is? It seems way out-of-wack to me.

9 posted on 10/21/2008 12:14:23 PM PDT by softwarecreator

To: PhilosopherStones

Very nice!

I’ve been hoping to come across a time series of 2008 Presidential poll data that includes error bands - let’s just be generous and say 2 sigma, although I’d really like to see 3 sigma. My gut feel is there would be no significant trend.

10 posted on 10/21/2008 12:15:03 PM PDT by bigbob

To: USFRIENDINVICTORIA
It's (R)ight vs. (W)rong on the Correctness column.
11 posted on 10/21/2008 12:15:11 PM PDT by 5thGenTexan

To: TruthHound

Looks like just the opposite. So it’s good news.

12 posted on 10/21/2008 12:18:30 PM PDT by scratcher (I trust the media about as much as I trust al Qaeda.)

To: PhilosopherStones
Feel free -- although those are the only things I could think of. It always seems that the "best" format only reveals itself after publication.
13 posted on 10/21/2008 12:19:13 PM PDT by USFRIENDINVICTORIA

To: 5thGenTexan
My point was that the (R)ight is too easy to confuse with (R)epublican. According to Murphy's law, some people will be confused, whenever they can be.
14 posted on 10/21/2008 12:22:46 PM PDT by USFRIENDINVICTORIA

To: scratcher

Ummm, by my count (maybe I’m not reading the chart right), the races where Z gave the Republican MORE percentage in his projection than the actual vote were 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 15, 16, 17, 20, 22, 23, 24, 25, 27, 30, 34...need I go on?

15 posted on 10/21/2008 12:30:49 PM PDT by TruthHound (You can keep the "change"!)

To: PhilosopherStones
Once again I'll point out that the MOE is almost always misinterpreted by the media. If a poll has an MOE of 4 points, that means that both the democrat and gop estimates are withing 4 points of the polled number 95% of the time. That means that the *difference* between the two numbers can be up to 8 points off. Under those circumstances, a candidate would have to be up by 9 full points to be "outside the margin of error".

I even see Rasmussen himself in interviews misstating this simple fact. I've seen him say things like "Obama is up 5 which is outside our margin of error of 3". Nope, he would have to be up by 7 to be outside the margin of error of 3, because MOE does not measure differences between the numbers, but rather applies to EACH of the candidates's numbers.

So, when you point out that you have to go down to column 39 to see actual results within the MOE of 4, what you should really be saying is that you have go down to column 21 before you find results that exceed the combined candidate MOE of 8.

Still, your point is well taken, 21 out of 61 races were outside the doubled MOE. In theory only 5% should have been outside that figure not 33%.

This can easily be explained though, by the second of the great polling fallacies. MOE only measures "statistical" error. That is error due solely to random sampling chance. There are many other kinds of errors that occur in polls and they are additive to MOE. Those range from 1 to 6 percent *in addition to MOE*, typically 2-4% when studied on major pollsters, far more on less reliable pollsters.

Applying these other sources of error, we find that in this set of polls Zogby had really a fully loaded error of 9.5 points per candidate, or 19 points overall. Only with that figure do we get to the 95% confidence level (95% of the races listed fell within that level of error).

In this set of polls, then, a candidate had to be up by 20 points to be "outside the margin of all errors, both MOE and pollster induced errors".

16 posted on 10/21/2008 12:32:34 PM PDT by drangundsturm

To: drangundsturm

I knew someone would correct me on that and I really didn’t feel like doing the math. You are also correct that the results STILL don’t square with a 95% confidence level and their 4% confidence interval, even if I fudged a bit.

Table 2 is clearer in what it shows.

17 posted on 10/21/2008 12:39:16 PM PDT by PhilosopherStones

To: TruthHound

Yea, something doesn’t seem right to me either. I wonder if columns 2 and 3 are the actual results.

18 posted on 10/21/2008 12:50:54 PM PDT by hudsonohio

To: PhilosopherStones

In the very first race you have the Republican winning in Wisconsin.

19 posted on 10/21/2008 12:53:43 PM PDT by NavVet ( If you don't defend Conservatism in the Primaries, you won't have it to defend in November)

To: NavVet
In the very first race you have the Republican winning in Wisconsin.

I'm not a "stats weenie" but I noticed that one too. In the first row, either one set of numbers are reversed or the "R" should be a "W."

20 posted on 10/21/2008 1:19:56 PM PDT by Tarantulas ( Illegal immigration - the trojan horse that's treated like a sacred cow)