You touched on it with respect to Gallup, but one thing that's always bugged me is that the only way to judge a polling organization's accuracy is to compare its final survey to the actual results. I don't think there's any way that swings anything like as big as those shown by Gallup (among others) actually took place, but there's no way to know. I'm sure it has crossed the minds of many that some of the wild fluctuations were agenda-driven.
At least Gallup was able to identify a shift in party identification toward the Republicans after the GOP convention. Zogby insisted on weighting his polls to a 3-point or so demo advantage, since that was how voters in 2000 identified themselves. The problem was that on voting day, there were actually 5% (if my memory serves me) more Republicans voting than democrats. Gallup does not weight for party ID, which is what enabled them to notice this sea change.
Thanks.
>>but one thing that's always bugged me is that the only way to judge a polling organization's accuracy is to compare its final survey to the actual results.<<
Your're right. First, I crunched the numbers based strictly on the final poll. Only later, did I look at what happened earlier in the races, adding that in after the fact.
The truth is ARG and Zogby were abominable all campaign season, often showing wild Kerry leads. But I had no way to objectively compare such things. For instance, I made myself forget that Zogby at one point predicted Kerry was running away with Tennessee; Bush beat him by ten points. I also ignored his last-day prediction that Virginia was too close to call, because it wasn't in his data.