Replies

From 2004 from polipundit:

Zogby: Back in 1996, pollster Zogby hit the bullseye in predicting the results of the Presidential election. In 2000, they were close again, though their aggregate error tied them with 5 other national polls. In 2002, Zogby appeared to show a lean in favor of the Democrats, and he was way off in his mid-term election predictions. This year, at the end of the spring, John Zogby actually came out and predicted John Kerry would win the election, which appeared to indicate his bias had reached the point of full-blown partisanship against the President, reflected in a growing number of opinions made out of personal preference, rather than on the evidence. Zogby’s refusal to show his work, only magnifies the apparent distortion of his results.

Zogby runs two polls; a telephone poll and an Interactive Internet poll. Unlike almost every other poll, Zogby’s telephone poll is not RDD. Zogby describes his list as follows: “The majority of telephone lists for polls and surveys are produced in the IT department at Zogby International. Vendor-supplied lists are used for regions with complicated specifications, e.g., some Congressional Districts. Customer-supplied lists are used for special projects like customer satisfaction surveys and organization membership surveys.
Telephone lists generated in our IT department are called from the 2002 version of a nationally published set of phone CDs of listed households, ordered by telephone number. Residential (or business) addresses are selected and then coded by region, where applicable. An appropriate replicate1 is generated from this parent list, applying the replicate algorithm repeatedly with a very large parent list, e.g., all of the US.
Acquired lists are tested for duplicates, coded for region, tested for regional coverage, and ordered by telephone, as needed.” Zogby notes that “regional quotas are employed to ensure adequate coverage nationwide.” That is, Zogby takes pains to insure that his respondent poll is not random.

As for his weighting, Zogby states “Reported frequencies and crosstabs are weighted using the appropriate demographic profile to provide a sample that best represents the targeted population from which the sample is drawn from. The proportions comprising the demographic profile are compiled from historical exit poll data, census data, and from Zogby International survey data.”

In other words, Zogby uses his own polls to drive some of his demographic parameters, a practice not approved, much less recommended, by either the NCPP or the AAPOR.

All in all, Zogby’s habit of confusing his personal opinion with data-driven conclusions, his admitted practice of manipulating the respondent pool and his demographic weights, by standards not accepted anywhere else, along with mixing Internet polls with telephone interview results, forces me to reject his polls as unacceptable; they simply cannot be verified, and I strongly warn the reader that there is no established benchmark for the Zogby reports, even using previous Zogby polls, because he has changed his practices from his own history.