Posted on 09/05/2004 4:01:55 AM PDT by dvwjr
Analysis of recent Newsweek/PSRAI post-RNC convention polling data, with derived and revised Kerry and Bush post-convention 'bounces' and poll leads. Comments welcome...
Excellent job and clear presentation. Thanks.
The figures were heavily skewed towards Republican, so I skewed them, roughly, to registration norms for Dems, Indies, and Reps. (Roughly: I didn't have exact data, so I used 35% Rep, 35% Dem, and 30% INdy.) After doing that, I found Bush leading 51.3-42.1, a 9.2% lead, reasonably close to the 11% reported. Since Bush's lead grew after he spoke, I feel comfortable that his lead is about 11%. Zogby was showing Bush down 5 while others had him up 2, (in other words, 7 points lean towards Kerry), so Iexpect Zog will show Bush about +4.
Not sure about Rasmussen, other than they're very shy about reporting shifts.
Dear dvwjr,
Nice job.
Two major problems, though.
The first is your attempt to correct the data for a particular ratio of R/D/I.
"Note the big dip in Republicans during the Democrat Convention."
"Now look at September, it swings the other way, now too few Dems and too many Republicans."
That is as it should be.
Party affiliation is not a static, unchanging demographic attribute like sex, race, or even religion (which can change, but not readily).
One ought to expect that during the Democrat convention, some number of people who would not ordinarily identify themselves as Democrats will so identify themselves as such. And vice versa for the Republican convention.
If you use R/D/I ratios from, say, the last election, you will distort the real outcome at the end of each convention, because some people who on THIS election day will identify as Republicans or Independents will identify as Democrats in the immediate aftermath of the Democrat convention, and conversely for the Republican convention.
It appears that Newsweek took that into account.
The second major problem (and not unrelated to the first) with your analysis is that it only corrects for party affiliation. How about for minority status? Or geographical location? Or sex? It may be that the pollsters tried to factor in these demographic attributes, as well.
Otherwise, nice piece of work.
sitetest
Are these results for registered voters? Shouldn't we add a few points onto the registered voters numbers, because the likely voter numbers always end up favoring the Republicans by a few points over the registered voters.
Isn't the sample of military households somewhat large in the sample as well?
Please add me to your ping list.
I can understand the concern about over sampling by some pollsters as a hidden attempt to provide a basis for a future claim that Bush's lead is eroding, but isn't that a bit of a risky propostion?
1. Time and Newsweek risk being over shadowed by more reputable polling. Especially if other polls settle in at a 7-8% lead.
2. They risk discouraging Kerry's base as there are some who will throw their hands in the air and simply give up.
I wouldn't put anything past these rags, just don't know how effective this is.
I really like the analysis you do, please add me to the ping list.
When the polls that fully take the Bush speech into account are released we may see an increase in the Bush bounce, and then it will fade. IMO the Bush lead could be 5-6% in a couple of weeks. 53-47 sounds real nice.
Thanks for the ping. I'm ewnjoying the post.
bookmark bump
Great work.
Excellent work! It's great to have someone with the ability to untangle poll internals willing to enlighten us. Thanks!
If you don't already have a blog, you should seriously consider starting one.
You got that right! The last time I checked Alexa, internet traffic listed FreeRepublic at #2559. DU came in at #7432.
Since the RATs did so well getting their vote out in 2000, I usually give them a 3% edge in weighting -- approximately 37% to 34% with Indies making up the rest. I doubt the RATs will do that well this time around, so if anything, I overstate their support.
Even with this weighting, W still holds a solid lead in this poll.
There is a valid argument that can be made (as several people have on this thread) that voter self-identification of party status can shift around conventions, especially for undecided or marginally-independent voters. That is one of the several weaknesses of the survey techniques by nationwide polling organizations. They have to ask the respondents what political party they belong to and how likely they are to vote (or infer the latter from other questions). Then the pollsters must apply some adjustment factors, and the uncertainties and varying methodologies in each of their adjustments can produce sharply different results (and biases).
That's why I much prefer starting with a database of registered voters when conducting polls. There's no self-identification problem; you know exactly how the respondent is registered. And you can apply an objective standard as to whether the respondent is a "likely" voter based on previous voting history.
That's probably much easier within individual states (I'm from California) than on a nationwide basis. In some states that information is difficult or impossible to obtain. Also, registration data varies in quality from state to state (some states are far better than others in cleaning up and updating their registration databases).
The main thing we can glean from nationwide polls is not the absolute numbers (i.e., which candidate is ahead and by how much) but the relative trends. Both the Time and Newsweek polls show a strong post-convention bounce and swing towards Bush, whereas Kerry got a small or non-existent convention bounce.
That's a tough thing for the Democrats to explain -- why the "small number of undecideds" which prevented Kerry from bouncing created no similar impediment to Bush.
I used the Newsweek 34.4%(R), 35.7%(D), 30.0%(I) for re-weighting only because it was an actual weighting used by Newsweek/PSRAI. My personal weighting is a combination of the 2000 VNS post-election polling on the high end, and the Gallup and Pew Research data on the low end. The post-election statistics will be nice to see how the actual voter R/D/I splits work out...
dvwjr
"Just my take..."
LOL
I'd like to be on your ping list, sir.
I think you need to correct the above line. You compared the unadjusted July 29-30 numbers with the adjusted Sept 2-3 numbers. If you use as I assume you intended a comparison of the adjusted July 29-30 numbers to the adjusted Sept 2-3 numbers, the Bush convention bounce is 8.95%.
Great work in any event.
There are two major issues, when weighting. Is the sample frame a random sample, or , were certain groups over sampled, in order to get enough respondents in each group, so that they can be compared?
If you take a random sample, people who refuse to be interviewed can skew the results. This is only a problem, if the people who refuse are significantly different from the people who agreed to be surveyed.
For example, if young people refuse at a higher rate than older people, the younger respondents need to be weighted up to compensate. If younger and older people refuse at the same rate, you don't need to weight the results.
These decisions are generally based on known demographic data. It's not a perfect system, but it does a pretty good job. The key is to reduce the refusal rate, by putting your best interviewers on 'refusal conversions', etc., and get the refusal rates down.
If you over sample certain groups, when looking at the sample universe in total, the groups need to be weighted back to the population. If you randomly survey a population that would result in a very small sample size of Group A and a large sample size of Group B, you have to over sample Group A, in order to get a large enough sample of Group A to be able to compare it to Group B. When looking at the total, you weight Group A down and Group A up.
If these latest polls appear to be over sampling Republicans, there could be an innocent explanation.
Say that you take a random sample. You then ask party affiliation and how likely the respondent is to vote. If you are surveying likely voters, respondents not likely to vote are not eligible.
More Republicans than Democrats could get into this survey, based on how likely these groups are to vote. I wouldn't find this odd, during and immediately after the Republican convention.
That being said, the people that pay for the poll have control over everything. So, keep that in mind, when looking at the results. Pollsters are independent in the same way that 'independent auditors' that audit the public companies that pay for the audit are independent. Arther Anderson comes to mind.
Polls work. People would stop paying me to conduct them, if they didn't work.
An over sampling of one party over the other, in a poll of likely voters, doesn't raise a red flag for me. A poll paid for by Time or Newsweek is a bigger concern to me.
I watch the reactions of the politicians. Kerry's recent gyrations lead me to believe the President Bush got a pretty big bounce out of the convention. And, that's a good thing.
Great post, even I can understand.
This oversampling of Republicans will allow
Newsweak to portray Kerry on their cover
in about 2 weeks as the
"New Comeback Kid".
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.