Posted on 05/11/2006 10:08:12 AM PDT by RWR8189
Thomas Riehle and Lance Tarrance offered a thought-provoking piece last week discussing the generic congressional ballot, i.e. the question in media polls that asks which party respondents will prefer in the congressional election.
They write:
The new Cook Political Report/RT Strategies national survey...shows Democrats leading Republicans by 12 points among registered voters...the Democratic advantage in these generic ballot tests is into double digits, beyond the reach of the normal overstatement of Democratic prospects that critics correctly cite as an historic fact when looking at either the partisan Congressional control question or the generic ballot test.It is true that critics (such as myself) cite the question's consistent Democratic skew. What Riehle and Tarrance do not mention is that, to use the poll as a predictive tool, one cannot simply correct this bias. Forecasting elections is not like golf -- where two players of different skill can compete once they have assigned handicaps.
The predictive value of a poll is based upon how it changes with the result it endeavors to predict. A poll that, on average, skews 7% in favor of the Democrats would still be valuable if it changes as the balance of power changes. In this instance, the skew would be like a thumb on an otherwise accurate scale. The final number is always wrong, but changes in the scale's estimate correspond with changes in the weight put on the scale. Riehle and Tarrance imply that this is the case - that we can correct for the Democratic thumb on the generic scale, and still read a distinct Democratic weight. Unfortunately, the generic ballot is more like a scale that is just plain broke.
At least it is this far from Election Day. Consider the history of Gallup's spring generic question (registration required), particularly the extent to which changes in it have preceded changes in the partisan composition of the House. This is the coefficient of determination, a value that tells us the percent of change in seats that can be anticipated by a change in Gallup's spring generic ballot. To return to the scale metaphor, a good scale with a skew would still have a high coefficient of determination - an increase in the weight put upon it would correspond almost perfectly to an increase in what the scale reports. A broken scale - one that offered results unrelated to the weight placed upon it, and whose average was still skewed in one direction - would have a low coefficient of determination.
For midterm congressional elections since 1958, the coefficient of determination of the spring question for the Democratic share of the House is about .335. In other words, changes in the spring generic ballot anticipate only about 33.5% of the change in the balance of power.
Why is this statistic such a poor indicator? The reason is that increases in the Democratic advantage barely correlate with increases in the Democratic share of seats. For instance, Gallup's spring 1958 poll gave the Democrats an 8% advantage; they would eventually enjoy a 153-seat majority in the next Congress. The spring 1966 poll gave them a 16% advantage; they would eventually enjoy a 60-seat majority. The Democrats doubled their lead in the poll, but lost nearly 100 seats. Another good example: in both 1974 and 1978, the Democrats had a lead of 26% this far from Election Day - in 1974 they won a 147-seat majority; in 1978 they won a 119-seat majority.
This low coefficient of determination is problematic in itself. You would not want to rely on the spring generic ballot too much in making your predictions! However, there are two additional problems, more theoretical in nature, that make me all the more wary. First is its inconsistent predictive power. If we consider a subset of the time-span in question - specifically 1958 to 1988, inclusive - the coefficient of determination drops to a paltry 2.25%. This is a sign that the statistic has more problems than we initially thought. When we find that an indicator performs unevenly, and we have no causal explanation for this unevenness, we should be skeptical of making use of it. This is a sign that chance, not causality, might be causing the relationship that we have discovered - and, if that is the case, there is no reason to believe that the spring generic ballot will be useful for future predictions.
The second concern is the high expectation we are inclined to place on the generic ballot. The spring ballot performs about the same when it is used to predict each party's share of the House vote (rather than share of House seats). This result is very unimpressive in light of the fact that the generic question claims to be a direct test of what will eventually occur on Election Day. The generic ballot does not claim to be one of many causal factors in the congressional elections (as we might take presidential job approval or candidate spending) - it claims to be a measure of the election results themselves. The fact that it does such a poor job predicting what we expect it to predict so well is another reason for concern. It is not just that it explains such a low amount of variation - it is that it explains an amount of variation that is so much lower than expected. We should be theoretically surprised by this result, and therefore skeptical of using the measure as a predictive tool, even to the limited extent the existing data allows.
In the mid-90s, several political scientists attempted to make use of the spring statistic by arguing that, when taken in the context of who is in control of the White House, its predictive value increases. However, these attempts were ultimately thwarted by the history defying 1998 and 2002 contests - and they were theoretically and statistically problematic to begin with. All in all, my sense is that the spring generic ballot adds more confusion than clarity to understanding this midterm. It is best left aside.
This might seem counter-intuitive. Should this not be an effective early indicator of what will happen? I do not think so. This is a candidate-centered age of politics. For the political elite, the party comes first; partisanship acts as a frame of reference and a cognitive screen, enabling them to understand the political world and evaluate new data. However, average voters do not think in the same manner - not because they are less intelligent, but because they possess too little political information. Partisanship is still very important to the average voter, but it is not like a religious or regional affiliation. Average voters can and do defect to the other party, and they almost always defect to the party of their incumbent. This is a consequence of the new political age in which we live. This is not the 1880s; parties do not exercise decisive control over who wins the nomination, what issues they promote, what they do in government, etc. Today, candidates are free of their party. Is it a surprise that the psychological power of partisanship has declined? Is it a surprise that voters often eschew their party identification to pull the lever for their incumbents, who spend millions to remind them of their independence?
If what will be decisive this fall is evaluations of candidates, not abstract expressions of partisanship, what good is today's generic ballot? The average voter has not yet put a name to the label "Republican" or "Democrat." When s/he thinks about 2006, s/he is not yet thinking about the candidates. When we enter the fall, it will be time to think more carefully about the generic congressional ballot - as that will be the time the average voter puts names to the labels.
The value pollsters inappropriately assign to today's generic ballot indicates why caution is prudent when approaching media polls. The mistake with the generic ballot is not exceptional. Too often, pollsters and pundits misinterpret the significance of polling results.
Consider again the spring generic ballot. It endeavors to predict the partisan division of election returns: if the Democrats poll 51% on the generic, they will win 51% of the vote. Behind this endeavor, however, lies the aforementioned false premise: the average voter, like the political elite, is actively thinking about congressional elections today along partisan lines. This is not true, and so the poll does not mean what its interpreters think it means. Similarly, consider congressional job approval. Its predictive value is based upon the premise that the average voter thinks about his representative in the context of Congress as an institution. Almost always, this is not true.
There are many interpretations of many questions that can be criticized along the same lines. Everyday and everywhere, one sees columns and articles that tout the electoral importance of this or that question, without any inquiry into whether the claim is justified. The fact that we know so many claims to be incorrect is why we should not give these polls the benefit of the doubt. I am not suggesting that we doubt the results. I am suggesting that we should doubt the electoral significance of the results. It is an issue of value - i.e. pollsters and pundits are too quick to assign a value to their results that is unwarranted.
We should ask media polls to meet a burden of proof prior to our use of them. The burden is a positive answer to the following query: does the poll have a history of predictive accuracy? If a question has electoral relevance, over time the poll will be a good predictor of electoral outcomes. Thus, we must surmise its historical accuracy before we use it as an analytical tool. If there is no history, or if the history is too short, the question must seem obviously well tailored to elicit the desired information from the average voter. In other words, it cannot be a question that presumes that average voters think in the ways that political elites think; it must, rather, be thoroughly grounded in what we know about how average voters think during the election season and how they act at the ballot box.
This is why I take presidential job approval seriously. It corresponds well to changes in the balance of power. This is why I do not take congressional job approval seriously. It corresponds poorly to changes in the balance of power. This is also why I ignore questions with a short history, unless they seem to obviously cohere with what I know about the electorate's knowledge base and thinking processes. So, for instance, the "whom would you rather have in control of Congress" question is one that I ignore. It is all well and good that the public responds however it responds; but I know that, when it comes time to pull the lever, the average voter is not thinking about Congress as an institution. So, of what value is this question? Absent a history of accurate predictions, it is only prudent to assume that it is of little value.
Ultimately, the spring generic congressional ballot fails to meet this burden. Despite whatever intuitive appeal it seems to have, the fact is that it poorly correlates with electoral outcomes; further, a closer inspection of 21st century American politics indicates that this should not be a surprise. When we get to Labor Day, it will be time to start taking this poll seriously. However, as long as we can still wear white - we can safely ignore it!
"Which do you prefer, Joe Blandguy, or someone who says you can have free health care, free school, free food, and free housing; who is for easy bankruptcy laws so you can walk away from your debts, who is for letting you file a lawsuit for huge amounts of money against anyone who annoys you, and who is for getting some rich guy to pay for all this... who do you like more in next November's election?"
generic poll analysis ping...
95% of the people polled live in areas where I can guarantee you today who will be their representative after the 2006 election. I don't need no stink'n poll.
Very informative. I have an idea that would take the weight factor away and be a bit more fair.
Take 1500 people divide 52% that voted for Bush and the 48% that voted for Kerry and ask 3 questions with the choice of one, 1) More often than not approve of Bush, 2) about the same, 3) more often than not disapprove of Bush. Take the "about the same" and divide equally amongst the other two questions. This method would eliminate all party affiliation and take a percentage based only on an elected result.
Your suggestion would seem to make sense. In any event, there are a couple lifetimes until voting time so any poll means little right now....
Yep. Times have changed though and they no longer say what is, is. Thankfully...
Yep. My rep is unopposed in the primary in a largely partisan district. He might as well have his name engraved on his congressional seat. You have to pull a Duke Cunningham or die to lose an incumbant seat in California. On second thought, Sonny Bono DID die, and I think his wife still holds his seat.
Absolutely. The Democrats had double digit leads (as big as 19 points) in the generic congressional ballot in May and June of 2004. The tide turned significantly by November. Not saying that will necessarily happen again in '06, but it is way too early to put much stock in the generic polling.
This is the nut, and this is why the GOP will gain one seat at least in the Senate and one to five in the House.
End of story.
We could add, however, that "committed voters may stay home, but they never defect to the opposite party."
This translates to a one seat gain (min.) in the Senate and a one to five seat gain in the House.
I love your optimism and do not share the Chicken Little mindset held by many. Indeed, Santorum was written off and has closed the gap. That Mellman is not in the news much means he is busy doing his job. GOTV is the key and there is plenty of reasons for the GOP to do so....
Ping for later. Cost is a never-miss read.
1) DeVos is running ahead of Granholm in Mich. This means the weak-running Stabenow is also vulnerable now.
2) Don Goldwater looks to be running even with Napolitano for the Az Gov. This won't help the Dem House candidates.
3) I'ves seen two columns, one by Jeff Greenfield, in the last week that are starting to provide excuses for the Dems' defeat. Greenfield was explaining how "difficult" it was to beat incumbents. Ruh-roh!
4) Both Kean and Kennedy are either ahead or slightly behind depending on the poll for the senate seats in NJ and MN.
5) Does anyone seriously think Connie Burns will lose Montana? Anyone? Bueller? Bueller?
Good morning. Yep. Nice recap. The presstitutes' and Rats' meme about taking back control is largely to raise $$, IMHO. The alternative message is "throw some more money away". That said, the GOP has to fight and vote like the country is at stake, because it is....
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.