Jay Cost: Poll Watching and Congressional Elections

One of the most enjoyable and exciting parts of following a presidential campaign is the relative ease with which you can figure out the state of the race. It all comes down to the head-to-head polls. Last October many close observers knew the day of the week and the time of the day that the important polls came out in the Kerry v. Bush race. It was a real treat to be able to get new information that was reliable and accurate.

Unfortunately, in congressional elections, such data is harder to come by. One must be much more wary about state polls. Many polling outfits that do state work are reputable, many others are not. But, even more important is the fact that there is a lot of polling data out there that seems like it provides a reliable gauge of the state of the congressional races, but really does not.

This does not have anything to do with good or bad polling firms or techniques. It has to do with the conclusions that we draw from polling data. Specifically, we can very easily run into what is called the ecological fallacy. Simply stated, the ecological fallacy rears its ugly head every time we try to draw inferences about parts of a whole based only on data about the whole. Take a simple example. Suppose you want to find out if the individuals at your church are giving more or less. Your pastor tells you that the revenues this year are exactly the same as last year. Well, you conclude, that settles it. Everybody has given the same amount. Can you conclude this? No, you cannot. Everybody could just as easily have altered their giving patterns very radically – and these changes simply cancelled each other out in the aggregate number. Indeed, there might be reason to suspect that this is the case if there has been some kind of theological or social controversy in the church in the last year – one in which some subset of the church was made more happy, and another made less happy. Thus, if you want to know how the members of the church are acting, and if you want to get a more sophisticated sense of how the church as a whole is acting, you have to go and ask them individually.

So it goes with analysis of congressional elections. It is very dicey for exactly the same reason. If you are trying to make an argument about how many extra seats the Democrats are going to win this year, you are really making an argument about how a subset of the population will act. Can you make such an argument based upon data on the whole population? Not really. Not directly, at any rate. You cannot look at the current poll results about which party all people would prefer to have representing them in Congress and draw any direct inferences about what that means for any part of the public. The reason for this is that everything boils down to the distribution of the public: who is more likely inclined to have a Democrat or a Republican representing them, are they usually supporters of Republicans, are they in the right place and in the right quantity to change members of Congress, etc. You cannot answer these (necessary) questions from national survey data.

As a practical matter, what this means is that if you have some question that a reputable polling firm asked of the entire nation about how they will vote in next year’s election, how they feel about Congress, whether they like their incumbent, whether they hate the Republicans, or anything of the kind – you cannot conclude, based upon that question, anything about what will happen. The reason for this is that Congress is elected by subsets of the nation at large, and changes in the composition of Congress really only depend upon those subsets. Theoretically, a party could lose all of its seats with only small changes in the national mood, or it could lose no seats even with large changes. That is why you cannot rely on a barometer of the national mood to tell you what will happen. You cannot even rely on dozens of barometers.

There is, nevertheless, an indirect way you can draw inferences from national barometers. You could ask the same question at roughly the same time every year and see whether or not changes in how people answer the question correspond to changes in the composition in Congress. For instance, if you happen to notice that increases in the number of people who say “Yes” to the question, “Would you like the Republicans to be in control of Congress?” tend to correspond with increases in the number of Republican congressmen, you could indeed conclude that this question is a good indicator of how many seats will change hands. This is a possible shortcut around the ecological fallacy.

There are two real problems with this type of shortcut, however. The first is that you almost always should consider one poll’s question independent of another poll’s question. Most polling firms that do work on congressional elections ask roughly similar questions. But all of the polling outfits have their own particular methods and theories that inform those methods. This means that you really need to gauge Gallup’s particular question separately from, say, Newsweek’s.

The other problem is even bigger. To determine whether or not a certain question gives you a good sense of what will happen on Election Day, you really have to spend a lot of time watching how the question performs. The reason for this is because we are looking for a strong correlation – how does a change in the results of a question correspond with changes in the balance of power in Congress? So, as you decrease the number of observations of a question’s performance, you are going to find it more and more difficult to determine whether there is a real correlation between the two, and therefore whether the question can be a good predictor of congressional seat changes.

Because a question has to be asked during a good number of election cycles, the only polling firm with the possibility of offering congressional analysts with enough data is Gallup. [Note that this is certainly not to say that other polls are worthless. They are all intrinsically valuable insofar as they are taking a pulse of the nation and we are interested in what the nation collectively thinks. It is only to say that we are very limited with what we can confidently use to make predictions about how they will act.] Gallup has been asking two congressional questions for a sufficient length of time that we can investigate whether they are good predictors of what will happen in next year congressional elections.

The first is what is known as “the generic congressional ballot.” This was an innovation in gauging public opinion of Congress that was designed by George Gallup himself. The question reads: “If the elections for Congress were being held today, which party's candidate would you vote for in your Congressional district -- [ROTATE: 1) The Democratic Party's candidate or 2) The Republican Party's candidate]?”

This question has real value to it – it enables us to draw some tentative conclusions about what will happen across districts even though it is a national question. In particular, the result of this question, when asked immediately before the election, does an excellent job of predicting which party will get which percent of the vote. The limitation of this, of course, is that just because a party increases its share of the vote does not necessarily imply how many extra seats it will pick up. Thus, for instance, the Democrats lost about 2.1% of the total congressional vote in 1956 (which is to say that they did that much worse in 1956 than in 1954), but they did not lose a single seat. Nevertheless, the final Gallup generic question does help us because, while vote changes and seat changes are not perfectly correlated, they are indeed strongly correlated. So if we find that on Monday, November 6, 2006, Gallup’s generic results show the Democrats have a 10% edge in the generic ballot, you had better be prepared for some big changes the next day.

The problem with the generic question, or rather the problem for those who are trying to draw inferences from the question, is that the further you are from election day, the less reliable a predictor it is. As it turns out, the generic question tends to skew systematically toward the Democrats. A year before the election the Gallup generic question tends to allocate about 6% more points than it should to the Democrats. This is not a criticism of Gallup, mind you. It is probably a function of the “mind” of the public. They are more inclined to vote Democratic a year out, or maybe Democrat-leaners are more enthusiastic about voting a year out than they are on Election Day, and Gallup is picking up that trend.

Knowing how the Gallup generic question tends to work enables us to draw a reasonable inference about the current state of the race. The most recent such question, from the end of October, showed the Democrats with a 7% edge. Probabilistically, we can say that because Gallup tends to skew toward the Democrats this far out, and because roughly even distributions of votes between parties means few seat changes, we can therefore conclude, based upon the generic question, that 2006 will see few seat turnovers. Now, this is a rough estimate. We are only speaking probabilistically. Sometimes Gallup’s numbers do not skew very much this far out and sometimes small changes in partisan vote totals can result in big changes. So, while we can make a guess based upon the generic question, it seems to be a little bit too “iffy” for the taste of the curious.

There is another question that has been asked for a long enough time that it may be a potential indicator of electoral outcomes – and that is the congressional job approval question. Gallup has been asking this question intermittently since 1974. Academic work on the relationship of this question to congressional seat changes has been sparse to date, and has been conducted only relatively recently. This work nevertheless suggests that there may be a statistically significant correlation between vote choice and congressional job approval, but the tests are not conclusive. In particular (and without getting too technical), if you compare changes in congressional composition to changes in congressional approval ratings, this relationship weakens significantly. Some years, Congress worsens its perception with the public and still the party of the President gains seats (as in 2004, when congressional approval had actually dipped 10 points from 2002, but the GOP still gained three seats; a similar phenomenon happened in 1994 where congressional approval had actually ticked up five points, but the Democrats still lost 54 seats). And, what is more, no work that I know of has been done to see how well this statistic performs against changes in the partisan composition of the Senate. Suffice to say that, while there has been some promising first steps in understanding the nature of the relationship between seat changes and the job approval statistic, much more research needs to be done before we can make any firm conclusions. Above all, more data needs to be collected – and that just requires more time.

This far from Election Day, then, we have to rely on something other than public opinion on Congress to give us a sense of what to expect. In my last column, I mentioned presidential approval as a possibility. Indeed, there is a strong correlation between presidential approval and aggregate vote shifts. In other words, shifts in the approval of the president tend to increase or decrease the number of seats the of president’s party in Congress. This, obviously, does not augur well for the GOP. But the other statistic that works well – changes in per capital real disposable income (RDI)– does augur well for the GOP. As RDI increases the party of the president historically enjoys improved results in Congress. So, one aggregate statistic cuts against the GOP and one cuts for the GOP.

The trouble with both statistics, however, is that when we ask voters, “Why did you vote for that guy?” they never seem to answer that it was because they were angry with the president or glad that they have more money in their wallet. Votes on the congressional level seem to boil down to issues of personality, character, capacity and local provision. There is a disconnection between the motivations of individual behavior and the seeming reasons the country as a whole votes. Even though the actions of every average voter aggregates into a situation where presidents are punished or rewarded for their approval rating or the state of the economy, individual average voters do not think about those things when they go to the ballot box.

As I mentioned last time, one possible way to bridge the gap between “micromotives and macrobehavior” is candidate recruitment. When national conditions favor the party of the president, the top-line candidates of the opposing party do not enter the race. Poor candidate recruitment means that incumbents tend to face second-rate opponents who are less well financed and not as good at running campaigns, and therefore can structure the election as a referendum on personality or the number of potholes that have been filled. One bit of evidence in favor of this is that competitive contests, particularly contests where there are open seats, tend to focus more on national issues, while non-competitive elections do not.

The logic behind this is that elite politicians are strategic. In every state, there is a group of professional politicians, on either side of the aisle, who stand a reasonable chance, at some point in the future, of obtaining a more powerful position. The logic of the strategic politician indicates that they wait until political conditions are most favorable to victory. This has been offered as an explanation of the big year that Democrats had in 1974. Voters did not intentionally punish Republican members of Congress for Watergate; however, Watergate imbued in strategic, ambitious Democrats a sense that 1974 would be a bad year for the GOP. So, the office-hungry, sophisticated politicians threw their hats into the ring.

And who fills the candidate vacancies when the elite office-seekers demure? Well, if you have ever lived in a non-competitive congressional district, you have probably seen them. They tend to be people who do not know they stand no chance, who just want to raise awareness of an issue, who have nothing to lose, who have been induced by the party to “take one for the team” for the promise of future rewards, etc. From the looks of things, 2006 seems to be like an election year replete with such hopeless candidates.

Thus, given that the two reliable aggregate statistics are cutting in opposite directions, and they do not seem to tell us about how the individual voter thinks anyway, we can turn to candidate recruitment as an early “tie-breaker,” as a way to get a sense of who has an edge. Currently, candidate recruitment has been poor for both parties, which means that the GOP has an advantage. Why? The GOP, to hold the majority, only needs to protect its incumbents. Low-quality candidates from the Democrats mean that GOP incumbents will have an easier time next year.

So, there are two “take home” points. First, before drawing conclusions about what will happen next year based upon national polling data, be careful. Doing so is much trickier than it might seem to be. Second, the balance of the data seems to indicate that 2006 will not be a year of big changes. Factor into this the structural situation we discussed last time, i.e. the fact that the public has since 1994 been largely realigned, and we can be pretty confident that, barring any major changes in the political landscape, this will be the case in 2006.

Jay Cost, creator of The Horse Race Blog, is a graduate student of political science at the University of Chicago. He can be reached at jay_cost@hotmail.com.

A year before the election the Gallup generic question tends to allocate about 6% more points than it should to the Democrats.

From 1942 until 1992 most states and most house districts were Gerrymandered by Democrats. From 1940 until 1990 the Democrats controled enough states to gerrymander a majority of house districts.

A Democratic Gerrymander works like this. The Republicans are grouped into a few districts where Republicans will have a 70 to 30 advantage. The Remainder of the Districts will be drawn to have about 53 to 47 Democratic advantage. A typical example would be a state with 15 congressional districts. Five of which would be drawn to have 70 to 30 Republican Advantage. The remaining 10 districts would have 53 to 47 Democratic advantage. Thus even if the state went 52.5 to 47.5 Republican, the Democrats would still win 10 of the 15 races.

That is Gerrymandering.

Starting in 1992 and continuing in 2000 the Republicans have won control of a majority of states containing a majority of congressional districts. As a result those states have been Gerrymandered for Republicans.

The 1990 gerrymandering gave the Republicans a slight advantage. The Perot race in 1992 allowed the Democrats to hold the house for two more years. But in 1994 with Perot a greatly reduced factor, the Republicans won the house.

Even a strong Clinton win in 1996 did not allow the Democrats to control the house.

The 2000 election gave the Republicans an even bigger advantage and the 2000 redistricting made even more seats safe for Republicans.

There are at best 35 competitive seats in the house. To win the house the Democrats would have to win all 35 seats. That is very unlikely.

The most the democrats can hope for is to win control of enough states to be in charge of redistricting in most states in 2010. That means the first real chance the Democrats have to take the house is in 2012.

If you wonder why Dick Gebhardt chose not to continue in the house, it was because he saw little chance to be speaker before 2013 and even that was not a high prospect.

The 2006 senate races have 17 Democrat and 15 Republican seats on the line. To win control of the Senate the Democrats would need to retake all of their 17 seats and take 6 of the 15 Republican seats. Five of the Repubican seats could be considered competitive and 3 Democrat seats are competitive. Even if the Democrats win them all they would still not control the senate. A 50-50 tie vote means Cheney casts the deciding vote.

To take the Senate the Democrats have to win all the competitive senate seats and win one that is not.

The Democrats in 2006 are a very long shot.. no matter what the media types tell you.

Jay has left out an important factor - the War in Iraq. The party in power is hurt by a lingering war; as in 1966 and 1968 the Democrats were hurt, so that - spite of an economy that was performing well - they lost seats in '66 and the presidency in '68. War and political scandals have to be factored into the analysis, as they are sometimes important. As consumer confidence grows, and with the beginning of our withdrawal from Iraq, prospects for the Republicans should improve.

In terms of candidate recruitment, the Republicans have come up with some very good candidates in some winnable Senate races (e.g., Md., Mn. and N.J.), but have not come up with top-notch candidates in others (e.g., Fl., Ne., N.D., Wa., and Wi.). I'm still hoping that certain people will do the right thing, and put us in position to swim upstream in what looks - at this time - to be a tough year for our side, but time is running short.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.