Jay Cost: Does This Math Add Up?

Stuart Rothenberg updated his House rankings a few days ago. And yowsa!! He sees 34 to 40 seats flipping. That is quite a large number.

But take a look at the assignment of races to each category. I don't think those numbers add up.

PURE TOSS-UP (20 R, 0 D)
CA 11 (Pombo, R)
CT 2 (Simmons, R)
CT 4 (Shays, R)
FL 16 (Open; Foley, R)
FL 22 (Shaw, R)
IL 6 (Open; Hyde, R)
KS 2 (Ryun, R)
MN 1 (Gutknecht, R)
MN 6 (Open; Kennedy, R)
NM 1 (Wilson, R)
NY 20 (Sweeney, R)
NY 26 (Reynolds, R)
OH 1 (Chabot, R)
OH 2 (Schmidt, R)
PA 4 (Hart, R)
PA 6 (Gerlach, R)
PA 8 (Fitzpatrick, R)
TX 22 (Open; DeLay, R)
VA 2 (Drake, R)
WI 8 (Open; Green, R)
TOSS-UP/TILT REPUBLICAN (10 R, 0 D)
AZ 1 (Renzi, R)
CA 4 (Doolittle, R)
CO 4 (Musgrave, R)
ID 1 (Open; Otter, R)
KY 3 (Northup, R)
KY 4 (Davis, R)
NV 3 (Porter, R)
NY 25 (Walsh, R)
NY 29 (Kuhl, R)
WA 8 (Reichert, R)
TOSS-UP/TILT DEMOCRATIC (7 R, 3 D)
AZ 5 (Hayworth, R)
CT 5 (Johnson, R)
FL 13 (Open; Harris, R)
GA 8 (Marshall, D)
GA 12 (Barrow, D)
IL 8 (Bean, D)
IN 9 (Sodrel, R)
NH 2 (Bass, R)
NY 24 (Open; Boehlert, R)
NC 11 (Taylor, R)
LEAN REPUBLICAN (3 R, 0 D)
NJ 7 (Ferguson, R)
OH 12 (Tiberi, R)
TX 23 (Bonilla, R)
LEAN DEMOCRATIC (6 R, 1 D)
IA 1 (Open; Nussle, R)
IA 3 (Boswell, D)
IN 2 (Chocola, R)
OH 15 (Pryce, R)
OH 18 (Open; Ney, R)
PA 7 (Weldon, R)
PA 10 (Sherwood, R)
REPUBLICAN FAVORED (8 R, 0 D)
CA 50 (Bilbray, R)
CO 5 (Open; Hefley, R)
KY 2 (Lewis, R)
NE 3 (Open; Osborne, R)
NV 2 (Open; Gibbons, R)
NY 3 (King, R)
NY 19 (Kelly, R)
WY AL (Cubin, R)
DEMOCRAT FAVORED (3 R, 1 D)
AZ 8 (Open; Kolbe, R)
CO 7 (Open; Beauprez, R)
IN 8 (Hostettler, R)
VT A-L (Open; Sanders, D)

That would be 57 Republican seats that are, in some way, vulnerable. Fair enough. As I argued this week, I think this list is a tad too long, but that is alright.

Here is where I run into difficulties. How does this list add up to a net pickup of 34 to 40 seats? Let us assume (1) that each category is -- to some degree -- vulnerable (i.e. that the two unmentioned categories are "Safe Republican" and "Safe Democrat"), (2) that all seats not mentioned here are in their parties' respective "Safe" categories, (3) that each category is equidistant from those immediately preceding and following it (i.e. each category is separated from the two closest by +12.5% or -12.5%), and (4) that the titles mean the same for one party as they do for the other (e.g. the Democrats have as good a chance in "Lean Democrat" as the Republicans do in "Lean Republican").

I think these are all fair assumptions. In fact, I am willing to bet that the average reader would implicitly make these assumptions upon a viewing of Rothenberg's list.

With these assumptions, that would mean that any given race in a given category would have the following probability of Democratic victory:

"Safe Democrat:" 100%
"Democrat Favored:" 87.5%
"Lean Democratic:" 75%
"Toss-Up/Tilt Democratic:" 62.5%
"Pure Toss-Up:" 50%
"Toss-Up/Tilt Republican:" 37.5%
"Lean Republican:" 25%
"Republican Favored:" 12.5%
"Safe Republican:" 0%

What we have, then, are 9 binomial distributions. The average, or expected value, for each distribution is simply the probability of victory for any given seat multiplied by the number of seats in the distribution.

Accordingly, for each category, we should expect the Democrats to win:

"Safe Democrat:" 100% * 198 Seats = 198 Seats
"Democrat Favored:" 87.5% * 4 Seats = 3.5 Seats
"Lean Democratic:" 75% * 7 Seats = 5.25 Seats
"Toss-Up/Tilt Democratic:" 62.5% * 10 Seats = 6.25 Seats
"Pure Toss-Up:" 50% * 20 Seats = 10 Seats
"Toss-Up/Tilt Republican:" 37.5% * 10 Seats = 3.75 Seats
"Lean Republican:" 25% * 3 Seats = 0.75 Seats
"Republican Favored:" 12.5% * 8 Seats = 1 Seat
"Safe Republican:" 0% * 175 Seats = 0 Seats

These expected values sum to 228.5. In other words, these distributions imply that the Democratic caucus will be 228 to 229 seats, which is to say that the Democrats should expect to net 25 to 26 seats.

This is 8 to 15 seats short of Rothenberg's estimate.

Take this from another perspective. Rothenberg's final estimate of net 34 to 40 means that the Democrats will win 63% to 73% of the seats on this list. That would put the mean probability between "Toss-Up/Tilt Democratic" and "Lean Democratic." However, in actuality the mean probability that the Democrats will win a seat is 49%. The median probability is 50%. The modal probability is 50%. In other words, the central tendency is "Toss-Up" (with an ever-so-slight nod to "Toss-Up/Tilt Republican"), not "Toss-Up/Tilt Democratic" and "Lean Democratic."

So -- even if we want to assign the probabilities differently, we will assuredly come short of an expected net of 34 to 40 seats -- provided that, for example, "Lean Democrat" means for Democrats what "Lean Republican" means for the Republicans. For instance, if we give the Republicans everything that has a Republican tilt to it, we give the Democrats everything that has a Democratic tilt to it, and we split the "Pure Toss-Ups" -- the Democrats would net 26 seats.

Maybe this gets to what I was hinting at in my recent critique of Cook. The major race rankers see a massive "wave" coming, but cannot really find the districts to upgrade to fit the wave. Cook's response has ostensibly been to develop a "Gimme a reason, punk!" kind of attitude toward Republican seats -- i.e. any seat where the Republicans blink is a seat that gets upgraded -- candidate financing, party involvement, district partisanship aside. The net result is a set of highly conservative seats that -- despite the negative mood toward the GOP and despite whatever drama might be happening on the ground -- are really unlikely to switch, and, minimally, do not justify the 1994 comparisons that Cook has been supplying with his list. 1994 saw Democrat-held 0 seats from districts in which George H.W. Bush did 9% or worse than his 1992 national average switch to the Republicans; Cook's list currently has 11 such Republican-held seats (i.e. seats from districts where Kerry did 9% or worse than his national average) rated as vulnerable.

Rothenberg's response? Well -- from the looks of it, he is implying that his race-by-race estimates will be wrong - and not just a little bit wrong. A lot wrong. Why does he not correct them so that they show something like 34 to 40? Maybe he does not because he just cannot find the races to fit into a 34 to 40 scenario, but thinks it will happen nonetheless.

This is, I think, what he thinks - though I am not sure he and his staff grasp just how divergent their aggregate estimate is from their race-by-race analysis. His political editor, Nathan Gonzalez, commented to the San Francisco Chronicle:

"As we looked back to 1994 in our analysis, Republicans even won half of the toss-up, tilting-Democratic seats back then," said Gonzales of the Rothenberg Political Report. "Because this is such a volatile environment, I think there will be a couple surprises, a couple members who will lose that no one was talking about.

In other words -- the list is different from the final number because Rothenberg and Gonzalez expect a surprise. Fair enough (well, not entirely - there are good reasons not to expect certain types of surprises - but we'll talk about that tomorrow). However, this does not cut the mustard. If we do what Gonzalez thinks we should do, if we altered the "Toss-Up/Tilt Republican" to give the Democrats 50% of the seats, that would only give the Democrats an extra 1.25 seats.

Gonzalez needs something much more extreme to get to 34 to 40. This might work: if we gave the Republicans nothing of the seats that are tilting/leaning/favoring Democrats and gave the Democrats half of everything that is "Toss-Up" or tilting/leaning/favoring Republicans, that would give the Democrats 36 to 37 seats. In other words, re-jigger the list to put 20 seats into "Safe Democratic" and 21 seats into "Pure Toss-Up" - which is to say, shift 66% of all vulnerable seats toward the Democrats - and you can hit the mid-point of their estimate.

So - that is the kind of "surprise" that Rothenberg and Gonzalez are expecting. Not a slight shocker on the margins - but a massive movement of seats toward the Democrats that these two have been unable to identify. And, I would note that I am positive that these two have been looking at each seat very closely to identify any kind of Republican weakness. To wit: they have about 20 Republican seats where (a) the Democratic challenger is under-funded, (b) the NRCC is not spending a dime on advertisements, or (c) the DCCC is not spending a dime. Either they know something the Hill committees and party donors do not, or they have inflated their list of Republican targets. Nevertheless, they still think they have underestimated Democratic strength.

In other words, Rothenberg and Gonzalez are expecting that, though they are convinced that the Democrats are as strong as they have been since Skynyrd's Second Helping, and though they have presumably completed a thorough search for any GOP seat with even the vaguest sign of weakness, they nevertheless believe that they have systematically underestimated Democratic strength by 30% to 60%!

That is a lot of error to commit when you are on the look-out for exactly that type of error.

Why are they doing this?

My read of Rothenberg, and Cook for that matter, is that they do not want to be on the low side of the next "1994." They want to minimize the probability of the false negative, i.e. Type II error. In other words, they do not want to fail to predict a seat will switch when it indeed will switch. Or, in the aggregate, they do not want to underestimate Democratic gains.

Meanwhile, neither of them seems to be all that concerned over Type 1 error, i.e. the error of the false positive. If they estimate that a seat is vulnerable when it in fact does not switch -- they don't seem to think that is a big problem. That is what I was getting at this week with Cook and his super-conservative Toss-Up districts, and also Rothenberg with his 20 or so Republican seats that just do not have the money situation to validate a vulnerability estimate,as well as his "fuzzy math" (oh...come on! You knew that was comin' eventually, right?). Rothenberg and Cook seem intent upon not underestimating Democratic strengths, even if it induces them to overestimate those strengths according to the internal logic of their arguments.

From my perspective, the prudent response is to minize total error, regardless of type. Type 1 error and Type 2 error are both error. We need to try to minimize all of it. A 10% reduction in the possibility of Type 2 error does us no good if it causes a 20% increase in the possibility of Type 1 error. You're still more wrong than when you started. You're just wrong-and-high. What is the value of that?

Simply stated, I think these two are tripping over themselves to amp up estimated Democratic gains. And, as I see it, both have stumbled in the last week. Cook cannot predict 1994-in-reverse using his list. He has districts on it that are up to 300% more conservative than the most liberal district to flip in 1994. Rothenberg cannot predict 34 to 40 using his list. My feeling is that they both have definitely minimized Type 2 error. With their lists, they have identified all of the seats that will flip. But the price they have paid is in Type 1 error. They both have a large cache of seats that just ain't gonna flip.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.