Replies

I’ve been digging — sorry, it’s taking me a while. The biggest problem is that I can’t find historical InTrade data online (at least for free) except in sporadic news reports. All the IEM data is online — Kevmo, in your experience does IEM track Intrade (a much bigger market) well enough for IEM to be a useful data set? Unfortunately, the IEM doesn’t have the state primary races — so it would be ideal if we could access historical Intrade data.

Another problem (at least as it relates to assessing this primary) is that we have data from only two similar, multiple-candidate primaries to evaluate (the GOP primary in 2000, and the donk primary in ‘04). Two-candidate general election races are much less complex and thus less pertinent to our current situation (’though still interesting).

Anyhoo, I’ve run across two instances so far where the prediction markets got it wrong. Caveat: it’s important to remember, as was noted in Kevmo’s research, that no one is claiming that prediction markets can tell us what’s going to happen. The issue is whether they are a better predictor than the polls. So, to fully evaluate the two instances below, we have to document what the polls said (i.e., were they even more off than Intrade?) In the first instance, the article says that the prediction market basically tracked the polls, so everyone got it wrong.

The biggest prediction market error was the dem primary run-up to the Iowa caucuses:

“Consider the Democratic caucus in Iowa. It is the first and most influential primary of the entire process, and it was the most anticipated primary of the season. There had been a solid year of campaigning leading up to it. The last three months were replete with intense local media coverage. If any prediction market should function correctly, this one was it.

“Well, right up until a few days before the primary, this market’s prediction was that Dean would win, and do so handily. Only he didn’t. Dean got crushed, and Kerry went on to be the nominee.”

http://www.thestreet.com/p/_rms/rmoney/barryritholtz/10185976.html

Another was from the general election in 2000:

“SM: The IEM predicted Bush would win the popular vote in 2000, which he didn’t. What went wrong?

“TR: IEM predicted about a two-thirds chance that he would win the popular vote, and Gore barely won. In our Winner-Takes-All market, we predict the probabilities of an event. In this case, it was either Bush or Gore taking the majority of the popular vote. When you predict the probability of an event, what you’re going to see is an outcome - either it happened or it didn’t. We predicted that there was a two-thirds chance that Bush would take the popular vote, but that means we also said there was a one-third chance that Gore would, and that’s what happened.”

http://www.smartmoney.com/theproshop/index.cfm?story=20041007