Election Update: Leave The LA Times Poll Alone! (Little Nate doesn't like it)

538 ^ | 8/23/2016 | Nate Silver

[usafa92]

I’m tired of hearing about the poll from Donald Trump fans such as Reince Priebus, Matt Drudge and Donald Trump himself.1There’s nothing wrong with being your own No. 1 fan! They frequently cherry-pick that poll because it consistently shows much better results for Trump than the other surveys. As of Tuesday morning, for example, the poll showed the race as virtually tied — Hillary Clinton 44.2 percent, Trump 44.0 percent — even when the national poll average has Clinton up by about 6 percentage points instead.

This has been a fairly consistent difference between this poll and most others. Take the LA Times poll, add 6 points to Clinton, and you usually wind up with something close to the FiveThirtyEight or RealClearPolitics national polling average. What’s the source of the LA Times poll’s Trump lean? There are good “explainers” from The New York Times’s Nate Cohn and Huffington Post Pollster’s David Rothschild. Long story short: The poll’s results are weighted based on how people said they voted in 2012. That’s probably a mistake, because people often misstate or misremember their vote from previous elections.2In particular, it’s likely that more people say they voted for the winner than actually did. Imagine, for example, that respondents in a poll claim they voted for Barack Obama by 10 percentage points, when he actually beat Mitt Romney by 4 percentage points. The LA Times poll will conclude that it has too many Obama voters, most of whom are also Clinton voters, and therefore downweight Clinton’s numbers. But some of those Obama “voters” actually voted for Romney or sat the election out.

The poll does some other things differently also, some of which I like. For instance, it allows people to assign themselves a probability of voting for either candidate instead of saying they’re 100 percent sure. And the poll surveys the same panel of roughly 3,000 people over and over instead of recruiting new respondents. That creates a more stable baseline and can therefore be a good way to detect trends in voter preferences, although it also means that if the panel happened to be more Trump-leaning or Clinton-leaning than the population as a whole, you’d be stuck with it for the rest of the year.

But I’m also tired of hearing from the LA Times poll’s critics. I’m not a fan of litigating individual polls, for several reasons. First, in my experience, these critiques tend to involve their own form of cherry-picking. Clinton fans will pick apart the LA Times poll and find a few things wanting — in this case, with good reason (in my opinion). But they’ll give a free pass to a poll like this one that shows Clinton ahead by 16 percentage points in Virginia, even though it’s also something of an outlier. You can almost always find something “wrong” with a poll you don’t like, even if you might have approved of its methodology before you saw its result.

It’s probably also harmful for the profession as a whole when poll-watchers are constantly trying to browbeat “outlier” polls into submission. That can encourage herding — pollsters rallying around a narrow consensus to avoid sticking out — which is bad news, since herding reduces the benefit of averaging polls and makes them less accurate overall.

Furthermore, the trend from LA Times poll still provides useful information, even if the level is off. Before the conventions, the poll had Trump ahead by an average of 2 or 3 percentage points. Trump then got a modest convention bounce in the poll and pulled ahead by 6 or 7 percentage points. But Clinton got a bigger bounce, and she’s been ahead by an average of 1 or 2 percentage points in the poll since the conventions, although it’s been a bit less than that recently, with Trump narrowly leading the poll at times. All of this follows the trend from other polls almost perfectly, as long as you remember that you have to shift things to Clinton by about 6 points.

And that’s pretty much what FiveThirtyEight’s forecast models do through their house effects adjustment. A pollster’s house effect is a persistent lean toward one candidate or another, relative to other polls. House effects are not the same thing as statistical bias — how the poll compares against actual results — which can be assessed only after the fact. Nor do they necessarily indicate partisan bias. For example, Public Policy Polling, a Democratic polling firm, has a very mild pro-Trump house effect this year.

Calculating house effects is simple, in principle — you compare a poll’s results against the average of other surveys of the same states (treating national polls as their own “state”). In practice, there are a few challenges, which you can read more about in our methodology primer. One of the important ones is defining what the average is. In the case of FiveThirtyEight’s forecasts, the average is weighted based on our pollster ratings.

Put another way, the house effects adjustment seeks to determine what the best pollsters are saying and not just what the most prolific ones are saying. In 2012, that made a difference: the higher-quality pollsters generally projected better results for Obama than the lower-quality ones. This year, any such effects are very minor. Although they may be increasing, with traditional telephone polls tending to show better numbers for Clinton recently. and neither Trump nor Clinton benefits much from the house effects adjustment overall, although it can matter more in individual states. Polls in Nevada happen to be a Trump-leaning bunch, for instance, so the house effects adjustment slightly helps Clinton there.

[Sooth2222]

"Nate was right in 2012."

I think he was right in 2008, too. But that one was a no-brainer in retrospect, given two unpopular wars plus an economic implosion.

In 2012, as the Rand Poll, the USC/LA Times poll was very accurate. Yes, it's cherry picking but I'm okay cherry picking as long as it's from the tasty cherries on the tree instead of the spoiled cherries on the ground. The polling techniques that worked in 1936 and failed miserably in 2012 probably haven't gotten any better in 2016. If Gallup did so badly in 2012 that it gave up on presidential election polling, the polls that did okay in 2012 using the same techniques were probably right for the wrong reasons.

The #1 poll in 2012, IBD/Tipp, had Trump -7 but that was two weeks ago, the week following the Democrat convention. #5 in 2012, CNN ORC had Trump -8 but that poll was taken immediately following the Democrat convention. Reuters/Ipsos had Trump -4 as of Wednesday. Their previous poll was Trump -5. It's going in the right direction. Several of the better polling firms haven't even started polling for 2016. What all the polls seem to have demonstrated is a post convention bump for Trump, followed by a post convention bump for Clinton and now it's settling back.

[Flavious_Maximus]

It’s because they fear nationalism. They equate patriotism with Nazism.

This is why all the neocons (William Kristol, Charles Krauthammer, Jonah Goldberg, Jennifer Rubin, Michael Medved, Mark Levin etc) hate Trump.

[catnipman]

“By the night before Election Day we should know who is going to win.”

I don’t think so. I don’t think many of the polls are accurate for several different reasons. Lland-line polling is essentially dead, and the pollsters have yet to find an accurate alternative; there have many articles written pollsters to that effect have been posted at FR. And i think there’s a ton of people that simply aren’t saying who they’ll vote for, particularly Trump people.

[jospehm20]

Is this the same guy who has been wrong about Trump at every step of the way?

[jospehm20]

Is this the same guy who has been wrong about Trump at every step of the way?

[jospehm20]

Is this the same guy who has been wrong about Trump at every step of the way?

[tcrlaf]

Nate was right in 2012.
But he hasn’t been right about Trump even once.

He is SHILLING to protect his paychecks.
He predicted Brexit would LOSE by 10% the day before the vote, IYR...

Dems ARE NOT going to turn out for Hillary at HIGHER levels than they did for Obama in 2012.
Blacks are NOT going to turn out in 2012 numbers to vote for the Old White Plantation Slavemaster.

They KNOW THIS (And it’s why #BLM exists), but he is demanding that other pollsters acoount that way, and some have.

[jospehm20]

Triple post. Sorry. Maybe this iPad thingy is not as cool as I thought.

[piytar]

“hat’s probably a mistake, because people often misstate or misremember their vote from previous elections.”

Huh, what? Talk about stupid or desperate...

[InterceptPoint]

You could be right about the difficulties in polling particularly when we have a race like this one that features (love ‘em or hate ‘em) two very polarizing figures.

That said, there is no reason why polls like the USC Poll can’t turn out to be fairly accurate. These polls measure changes in the voting intentions of a fixed and large population of voters over time. That is very different than your classic telephone survey that, as you pointed out, is inherently a flawed methodology.

Will it happen that the USC Poll calls it right? Time will tell. If we really are looking at a big movement to Trump from the sick and corrupt Hillary over the next 70 days it will be visible in that poll.

Fingers crossed.

[itsahoot]

Still haven’t figured out why the enlightened jews in this country hate Trump so much.

Apparently it is possible to be extremely intelligent and extremely stupid at the same time.

[catnipman]

“If we really are looking at a big movement to Trump from the sick and corrupt Hillary over the next 70 days it will be visible in that poll.”

You could very well be correct about the USC poll and its new methodology. I suspect the old, land-line-based polls are worth than useless.

[itsahoot]

“hat’s probably a mistake, because people often misstate or misremember their vote from previous elections.”

Straight ticket voters rarely know more than the top of the ticket and vote mechanically for people they have never heard of.

[monkeyshine]

Any time you start censoring data points from a study (such as a poll) you threaten accuracy. There are already so many types of bias inherent in polling, things like confirmation bias, proper sample distribution, selection bias etc as to make them an iffy proposition. To then go back and censor 5% or 6% of the respondents because you think they were over-represented is a perilous endeavor. Your reasoning may be correct (or maybe not), but your sample size is still relatively small in the grand scheme of things so you cannot really be sure you captured a sample that really is overweighted. Plus your assumption ignores unknown variables such as whether and how this election may differ from previous elections. Just because 6% may have misremembered who they voted for (or intentionally lied and said they voted for the winner) does not mean that they are lying about their current intentions. Obama may have had a 4% overall lead in popular vote in 2012, but Obama did not win every state nor did he do so by 4% in every state he won. North Carolina for example went to Romney by 2%. How do you adjust a national poll by 6% when you know that this 6% number did not apply in a large % of the voting districts, and in fact went the other way in say 40% of the districts or an 8% swing from your hypothesis? His idea is fraught with problems.

[piytar]

Yeah, but I seriously doubt anyone (or at least very few people) mistake or misremember whom they voted for President!