| Perspectives on Texas

In the last few weeks, a handful of polls showing two rather divergent pictures of the Texas governor’s race have surfaced. On the one side are polls showing Attorney General Greg Abbott up by as many as 18 points. On the other are polls showing state Sen. Wendy Davis trailing by “only” single digits — 8, to be exact, according to a Rasmussen Reports survey that quickly became the Davis campaign’s pick to hit.

The public — and reporters trying to write about the race — have been left to wonder whether they should expect Abbott to cruise to victory in November or whether Davis has a fighting chance based on the “momentum” her campaign keeps emailing reporters and supporters about several times a day.

Given this discussion and the likely release of a cluster of new polls in the next few weeks, it seems like a good time to answer some common questions being asked about polling in this race (and in general).

This thing has really been a roller coaster. Don’t these shifting poll results reflect the volatility of the Abbott-Davis race?

Differences in polling results can reflect real changes in the underlying state of a race, but more often, especially when polls are released in close proximity to one another, they just represent the error associated with asking a question of a sample instead of asking it of everyone. Even if two polls were conducted over the exact same period using gold-standard polling practices, their results would still differ slightly.

Imagine trying to gauge the average height of a room of 100 people by measuring 20 of them. Even if you randomly selected 20 people from the room a number of times, you likely wouldn’t get the exact same average. Does this mean that the average height of the room is volatile, or the underlying dynamics of heights are shifting? That people in the room are growing or shrinking? Of course not. Rather, there's some error associated with taking a sample instead of measuring the whole. That's okay, and it’s part of the reason why polls have margins of error.

But why are different polls showing such different results? Shouldn't I just ignore them?

Most polls aren’t as perfect as the hypothetical example above and, on a poll by poll basis, can reflect a number of potential factors that can alter the results. All polls are subject to response bias — in short, who chooses to participate and who doesn’t. But there are a number of other issues that could cause two polls to produce different results. For example, different pollsters might ask the same question slightly differently. One might randomize the response options while the other might not. One might follow the initial question with a prompt to force the respondent to make a choice, while the other might not. And different pollsters might put their trial ballot questions at different places in the survey, producing effects based on the order. In short, a lot of factors can affect a result.

And even after the data has been collected, pollsters have to construct likely-voter screens, removing respondents from their estimates who don’t seem likely to vote, and apply weights to the data — basically, slight adjustments to how much each respondent is counted when estimating results to make up for the initial response bias.

All of this is standard practice but, from one survey to the next, can easily contribute to differing results.

No, really. There are so many polls, but every time I hear or read about them, someone is saying they’re all wrong because there's a crisis in polling. Why bother?

If the list from which potential respondents are sampled is well constructed and some basic “best practices” are followed in terms of question wording, placement, and screening and weighting, polls tend to gauge public opinion pretty well, even if their individual methods and ingredient lists differ as practitioners work to adjust to a rapidly evolving communication environment.

Psst. Someone just told me about an internal poll, and it says my candidate is up. What about the internal polls?

We’re on the record on that one, and the point is worth revisiting. In short, under most circumstances, partial releases or “leaks” of campaign polls should be taken with a shaker of salt.

Hey, c'mon. Really. Given all these different results, how are we supposed to know what’s really going on? Which poll is right?

Ours is! Just kidding. The best way to “know” what’s truly going on, besides polling everybody (or having an actual election), is to use an estimate based on the average (loosely speaking) of the public (i.e., non-internal) polls that have followed professional norms of disclosure. As already stated, one poll is usually pretty robust if that’s all you have, but two are better than one, and three are better than two. In the case of the Abbott-Davis contest, the averages show — even with intermittent polling from various sources — about a 12- to 13-point gap in Abbott’s favor.

This estimate makes a fair amount of intuitive sense, particularly if you factor in the margins of error in the public polls we have to work with in Texas. Polls come with a margin of error, which in most of the Abbott-Davis polling has been 3.5 to 4.5 percentage points. So a poll that posits an 8-point race could be a 4- or 12-point one, and an 18-point race could be a 22- or 14-point one. Based on previous vote margins in Texas (for example, Rick Perry’s 55-42 defeat of Bill White in 2010), a 12- to 14-point race — somewhere between the two numbers that have been so heavily reported — sounds plausible right now. Of course, this often raises the question:

If all of these efforts to get more Hispanics registered to vote are succeeding, aren't the assumptions of all these polls going to be wrong?

It depends, but probably not — and if so, probably not by a lot. Given that most of the discussion about Texas’ changing electorate focuses on previously unregistered Hispanic voters, it's possible that polls are miscalculating their results for a few big reasons: because they lack coverage for these newly engaged groups (i.e., they’re not reaching out to them); response biases might be more problematic (i.e., they’re reaching these potential voters, but the voters are less inclined to respond); and/or the weights pollsters are using to adjust their data might not accurately reflect this new electorate.

A monumentally successful effort on the part of groups aiming to boost Hispanic turnout would likely shift the vote margin by 1 to 3 points. At the same time, midterm election polls tend to overestimate Democratic support because the party’s coalition tends to be less likely to show up in off-year elections, making an under-the-radar Hispanic mobilization blitz a likely wash given the regular dip in turnout among Democratic voters. Until we start seeing multiple polls showing a race in the low-single digits, increased Hispanic mobilization isn’t likely to significantly alter the outcome, or predictions based on these polls.

But stay tuned. Beyond the two disparate poll results over the last couple of weeks, we haven't seen much public polling in the last month. We'll likely see at least one more poll in September and a few more in October. Don’t be surprised if you continue to see variations — and keep your eye on those trend lines, however blurry they may seem.