How Serious Is Polling's Cell-Only Problem?

Twenty years ago the survey research profession -- having grown comfortable with telephone interviewing as an alternative to personal interviewing for conducting surveys -- worried mostly about the roughly 7% of U.S. households that could not be interviewed because they had no telephone. Today our concern is somewhat different, and potentially more serious. According to government statistics released last month, nearly 13% of U.S. households (12.8%) cannot now be reached by the typical telephone survey because they have only a cell phone and no landline telephone.1

If people who can only be reached by cell phone were just like those with landlines, their absence from surveys would not create a problem for polling. But cell-only adults are very different. The National Health Interview Survey found them to be much younger, more likely to be African American or Hispanic, less likely to be married, and less likely to be a homeowner than adults with landline telephones. These demographic characteristics are correlated with a wide range of social and political behaviors.

Polling's cell phone problem is a new one. In early 2003, just 3.2% of households were cell-only. By the fall of 2004, pollsters and journalists were openly worrying about the potential bias that cell-only households might create for political surveys. The National Election Pool's exit poll found that 7.1% of those who voted on Election Day had only a cell phone, and these cell-only voters were somewhat more Democratic and liberal than those who said they had a landline telephone. But pre-election telephone polls in that election were generally accurate, and pollsters felt that they had dodged the proverbial bullet. This fortunate outcome was a result of the fact that the statistical weighting employed by most telephone polls helped to correct for the missing respondents. The fact that the cell-only group in 2004 was still a relatively small part of the overall population also helped mitigate the impact of the problem.

But given the speed with which the number of cell-only households has increased, there is growing concern within the polling business about how long the landline telephone survey will remain a viable data collection tool, at least by itself. At last month's annual meeting of the American Association for Public Opinion Research (AAPOR), survey research's top professional organization, an entire series of research panels focused on the cell phone issue. At one of the panels, a government researcher told the audience that the size of the cell-only group could approach 25% by the end of 2008 if the current rate of increase is sustained.

To monitor the impact of the cell-only phenomenon, the Pew Research Center conducted four studies in 2006 that included samples of cell phone numbers as well as a full sample of landline numbers.2 The four surveys covered a very wide range of topics, including use of technology, media consumption, political and social attitudes, and electoral engagement. Comparing the cell-only respondents with those reached on landlines allowed us to assess the degree to which our traditional surveys are biased by the absence of the cell-only respondents.

We compared the cell-only and landline respondents on 46 different survey questions. Across these questions, the average difference between cell-only and landline respondents was approximately 8 percentage points (7.8%), with the range of differences running from 0% (for a question about whether the respondent is "bored" by what goes on in Washington, DC) to 29% (being registered to vote). But the good news is that none of the measures would change by more than 2 percentage points when the cell-only respondents were blended into the landline sample. Thus, although cell-only respondents are different from landline respondents in important ways, they were neither numerous enough nor different enough on the questions we examined to produce a significant change in overall general population survey estimates when included with the landline samples and weighted according to U.S. Census parameters on basic demographic characteristics.

The picture is not entirely positive, however. While the cell-only problem is currently not biasing polls based on the entire population, it may very well be damaging estimates for certain subgroups in which the use of only a cell phone is more common. This concern is particularly relevant for young adults. According to the most recent government estimate, more than 25% of those under age 30 use only a cell phone. An analysis of young people ages 18-25 in one of the Pew polls found that the exclusion of the cell-only respondents resulted in significantly lower estimates of this age group's approval of alcohol consumption and marijuana use. Perhaps not surprisingly, excluding the cell-only respondents also yields lower estimates of technological sophistication. For example, the overall estimate for the proportion of 18-25 year olds using social networking sites is 57% when the cell-only sample is blended with the landline sample, while the estimate based only on the landline sample is 50%.

Including a cell-only sample with a traditional landline-based poll is feasible, as the four studies conducted last year indicate. But even if feasible, cell-only surveys are considerably more difficult and expensive to conduct than landline surveys. Federal law prohibits the use of automated dialing devices when calling cell phones so each number in the cell phone sample must be dialed manually. It also is common practice to provide respondents with a small monetary incentive to offset the cost of the airtime used during the interview. And the screening necessary to reach cell-only respondents among all of those reached on a cell phone greatly increases the effort needed to complete a given number of interviews. Pew estimates that interviewing a cell-only respondent costs approximately four to five times as much as a landline respondent.

Pollsters recognize that some type of accommodation for the cell-only population will have to be made eventually, as was clear from the large amount of research on the topic presented at the AAPOR conference last month. In addition to the use of so-called "dual frame samples" such as those described above (calling both a cell phone sample and a landline sample), practitioners are discussing other alternatives, including the establishment of panels of cell-only respondents that can be surveyed periodically to track their opinions, and employing mail or internet surveys to reach the cell-only population.

This article draws on research presented at the AAPOR conference, "What's Missing from National RDD Surveys? The Impact of the Growing Cell-Only Population," by Scott Keeter (Pew Research Center), Courtney Kennedy (University of Michigan and Pew Research Center), April Clark (Pew Research Center), Trevor Tompson (The Associated Press), and Mike Mokrzycki (The Associated Press).

1Adding in the 2.2% of households with no phone service whatsoever, a total of 15.0% cannot be reached by landline surveys. The government report is by Stephen J. Blumberg and Julian V. Luke. "Wireless Substitution: Early Release of Estimates Based on the National Health Interview Survey, July – December 2006." Report by the U.S. Centers for Disease Control and Prevention, May 14, 2007.

2Details about the four studies can be found at the following links: "The Cell Phone Challenge to Survey Research"; "Online Papers Modestly Boost Newspaper Readership"; "A Portrait of Generation Next"; and "Cell-Only Voters Not Very Different."

There is a large and growing class of cell only users they forgot. The full time RVers. There are a few million of us now. Some own houses and some don’t; I don’t.

I work in Houston and have worked in five states in five years. I have a Corpus Christ number and the wife has a ‘Vegas number because that is where we are from and have family & friends.

A large chunk of the full timers are construction, especially industrial construction types. We tend to be gun tote’n physical conservatives...

I said they were confused and one thought I was giving a false address hence thought I was lying.

First, what you say you are doing is arguing. Second, why do you care what a telemarketer says or thinks. Don't argue (go look the word up since you don't know what it means). Just hang up on the offensive twits.

It is because people like you that telemarketers think it is worth their time to bother people like me.

WOW

speechless

Because I believe in treating people like I would like to be treated.

I believe in respect to others and I don’t start arguments with people I do not know or accuse people or something.

I don’t call people names

I look at it that they are doing this because they need the money and that is good in my book instead of sponging off the Govt

I teach my kids the same thing for if we all treated each other better than this world would be better off instead of having gobshites bigging themselves up over a phone or the internet.

I believe in treating people as if I was face to face not being cyber brave behind the safety of my keyboard or phone

As for my fault telemarketers calling you , well

whatever.

You think me being rude to telemarketers will stop calling you.

yea OK

And for the record it was my call I know what was said and no arguments were done.
You were not there so therefore you do not know.

My husband and I, in our 50’s, use cell phone exclusively. We haven’t had a landline for over a year. Neither of us will answer an unknown caller.

Whenever the Dems are polling badly the Left always screams about how cell phone users, who they think are all in their 20’s, are not being polled.

I think young and old and in between hesitate to answer unknown callers, cell or landline.

You need to find a dictionary and look up the word argue, since you seem to think that it means having a heated altercation. A discussion where you politely correct someone's incorrect information is an argument.

And now I know why telemarketing works. I always considered it an affront to use my equipment to invade my privacy and take up my time uninvited by me. Apparently there are saps like you who think it is just ok.

I had the conversation you never and yet you think you know how it went, strange indeed.
second it was not heated and I have not said heated you have.

As for the name calling, well just looking at your past posts to others I can only come to the conclusion that you seem to have a bad attitude or you seem angry at the world.

As for thinking it is OK well I think it is great that people are working and making money don’t you?
It’s a job they have annoying to you yes but at least they have a job and are working.
They are not taking your tax money because they are lazy

They may be annoying and that is your choice my choice is to be polite and show respect .
I would not like to be treated rudely so I would not treat them rudely.
Maybe next time don’t answer or just simply say I am not in the mood or I am busy please don’t call me back either.

My kids know this too and the youngest is 5.

Something for you to ponder.

o and if you look at it as uninvited then are you rude to those who get the wrong number as they were never asked to call you either.

I can only come to the conclusion that you seem to have a bad attitude or you seem angry at the world.

Not the least. My principal issue is that a lot of our problems in this country come from substituting sloppy sentimentality for clear thinking, as you have just done. When I call folks idiots, it is to grab their attention and point out to them that they are not using their brains. They are not arguing a point from facts that we can verify and clear logic to reach a conclusion that we can first undrestand and then second accept, reject, or improve upon.

The fact is that you and others need to start thinking about what you say and the consequence for others of what you do.

As a rhetorical question, did you, when you elected to instruct me on the art of being polite to those who invade my privacy using my property paid for with my money, that telemarketing is now largely an offshore business, call centers in India backed up by random dialing machines to pick on anyone who will answer. So it is not even your fellow Americans you are supporting in your polite addresses to those who invade your home using your telephone and your kind manners to do so.

I am not particularly polite to mice and cockroaches that attempt to invade my house either. It is this my home is my castle sort of principle that we seem to have abandoned long ago.

then are you rude to those who get the wrong number as they were never asked to call you either.

As Judge Souter said famously in the 2nd review of the Fl SC decisions on chads, "even a dog knows the difference between being tripped over and being kicked." Someone who has the wrong number is innocently using my phone. A neighbor with an inquiry or concern is well meaning.

Someone who calls me demands to know my name and then tries to tell me I am his best friend and I only need to tell him all of my personal information before I can get a free trip to Miami for a mere cost of $2000 for my partner's ticket, is a fraud a cheat and a swindler and deserves to have the phone slammed down on him accompanied by a stream of unrepeatable epithets.

And idiots like you thinking it is right to be polite to those kinds of folks only encourage them to call me.

And in pondering my words bear Souter's observation in mind, and in future try thinking with thinking part of your grey matter. I am not trying to molly-coddle you. I am trying to induce you to think for a change. The world will be a far better place for it.

And finally, to get back to the original point before you diverted it, I have no clue what purpose you thought you were serving by discussing the fact with a telemarketer that the information he had on where you lived was incorrect. From the sounds of it, other than trying to be polite, you have no clue either.

I have no clue what purpose you thought you were serving by discussing the fact with a telemarketer that the information he had on where you lived was incorrect. From the sounds of it, other than trying to be polite, you have no clue either.

you said clue twice

First you admit you have no clue and then you say I have no clue.

Then if you had no clue what purpose I was bringing up why get involved in the subject with me, why jump in with name calling and thinking you know how my conversation went?
Bizarre or know it all.

Then you say I have no clue.

OK then.

Indian centers.
Ah I bought a HP laptop and had to endure for 4 months an
Indian call center but even though it was frustrating I knew that they were doing their job and it wasn’t their fault for not being able to speak English that well.

It was HP’s fault getting cheaper labour.

I do not take my frustration out of someone who is not at fault nor do I jump into conversations I do not have a clue about and then assume I know all about what they are saying

Out of curiosity if you have to talk to an Indian call center then are you rude to them?

see this is where you and I differ the way I get peoples attention is be getting my point across in a civilised manner and not by calling people names.

You see when you call people names and I do wonder if you would do it face to face instead of being behind a keyboard in the safety of your home then it doesn’t start of a debate or conversation to well does it?

I’ll be honest if someone is wanting to call me a name face to face then he is disrespecting me and I do not just stand there and listen to that.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.