Evaluation of ARRA Comparative Effectiveness Research Dissemination Contract Efforts

Throughout the 1990s, landline surveys were the dominant form of data collection for telephone survey research.²³ However, research shows that the increase in the "cell-phone only" population is particularly skewed towards younger adults, which has led to difficulties in recruiting nationally representative samples. The proportion of the population forgoing a landline in favor of cell phones has been steadily rising.²⁴

A report based on the National Health Interview Survey (NHIS) data collected between July-December 2011 illustrated that 34 percent of American homes were wireless only and 16 percent were wireless mostly (Exhibit G-1, below). It furthermore found that age is highly correlated to the distribution of wireless only or wirelesses mostly households. NHIS data found that the percent (by age groups) that lived in households with only a wireless telephone are as follows:

Exhibit G-1: Percent of Households with Only Wireless (Between July-December 2011)

Other national level surveys, including those from the American Association for Public Opinion Research (AAPOR), have similarly noted a steady decline in telephone survey participation rates of younger generations. PEW Research Centers have experienced a decline in the average proportion of 18-34 year olds from 31 percent in 2000 to 20 percent in 2006 (PEW 2006). This upward trend in the prevalence of "cell-phone only" households has presented unique challenges to telephone survey researchers of all sizes.

Survey researchers are struggling with this issue on a large scale and developing field procedures to overcome this obstacle. We are being proactive in this area and we are not alone in dealing with this issue. Numerous survey centers including the PEW Research Center, AAPOR, and The Behavioral Risk Factor Surveillance System (the world’s largest ongoing, list-assisted random digit dialing telephone-interview health survey system) have realized the need to implement dual approach telephone surveys which include a cell phone sample to account for this and other potential differences in the cell-phone only population. This method has been shown to have a positive effect on evening out the response rates across age groups. In one PEW study conducted in 2006, it was found that 48 percent of the cell-phone survey participants were under the age of 30 (compared with 14 percent in the landline sample and 21 percent in the population as a whole); 41 percent of the landline sample respondents were age 65 or older (compared with 16 percent of the general public).

Given these data, IMPAQ initiated a discussion with AHRQ around including a new cell phone wave in its survey sample for the ARRA CER dissemination survey. For the first consumer survey, AHRQ determined that the age groups completing the consumer telephone survey were appropriate and that the evaluation contractor need not initiate a separate cell phone wave.

²³ Vicente, P., Reis, E., & Santos, M. (2009). Using mobile phones for survey research. International Journal of Market Research 51(5), 613-633.

²⁴ Raine L, Keeter S. "Americans and their cell phones." Pew Internet & American Life Project. Pew Research Center. April 3, 2006. Go to http://www.pewinternet.org/Reports/2006/Americans-and-their-cell-phones.aspx.

Return to Contents

Proceed to Next Section