OJPHI: Vol. 5
Journal Information
Journal ID (publisher-id): OJPHI
ISSN: 1947-2579
Publisher: University of Illinois at Chicago Library
Article Information
©2013 the author(s)
open-access: This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes.
Electronic publication date: Day: 4 Month: 4 Year: 2013
collection publication date: Year: 2013
Volume: 5E-location ID: e24
Publisher Id: ojphi-05-24

An Experimental Study Using Opt-in Internet Panel Surveys for Behavioral Health Surveillance
Carol A. Gotway Crawford*1
Catherine A. Okoro1
Haci M. Akcin1
Satvinder Dhingra21
1Centers for Disease Control, Atlanta, GA, USA;
2Northrop Grumman, Atlanta, GA, USA
*Carol A. Gotway Crawford, E-mail: cdg7@cdc.gov


To present the design and preliminary results of a pilot study to investigate the use of opt-in Internet panel surveys for behavioral health surveillance.


Today, surveyors in both the private and public sectors are facing considerable challenges with random digit dialed (RDD) landline telephone samples. The population coverage rates for landline telephone surveys are being eroded by wireless-only households, portable telephone numbers, telecommunication barriers (e.g., call forwarding, call blocking and pager connections), technological barriers (call-blocking, busy circuits) and increased refusal rates and privacy concerns. Addressing these issues increasingly drives up the costs associated with dual-frame telephone surveys designed to be representative of the target population as well as hinders their ability to be fully representative of the adult population of each state and territory in the United States.

In an effort to continue to meet these challenges head on and assist state and territorial public health professionals in the continued collection of data that are representative of their respective populations, novel approaches to behavioral health surveillance need continued examination. Both private and public sector researchers are evaluating the use of Internet opt-in panels to augment dual-frame RDD survey methods. Compared to dual-frame RDD, opt-in Internet panels offer lower costs, quick data collection and dissemination, and the ability to gather additional data on panelists over time. However, as with dual-frame RDD, this mode has similar challenges with coverage error and non-response. Nevertheless, survey methodologists are moving forward and exploring ways to reduce or eliminate biases between the sample and the target population.


A collaborative pilot project was designed to assess the feasibility and accuracy of opt-in Internet panel surveys for behavioral health surveillance. This pilot project is a collaboration between the CDC, four state departments of health, opt-in Internet panel providers and the leads of several large surveys and systems such as the Patient-Reported Outcome Measures Information System (PROMIS) and the Cooperative Congressional Election Study (CCES). Pilot projects were conducted in four states (GA, IL, NY, and TX) and four Metropolitan Statistical Areas (Atlanta, Chicago, New York City, and Houston). Data were collected using three different opt-in Internet panels and sampling methods that differ with respect to recruitment strategy, sample selection and sample matching to the adult population of each geography. A question bank consisting of 80 questions was developed to benchmark with other existing surveys used to assess various public health surveillance measures (e.g., the Behavioral Risk Factor Surveillance System, the PROMIS, National Survey on Drug Use and Health, and the CCES).


We present comparative analyses that assess the advantages and disadvantages of different opt-in Internet panels sampling methodologies across a range of parameters including cost, geography, timeliness, usability, and ease of use for technology transfer to states and local communities. Recommendations for future efforts in behavioral health surveillance are given based on these results.

Ansolabehere, S. Schaffner, BF. 2010Re-examining the validity of different survey modes for measuring public opinion in the U.S.: Findings from a 2010 multi-mode comparisonhttp://projects.iq.harvard.edu/cces/files/ansolabehere_schaffner_mode.pdf.
Curtin R, Presser S, Singer E. 2005;Changes in telephone survey nonresponse over the past quarter centuryPublic Opinion Quarterly 69(1):87–98.
Liu H, Cella D, Gershon R, Shen J, Morales LS, Riley W, Hays RD. 2010;Representativeness of the Patient-Reported Outcomes Measurement Information System Internet panelJ Clin Epidemiol 63(11):1169–78.
Rivers D. 2007Sampling for Web Surveys Paper presented at the Joint Statistical Meetings. http://www.laits.utexas.edu/txp_media/html/poll/files/Rivers_matching.pdf

Article Categories:
  • ISDS 2012 Conference Abstracts

Keywords: Random Digit Dialing, BRFSS, Survey Methods, Sample Matching, Representativeness.

Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org