Web survey experiments on fully balanced, minimally balanced and unbalanced rating scales

Mingnan Liu, SurveyMonkey, Palo Alto, California, U.S.A.
Sarah Cho, SurveyMonkey, Palo Alto, California, U.S.A.


When asking attitudinal questions with dichotomous and mutually exclusive response options, the questions can be presented in one of three ways: a full balanced question, a minimally balanced question, and an unbalanced question. Although previous research has compared the fully vs. minimally balanced rating scales, as far as we know, these three types of rating scales have not been tested in a strict experimental setting. In this study, we report two web survey experiments testing these three types of rating scales among 16 different questions. Different from most previous studies, this study used visual display only without any auditory component. …


, , ,

No Comments

Comparing smartphones to tablets for face-to-face interviewing in Kenya

Sarah M. Hughes, Mathematica Policy Research, U.S.
Samuel Haddaway, Yale School of Management, U.S.
Hanzhi Zhou, Mathematica Policy Research, U.S.


Research conducted over the past 30 years has demonstrated a reduction in errors and improvement in data quality when face–to-face social surveys are carried out using computers instead of paper and pencil. However, research examining the quality of data collected by interviewers using mobile devices is in its infancy and is based in developed countries. In a small pilot study conducted during the World Bank’s Kenya State of the Cities Baseline Survey, a face-to-face survey on living conditions, infrastructure and service delivery, the authors compared the quality of data collected using smartphones to data collected using tablets. The study of …


, , , , ,

No Comments

Altering the Survey-taking Climate: The Case of the 2010 U.S. Census

Ting Yan (Westat) Maryland, U.S.
A. Rupa Datta (NORC at the University of Chicago), Illinois U.S.


Response rates to household surveys have been declining in the past several decades and survey researchers and practitioners have been working on ways to change the survey-taking climate to combat the declining response rates. As part of the 2010 Decennial Census, the U.S. Census Bureau waged the 2010 Integrated Communications Campaign (2010 ICC), a multi-faceted effort to improve public awareness of, attitudes towards, and knowledge about the Census in order to increase Census participation. This type of communications program is a unique case of an attempt to alter the external survey-taking climate and thus potentially affect survey participation. This paper …


, ,

No Comments

What do web survey panel respondents answer when asked “Do you have any other comment?”

Matthias Schonlau, University of Waterloo, Statistics and Actuarial Science, Canada


Near the end of a web survey respondents are often asked whether they have additional comments. Such final comments are usually ignored, partially because open-ended questions are more challenging to analyze. A random sample of final comments in the LISS panel and Dutch immigrant panel were categorized into one of nine categories (neutral, positive, multiple subcategories of negative). While few respondents chose to make a final comment, this is more common in the Immigrant panel (5.7%) than in the LISS panel (3.6%). In both panels there are slightly more neutral than negative comments, and very few positive comments. The number …


, ,

No Comments

Obtaining Record Linkage Consent: Results from a Wording Experiment in Germany

Joseph W. Sakshaug, University of Manchester and Institute for Employment Research, Nuremberg, Germany
Stefanie Wolter, Institute for Employment Research, Nuremberg, Germany
Frauke Kreuter, University of Maryland, University of Mannheim, and Institute for Employment Research, Nuremberg, Germany


Many sample surveys ask respondents for consent to link their survey information with administrative sources. There is significant variation in how linkage requests are administered and little experimental evidence to suggest which approaches are useful for achieving high consent rates. A common approach is to emphasize the positive benefits of linkage to respondents. However, some evidence suggests that emphasizing the negative consequences of not consenting to linkage is a more effective strategy. To further examine this issue, we conducted a gain-loss framing experiment in which we emphasized the benefit (gain) of linking or the negative consequence (loss) of not linking …


, , , ,

No Comments

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 4.0 International License. Creative Commons License