Mitigating Fraud in Incentivized Online Surveys: Lessons from Facebook Recruitment in Nigeria

Arnim Langer KU Leuven, Belgium
Lucas Leopold KU Leuven, Belgium
Bart Meuleman KU Leuven, Belgium & University of Bremen, Germany
Line Kuppens University of Amsterdam, The Netherlands
Abhishegh Menon KU Leuven, Belgium


In this paper we document our experience with conducting the Perceptions of Inequality and Redistribution Survey (PIRS), which is an online, multi-round survey conducted among Nigerian adults who were recruited via a Facebook ad campaign. The paper discusses our initial and unsuccessful recruitment strategy as well as our subsequent revised strategy in which we were able to effectively address the problems and pitfalls which caused our initial recruitment strategy to be unsuccessful. By discussing both strategies in detail, we provide original methodological and practical insights into how best to conduct online surveys with compensation in low- and middle-income countries (LMICs) …


, , , ,

No Comments

Examining Predictors of Unsuccessful Mail Delivery for a National Address-Based Sample During the COVID-19 Pandemic

Raphael Nishimura University of Michigan, USA
Rebecca Gatward - University of Michigan, USA
Brady T. West University of Michigan, USA
Htay-Wah Saw University of Michigan, USA


Many surveys have changed modes because of the COVID-19 pandemic. Surveys using mailed letters and/or reminders may have been adversely affected by staff shortages affecting the delivery times of the United States Postal Service (USPS). In this study, we analysed daily USPS mail delivery reports for each of 21 National Distribution Centers (NDCs) for two specific mailings that were sent to a national probability sample of addresses in 2020. We linked concurrent aggregate rates of COVID-19 infection and mortality to each of the NDCs and examined the variability in cumulative delivery rates across the NDCs as a function of these …


, , , ,

No Comments

Does matching interviewer and respondent gender improve data quality and reduce social desirability bias? Evidence from a mobile phone survey in India

Payal Hathi University of California, Berkeley, USA
Diane Coffey University of Texas at Austin, USA
Amit Thorat Jawaharlal Nehru University, India
Amanda Nagle University of Texas at Austin, USA


Gender matching between interviewers and respondents has been used to improve survey quality. Research on interviewer gender effects from the United States and Europe finds that men respondents are sometimes more likely to report conservative attitudes to men interviewers than women interviewers. This study from a phone survey in India does not find that men report significantly different social attitudes to men and women interviewers. This suggests that in places where conservative and patriarchal social attitudes are widespread and accepted, common phenomena such as social desirability and gender performance may operate differently than in western contexts. We additionally find that …


, , , ,

No Comments

Who chooses a QR code over a URL to access a web screener in a national probability survey of older adults, and the impact on data quality

Andrew L. Hupp University of Michigan, USA
Heather M. Schroeder University of Michigan, USA
Brady T. West University of Michigan, USA
Eva Leissou - University of Michigan, USA
David R. Weir University of Michigan, USA


There has been limited research on the impact of the inclusion of a quick response code (QR) code option on recruitment to a web screening survey, and on data quality when a screening instrument can be either self-administered or interviewer-administered. As part of a screening effort to recruit a new birth cohort to a long-standing panel study, an area probability sample of households was randomly assigned to one of two data collection protocols: web-first with in-person follow-up, or in-person first, with nonresponding cases followed up via the web. Both treatment arm protocols included a mailed letter, either as an invitation …


, , , ,

No Comments

Effects of mode and incentives on response rate, sample composition, and costs – experience from a self-administered mixed-mode survey of movers

Manuela Schmidt University of Bonn, Germany
Alice Barth University of Bonn, Germany
Jörg Blasius - University of Bonn, Germany


Using incentives to reduce unit nonresponse in surveys is an established practice, with prepaid incentives being particularly effective in increasing participation rates. In this paper, we investigate how incentives affect participation rates and sample composition in an offline-recruited mixed-mode survey (concurrent paper/web and push-to-web), using an experimental design. We conducted a survey on residential satisfaction and reasons for moving place, sampling residents who moved away from two urban neighborhoods in Cologne, Germany, between 2018 and 2022. Addresses were provided by Cologne’s Office for Urban Development and Statistics, including information on the residents’ age and gender. Of a random sample of …


, , , ,

No Comments

Is it Helpful to Include QR Codes on Mail Contact Materials for Self-Administered Web Surveys?

Taylor Lewis RTI International, USA
Nicole Lee - RTI International, USA
Dain Palmer RTI International, USA
Hannah Matzke Chicago Department of Public Health, USA
Nik Prachand - Chicago Department of Public Health, USA


The Healthy Chicago Survey (HCS) is a self-administered, multimode survey conducted annually on a random sample of Chicagoans aged 18 years or older. The survey uses an address-based sampling frame and offers both web and paper modes, with a Choice+ data collection protocol offering a higher promised incentive for completing by web. Overall, about 90% of individuals take the HCS via web. Sampled households are sent a series of mailings inviting the adult who is next to celebrate a birthday to complete the survey. In the 2023 HCS administration, an experiment was fielded in which half of the mailings contained …


, , , ,

No Comments

The Effect of Operationalization on the Correlation between Objective and Subjective Socioeconomic Status: A Systematic Review and Meta-Analysis

Luise Richter - Dresden University of Technology, Germany
Natalja Menold - Dresden University of Technology, Germany


Socioeconomic status (SES) plays a fundamental role in social science research and beyond, serving as an important indicator for understanding societal structures and social inequalities. Measuring SES involves diverse approaches, including objective and subjective measures, raising questions about their relationship and the factors influencing its size. This study investigates the correlation between objective and subjective socioeconomic status (OSS and SSS) and evaluates the effect of study context, study design, and operationalization of the measurement on this correlation. We conducted a meta-analysis, starting with a systematic literature search of relevant databases up to July 2021. Multilevel random effects models were employed …


, , , ,

No Comments

Using surveys to assess students’ political knowledge: Evidence of a gender gap or disparate response styles?

Christa E. Winkler - Mississippi State University, Department of Counseling, Higher Education Leadership, Educational Psychology, and Foundations, Starkville, Mississippi, USA


The aims of public education often include preparing students to be actively engaged in the civic and political realms of society. However, research findings have consistently provided evidence of a gender gap in political knowledge, with men demonstrating significantly higher levels of political knowledge than women. Recently, scholars have begun questioning the validity of such findings due to the nature of the measures used to assess political knowledge. This study applies item response theory, specifically IRTree modeling, to evaluate the extent to which a reported gender gap in high school students’ political knowledge can be attributed to survey response styles. …


, , , ,

No Comments

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 4.0 International License. Creative Commons License