The Interviewer Performance Profile (IPP): A Paradata-Driven Tool for Monitoring and Managing Interviewer Performance

Heidi M. Guyer, RTI International, Research Triangle Park, NC, USA
Brady T. West, Survey Research Center, University of Michigan, Ann Arbor, MI, USA
Wen Chang, Survey Research Center, University of Michigan, Ann Arbor, MI, USA


Monitoring interviewer performance during data collection is essential for ensuring optimal performance at the interviewer level, thereby leading to improved data collection parameters in areas such as effort and efficiency. Paradata are widely utilized to enhance typical measures of performance, such as response rates, and provide a more nuanced view of interviewer performance characteristics. We describe a paradata tool developed to monitor interviewer performance in real time: the Interviewer Performance Profile (IPP). Daily data updates allow for ongoing monitoring to detect areas of performance concern as well as areas of improvement over time. Additionally, the flexible nature of the IPP …


, , , ,

No Comments

Integrating online data collection in a household panel study: effects on second-wave participation

Marieke Voorpostel, FORS (Swiss Centre of Expertise in the Social Sciences), Switzerland
Caroline Roberts, University of Lausanne, Switzerland
Margarita Ghoorbin, University of Lausanne, Switzerland


Received wisdom in survey practice suggests that using web mode in the first wave of a panel study is not as effective as using interviewers. Based on data from a two-wave mode experiment for the Swiss Household Panel (SHP), this study examines how the use of online data collection in the first wave affects participation in the second wave, and if so, who is affected. The experiment compared the traditional SHP design of telephone interviewing to a mixed-mode design combining a household questionnaire by telephone with individual questionnaires by web and to a web-only design for the household and individual …


, , ,

No Comments

Sequentially mixing modes in an election survey

Oliver Lipps, FORS (Swiss Centre of Expertise in the Social Sciences) and University of Bern, Switzerland
Nicolas Pekari, FORS (Swiss Centre of Expertise in the Social Sciences), Switzerland


In a sequential mixed mode survey, instead of offering different modes from the onset, the data collection usually starts with an inexpensive mode for the whole sample, followed by more expensive modes to attract non-respondents. In this paper, we analyze to what extent this type of design, consisting in this case of web, telephone and mail-paper modes, is able to improve representativeness in terms of socio-demographic variables and reduce bias in terms of voting behavior compared to a single mode survey. In addition, we study whether changes in mode lead to measurement error issues by focusing on income. We find …


, , , , , ,

No Comments

Shaking hands in a busy waiting room. The effects of the surveyor’s introduction and people present in the waiting room on the response rate

Yfke P. Ongena, Centre for Language and Cognition, University of Groningen, Groningen, The Netherlands
Marieke Haan, Department of Sociology, University of Groningen, Groningen, The Netherlands
Thomas C. Kwee, Department of Radiology, Medical Imaging Center, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
Derya Yakar, Department of Radiology, Medical Imaging Center, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands


Although waiting room surveys are frequently conducted, methodological studies on this topic are scarce. Behaviour of surveyors in waiting rooms can easily be controlled, and these surveys also allow for collection of paradata; relevant information on the circumstances of a request to participate in survey research. In this paper, we present the results of an experiment systematically manipulating surveyor’s handshakes and verbal introduction of their names. Patients scheduled for radiological examinations were approached to take part in a survey. An observer noted circumstances in the waiting room (CT or MRI) such as the number of people present. In the CT …


, , ,

No Comments

Do shorter stated survey length and inclusion of a QR code in an invitation letter lead to better response rates?

Peter Lugtig, Utrecht University, The Netherlands
Annemieke Luiten, Statistics Netherlands, The Netherlands


Invitation letters to web surveys often contain information on how long it will take to complete a web survey. When the stated length in an invitation of a survey is short, it could help to convince respondents to participate in the survey. When it is long respondents may choose not to participate, and when the actual length is longer than the stated length there may be a risk of dropout. This paper reports on an Randomised Control Trial (RCT) conducted in a cross-sectional survey conducted in the Netherlands. The RCT included different version of the stated length of a survey …


, , , , ,

No Comments

Multi-mode question pretesting: Using traditional cognitive interviews and online testing as complementary methods Special issue

Robin L. Kaplan, U.S. Bureau of Labor Statistics, BLS, USA
Jennifer Edgar, U.S. Bureau of Labor Statistics, BLS, USA


Questionnaire development, evaluation, and pretesting research is critical for ensuring that survey questions, materials, and data collection procedures produce the highest quality data possible. Interviewer-administered cognitive interviews is a common pretesting method used to collect rich, qualitative data. As technology has advanced, researchers can conduct similar research online in self-administered modes (Behr 2016), allowing for pretesting with larger samples. Each approach has strengths and limitations that researchers can leverage to address their pretesting goals. This research presents a multi-study, iterative project using traditional and online pretesting to evaluate new confidentiality language. Study 1 used traditional cognitive interviews to collect information …


, , , , ,

No Comments

Using Geofences to Collect Survey Data: Lessons Learned From the IAB-SMART Study Special issue

Haas, Georg-Christoph, Institute for Employment Research, University of Mannheim, Germany
Trappmann, Mark, Institute for Employment Research; University of Bamberg, Germany
Keusch, Florian, University of Mannheim, Germany
Bähr, Sebastian, Institute for Employment Research, Germany
Kreuter, Frauke, Institute for Employment Research, Germany; University of Mannheim, Germany; University of Maryland, USA


Within the survey context, a geofence can be defined as a geographical area that triggers a survey invitation when an individual enters the area, dwells in the area for a defined amount of time or exits the area. Geofences may be used to administer context-specific surveys, such as an evaluation survey of a shopping experience at a specific retail location. While geofencing is already used in other contexts (e.g., marketing and retail), this technology seems so far to be underutilized in survey research. We implemented a geofence survey in a smartphone data collection project and geofenced 410 job centers with …


, ,

No Comments

Collecting and using always-on location data in surveys Special issue

Stephanie Eckman, RTI International, USA
Rob Chew, RTI International, USA
Herschel Sanders, RTI International, USA
Robert Furberg, RTI International, USA


As survey costs increase and response rates decrease, researchers are looking for alternative methods to collect data from study subjects. Passively collected data may offer a way to reduce the burden on research subjects while also collecting high-quality data needed for social science research. Examples of passive data collection tools are applications installed on mobile devices and sensors in subjects’ homes or worn on the body. In this study, we focus on always-on location data collected from subjects’ iPhones. To explore the promise of passively collected data to augment and improve survey data, we conducted a 2-week pilot study with …


, ,

No Comments

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 4.0 International License. Creative Commons License