Special issue: ‘Probability and Nonprobability Sampling: Sampling of hard-to-reach survey populations’

Editors: Johann Bacher, Johannes Lemcke, Andreas Quatember & Patrick Schmich


With the following collection of articles, Survey Methods: Insights from the Field’ aims to give an overview about the current state of the hard-to-reach research and the ongoing dispute between the two above mentioned sampling methods. Moreover, this Special Issue attempts to combine theoretical discussions, methodological considerations with experiences from the fields. It offers insights into possible links between non-probability sampling and hard-to-reach populations on the one hand, and, on the other hand, different approaches to address the aforementioned problems via the praxis of each methodology.

This special issue was inspired by a PUMA-Symposium 2017, which was organized by two of the guest editors of  the Johannes Kepler University Linz (JKU), Johann Bacher and Andreas Quatember, within the PUMA-project of the Austrian social sciences. In this context, different sampling issues and different solution attempts were discussed. The editorship by two researchers of the Robert Koch Institute (RKI), Johannes Lemcke und Patrick Schmich, was guided by the Institute´s interest to a better integration of the hard-to-reach groups into its health monitoring system. The RKI, as the public health institute in Germany, has the obligation to monitor the health status of the whole population in order to offer policy makers valid information within the decision process. As a result of this commitment, various feasibility studies were carried out, which are presented in this special issue alongside initial results. In this respect the inclusion of elderly and people with migration background is a crucial task for the RKI.

The contributions can be divided into four groups

Of course, the assignment of a paper to a certain group is not distinct in every case. Nonetheless, the below grouping should provide some guidance.

  1. Two papers discuss the fundamental theoretical aspects of our topic.
  2. Two papers provide a comparison of probability and non-probability sampling from an applied perspective.
  3. Reports about experiences of concrete studies build the majority of the paper. Nine papers belong to this group. However, these are not only reports in a strict sense; they provide literature reviews, describe the designs and the underlying assumptions and reflect on the design. They cover migrants and refugees as hard-to-reach-groups mentioned by Willis et al. (2014) as well the elderly as one group sometimes ignored.
  4. The fourth group contains papers that can be labeled as “reflections and methodological proposals”. One paper suggests a model to increase the recruitment of old people; the second one applies simulations methods.

 

Probability and Nonprobability Sampling: Representative Surveys of hard-to-reach and hard-to-ask populations. Current surveys between the poles of theory and practice Special issue

Johann Bacher, Johannes Kepler University (JKU) Linz, Austria
Johannes Lemcke, Robert Koch Institut (RKI), Germany
Andreas Quatember, Johannes Kepler University (JKU) Linz, Austria
Patrick Schmich, Robert Koch Institut (RKI), Germany


Inferences based on Probability Sampling or Nonprobability Sampling – Are They Nothing but a Question of Models? Special issue

Andreas Quatember, Johannes Kepler University (JKU) Linz, Austria


The inferential quality of an available data set, be it from a probability sample or a nonprobability sample, is discussed under the standard of the representativeness of a sample with regard to interesting characteristics, which implicitly includes the consideration of the total survey error. The paper focuses on the assumptions that are made when calculating an estimator of a certain population characteristic using a specific sampling method, and on the model-based repair methods, which can be applied in the case of deviations from these assumptions. The different implicit assumptions regarding operationalization, frame, selection method, nonresponse, measurement, and data processing are …


, , , ,

No Comments

Possible Uses of Nonprobability Sampling for the Social Sciences Special issue

Ulrich Kohler, Faculty for Economics and the Social Sciences, University of Potsdam, Germany


This paper compares the usability of data stemming from probability sampling with data stemming from nonprobability sampling. It develops six research scenarios that differ in their research goals and assumptions about the data generating process. It is shown that inferences from data stemming from nonprobability sampling implies demanding assumptions on the homogeneity of the units being studied. Researchers who are not willing to pose these assumptions are generally better off using data from probability sampling, regardless of the amount of nonresponse. However, even in cases when data from probability sampling is clearly advertised, data stemming from nonprobability sampling may contribute …


, , , , , ,

No Comments

Measuring sexual behaviours and attitudes in hard-to-reach groups. A comparison of a non-probability web survey with a national probability sample survey Special issue

Rebecca S Geary, University College London, London, UK
Mick P Couper, University of Michigan, Ann Arbor, MI, USA
Bob Erens, London School of Hygiene and Tropical Medicine, London, UK
Andrew J Copas, University College London, London, UK
Sarah Burkill, University College London, London, UK
Pam Sonnenberg, University College London, London, UK
Frederick Conrad, University of Michigan, Ann Arbor, MI, USA
Catherine H Mercer, University College London, London, UK


Introduction: Hard-to-reach and minority groups are often at higher risk for adverse sexual health outcomes. While such groups are therefore of interest to sexual health researchers, it can be difficult to locate and recruit sufficient sample sizes using probability sampling methods. This study aims to establish whether web-panel surveys can provide a viable less resource intensive means of boosting sample sizes of two hard-to-reach groups (people of Black African ethnicity, and gay men) for a sexual health survey, and the extent of any bias. Methods: Results from a national probability sample survey (Natsal-3, administered using a computer-assisted personal interview (CAPI) …


, , , ,

No Comments

Needles in Haystacks and Diamonds in the Rough: Using Probability and Nonprobability Methods to Survey Low-incidence Populations Special issue

Ellen L. Marks, RTI International (formerly)
Bryan B. Rhodes, RTI International


While probability samples are generally the preferred approach in survey research, nonprobability samples continue to be of interest and are used for multiple purposes. We discuss the use of a convenience sample in one study and social media recruitment in another when probability-based samples fell short of reaching target sample sizes for low-incidence populations. Both studies collected rich survey data, particularly regarding household finances, enabling comparisons between respondents in the two types of samples. Analysis shows that while demographic characteristics were not consistent across the two types of samples, the source of the data—the probability sample or the nonprobability sample—was …


, , , , ,

No Comments

From peer to peer: Reaching migrants from sub-Saharan Africa with research on sexual health utilizing community-based participatory health research Special issue

Carmen Koschollek, Robert Koch Institute (RKI), Berlin, Germany
Anna Kuehne, Robert Koch Institute (RKI), Berlin, Germany
Stephen Amoah, Afrikaherz Berlin, Verband für interkulturelle Arbeit, Regionalverband Berlin/Brandenburg e.V., Berlin, Germany and and Institute for Social Medicine, Epidemiology and Health Economics, Charité -Universitaetsmedizin Berlin, Berlin, Germany
Helene Batemona-Abeke, Pamoja Afrika e.V., Cologne, Germany
Taty Dela Bursi, Hannöversche Aids-Hilfe e.V., Hanover, Germany
Pierre Mayamba, Aidshilfe Essen e.V., Essen, Germany
Rosaline M‘bayo, Afrikaherz Berlin, Verband für interkulturelle Arbeit, Regionalverband Berlin/Brandenburg e.V., Berlin, Germany
Christina Mputu Tshibadi, Münchner Aids-Hilfe e.V., Munich, Germany
Garnet Parris, Maisha e.V., Frankfurt am Main, Germany
Adama Thorlie, Berliner Aids-Hilfe e.V., Berlin, Germany
Virginia Wangare Greiner, Maisha e.V., Frankfurt am Main, Germany
Viviane Bremer, Robert Koch Institute (RKI), Berlin, Germany
Claudia Santos-Hövener, Robert Koch Institute (RKI), Berlin, Germany


Migrants from sub-Saharan Africa (misSA) in Germany are disproportionally affected by HIV. To develop targeted interventions, it is necessary to collect data on knowledge, attitudes, behaviour and practices (KABP) regarding HIV and sexual health. However, misSA are difficult to reach and to sample: a) it is unknown how many people with an African migration background are living in Germany, and b) HIV and sexual health topics are highly stigmatized in these communities. We utilized a community-based participatory health research approach to develop a study protocol and conducted a KABP survey on HIV and sexual health among misSA in six German …


, , , , ,

No Comments

Integration of Migrant Populations into Health Monitoring in Germany: Results from a Feasibility Study Special issue

Marie-Luise Zeisler*, Robert Koch Institute (RKI), Berlin, Germany
Johannes Lemcke*, Robert Koch Institute (RKI), Berlin, Germany
Leman Bilgic, Robert Koch Institute (RKI), Berlin, Germany
Thomas Lampert, Robert Koch Institute (RKI), Berlin Germany
Claudia Santos-Hövener, Robert Koch Institute (RKI), Berlin Germany
Patrick Schmich, Robert Koch Institute (RKI), Berlin, Germany

*shared first authorship


Background: Persons with migrant backgrounds (PMB) are considered ‘hard to reach’. To sustainably integrate migrant populations into health monitoring in Germany, the Robert Koch Institute (RKI) has launched the project Improving Health Monitoring in Migrant Populations (IMIRA), providing results on response rates, sample composition and the effectiveness of sequential interventions. Method: A multilingual feasibility study had been conducted in two German federal states, utilizing data from the residents’ registry. The target populations were persons with Turkish, Polish, Romanian, Syrian and Croatian citizenship living in Germany (n=9,068). Different modes of administration and interventions (study hotline, home visits) were used sequentially. To …


, , , ,

No Comments

Building a Sampling Frame for Migrant Populations via an Onomastic Approach – Lesson learned from the Austrian Immigrant Survey 2016 Special issue

Dimitri Prandner (dimitri.prandner@jku.at), Johannes Kepler University of Linz
Martin Weichbold (martin.weichbold@sbg.ac.at), Paris Lodron University of Salzburg



Immigrants are traditionally seen as hard to survey. Their number is often too small to be analysed via data gained in general population surveys, and registers to identify them are often missing or incomplete. Therefore, researchers are forced to use alternatives for sampling. In the case of the Austrian Immigrant Survey 2016, an onomastic (name-based) approach was used, establishing a sampling frame in a two-step procedure. This article describes the concept and the implementation of the sampling and evaluates the sample that could be realised.


, , , ,

No Comments

Sampling Refugees for an Educational Longitudinal Survey Special issue

Hans Walter Steinhauer, Leibniz Institute for Educational Trajectories, Germany
Sabine Zinn, Leibniz Institute for Educational Trajectories. Germany
Gisela Will, Leibniz Institute for Educational Trajectories, Germany


In the years 2015 and 2016, in Germany more than half a million refugees were granted asylum or they were accepted being eligible for subsidiary protection. Thus, they got a residence permit. About 29% of the accepted refugees were younger than 18 years. To study education related integration issues in this group, in 2016 the large-scale survey study “Refugees in the German Educational System (ReGES)” had been established. In this study, refugee children not yet in elementary school and students in lower secondary education are surveyed. This article gives some first insights from the field of the study, details its …


, ,

No Comments

How to Implement Respondent-Driven Sampling in Practice: Insights from Surveying 24-Hour Migrant Home Care Workers Special issue

Lena Hipp, Berlin Social Science Center (WZB) & University of Potsdam, Germany
Ulrich Kohler, University of Potsdam, Germany
Sandra Leumann, Berlin Social Science Center (WZB), Germany


This article draws on the experience from an ongoing research project employing respondent-driven sampling (RDS) to survey (illicit) 24-hour home care workers. We highlight issues around the preparatory work and the fielding of the survey to provide researchers with useful insights on how to implement RDS when surveying populations for which the method has not yet been used. We conclude the article with ethical considerations that occur when employing RDS.


, , ,

No Comments

Sampling in Times of High Immigration: The Survey Process of the IAB-BAMF-SOEP Survey of Refugees Special issue

Simon Kühne, Bielefeld University, Germany
Jannes Jacobsen, DIW Berlin, Germany
Martin Kroh, Bielefeld University & DIW Berlin, Germany


Over the course of 2013 to 2016, over one million asylum seekers arrived in Germany, around 890,000 of them in 2015 alone. The growing refugee population posed a major challenge for Germany’s policy makers, civic administrators, and society at large, in finding new approaches to registration procedures, housing, and social and economic integration. To design policies and programs that meet these needs, government administrators, politicians, and the public require robust analyses of the accompanying social and demographic changes based on timely, valid, and reliable empirical data. Yet despite the urgent need for quantitative data on this target group, survey organizations …


, , ,

No Comments

Including nursing home residents in a general population health survey in Germany Special issue

Beate Gaertner, Robert Koch Institute (RKI), Berlin, Germany
Carmen Koschollek, Robert Koch Institute (RKI), Berlin, Germany
Maike M. Grube, Robert Koch Institute (RKI), Berlin, Germany
Denise Lüdtke, Robert Koch Institute (RKI), Berlin, Germany
Judith Fuchs, Robert Koch Institute (RKI), Berlin, Germany
Christa Scheidt-Nave, Robert Koch Institute (RKI), Berlin, Germany
Antje Gößwald, Robert Koch Institute (RKI), Berlin, Germany
Matthias Wetzstein, Robert Koch Institute (RKI), Berlin, Germany


Nursing home residents (NHRs) are systematically excluded from the target populations of most population-based health surveys, which may result in biased prevalence estimates. Researchers who wish to include NHRs in surveys face several challenges including difficulty sampling and contacting NHRs and greater levels of functional impairments impeding participation. A population-wide, register-based, random sample of 8,000 older individuals (57.1% women, mean age=76.2 years) in six primary sampling units (PSUs) in Germany was used to analyse NHR coverage. The contact and response rates among NHRs were compared to those among persons living in private households in two PSUs (N=2,000) by applying an …


, , , ,

No Comments

Effects of a sequential mixed-mode design on participation, contact and sample composition – Results of the pilot study “IMOA – Improving Health Monitoring in Old Age” Special issue

Beate Gaertner*, Robert Koch Institute (RKI), Berlin, Germany
Denise Lüdtke*, Robert Koch Institute (RKI), Berlin, Germany
Carmen Koschollek, Robert Koch Institute (RKI), Berlin, Germany
Maike M Grube, Robert Koch Institute (RKI), Berlin, Germany
Jens Baumert, Robert Koch Institute (RKI), Berlin, Germany
Christa Scheidt-Nave, Robert Koch Institute (RKI), Berlin, Germany
Antje Gößwald, Robert Koch Institute (RKI), Berlin, Germany
Judith Fuchs, Robert Koch Institute (RKI), Berlin, Germany
Matthias Wetzstein, Robert Koch Institute (RKI), Berlin, Germany
*shared first authorship


Existing health survey data of individuals who are 65+ years of age is limited due to the exclusion of the oldest old and physically or cognitively impaired individuals. This study aimed to assess the effects of a sequential mixed-mode design on (1) contact and response rates, (2) sample composition and (3) non-response bias. A register-based random sample of 2,000 individuals 65+ years was initially contacted by mail to answer a health questionnaire. Random subgroups of initial non-responders were further contacted by telephone or home visits. Participation by interview or proxy was possible. After postal contact only, the initial contact and …


, , , , ,

No Comments

Interviewing elderly in nursing homes – Respondent and survey characteristics as predictors of item nonresponse Special issue

Patrick Kutschar, Institute of Nursing Science and Practice, Paracelsus Medical University, Salzburg, Austria (patrick.kutschar@pmu.ac.at)
Martin Weichbold, Department of Sociology, Paris Lodron University, Salzburg, Austria (martin.weichbold@sbg.ac.at)


Survey methodology is applied regularly in medical, nursing or social science studies examining elderly populations. Research in nursing home residents, where age-related or pathological declines in cognitive function are highly prevalent, faces several methodological challenges. The quality of survey data may be subject to population-specific measurement errors. In this article, data of two studies about pain in nursing homes are used to examine which respondent-, survey- and item characteristics predict item nonresponse. Chances for non-substantial answers are higher for older residents, for females and for those with more cognitive impairment. If residents are in pain, valid answers are more likely. …


, , , ,

No Comments

How to reach ‘hard-to-reach’ older people for research: The TIBaR model of recruitment Special issue

Kerstin Kammerer, Institute for Gerontological Research, Torstraße 178, D-10115 Berlin, Germany
Katrin Falk, Institute for Gerontological Research, Torstraße 178, D-10115 Berlin, Germany
Anna Herzog, Institute for Gerontological Research, Torstraße 178, D-10115 Berlin, Germany
Judith Fuchs, Robert Koch Institute, Department of Epidemiology and Health Monitoring, General-Pape-Str. 62-66, D-12101 Berlin, Germany


Recruiting older persons with diverse health statuses as participants in research projects is a challenge for health researchers, particularly because persons with health impairments and in socially disadvantaged living conditions are difficult to reach. This article presents a step model for gaining access to older people who are difficult to contact. The step model is based on the literature and a qualitative analysis of documentation, field notes and memos that stem from the recruitment processes of two studies from the German research consortium ‘Autonomy despite multimorbidity in old age’, both of which also included older persons who would qualify as …


, , ,

No Comments

Undercoverage of the elderly institutionalized population: The risk of biased estimates and the potentials of weighting Special issue

Jan-Lucas Schanze, GESIS - Leibniz Institute for the Social Sciences, Germany
Stefan Zins, GESIS - Leibniz Institute for the Social Sciences, Germany


In most social surveys, the elderly institutionalized population is not part of the target population because it is considered as hard-to-reach and hard-to-interview. The deliberate exclusion of institutionalized elderly from survey samples might cause bias, like previous studies investigating institutionalized elderly persons and their transition to institutions implied. We use a Monte Carlo simulation based on cross-national samples of the Survey of Health, Ageing and Retirement in Europe (SHARE) to test whether the noncoverage and undercoverage of the elderly institutionalized population lead to biased estimates. Moreover, we examined to what extent weights could be used to correct for the underrepresentation …


, , , , ,

No Comments

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 4.0 International License. Creative Commons License