Asking Volunteers: Pitfalls and Solutions for Surveying Voluntary Organisations in Germany

---

Abstract

Volunteering is seen as an effective way to address a wide range of issues that are present in modern societies, in particular in rural areas. In order to gain insight into the current state of volunteering, a survey was carried out with the aim of providing an inventory of the digitalisation of volunteering and addressing the influence of spatiality in this context. This article examines the traditional data collection methods of surveys that have been used in the past to collect information on volunteering and presents a different sampling procedure based on a random sample of municipalities, due to insufficient data available from official authorities. Finally, the article discusses the results of the survey and analyses how the unique sampling procedure differs from previous approaches. The comparison with a larger survey shows that the different sampling methods produce largely identical results, at least for the central variable of organisational area. As long as no structured data are available from the public authorities, there will always be a trade-off with regard to sampling strategies and the method of contact when doing research on volunteering organisations.

Keywords

Associations, Digitalisation, Register data, Rural areas, survey methods, Voluntary work

Acknowledgement

This work was funded by the German Federal Ministry of Food and Agriculture (grant number 2821FE007B) and carried out in collaboration with neuland21 e.V. Special thanks go to our partners Berit Barutzki, Hanna Kribbel, Silvia Hennig and Janine Hoelzmann. We would also like to thank Anna Magin and Mascha Goerlich for an initial literature search and overview of (inter)national debates. I would like to thank Ariane Sept and Ralph Richter in particular for all they have done throughout the project.

Introduction

Volunteering is considered an efficacious remedy for a broad range of issues that exist in modern societies, especially in rural areas. Volunteer work fulfils a crucial role in cultural life, disaster control, local sports clubs, social services, and mutual aid, and thus plays an essential role in the provision of public services, especially in rural areas (Schwarzenberg, Miggelbrink and Meyer, 2017; Braun and Hansen, 2004). However, voluntary work also encounters significant challenges in rural regions: In times of demographic change, the age structure of associations is changing. Many of them are struggling with recruitment problems. In the countryside, there are typically longer distances to cover (Gilroy et al., 2018). A factor that has a substantial influence on engagement is demographic change and public transport, which is often in a bad state. The bureaucratic requirements for associations, for example, because of data protection, are increasing. The trend among younger people towards short-term and project-related engagement is challenging for all voluntary work. Not least, the Covid-19 pandemic has severely restricted many voluntary activities and forced a digitalization push, which poses significant challenges for many organisations to implement in the long term (Dederichs, 2023; Lachance, 2021; Barutzki et al., 2023).

The crucial role of voluntary work for society and the problems this field faces raise the need for further attention. Both politics, with target-oriented funding schemes, and research, with profoundly recommended procedures, are under obligation here. It should be noted, however, that campaigning and funding practices for the voluntary sector in Germany currently lack a scientific inventory. One of many reasons is the lack of adequate data to explore this area. Studies examine volunteering from the perspectives of those who perform volunteer work as well as those of volunteer organisations, particularly associations or NGOs. The largest and best-known surveys in this context are those that operate at the individual level (For example general surveys like ESS, SOEP, UKHLS and Allbus with modules about volunteering or more specialised surveys such as the FWS, CEV and CLS). In contrast to the individual level, where personal motivations and challenges of volunteering can be identified, research on and within volunteering organisations can provide insights into the challenges and opportunities that organisations face in engaging volunteers, including issues related to training, retention and recognition. Similarly, understanding the dynamics of volunteer engagement in organisations can help in developing effective strategies to improve volunteering and its impact on society. In this way, critical social needs can be addressed while supporting individual volunteering. However, research data about voluntary organisations is scarce. Apart from smaller one-off studies such as Reporting Volunteer Labour, a Canadian study of non-profit organisations from 2007 (Mook, Handy and Quarter, 2007), there has only been the National Congregations Study (NCS) in the USA since 1998 (Chaves, 2022) and the ZiviZ survey of the Stifterverband in Germany since 2012 (Schubert, Tahmaz and Krimmer, 2023).

To gain insights into the current state of volunteering and related challenges, in-depth research is essential. For this purpose, the AppVeL-project was launched in 2021 with the aim of providing an inventory of digitalisation in voluntary work while addressing the influence of rurality in this context. As we are conducting our research against the backdrop of current funding and support schemes for rural associations, it was most promising to look at volunteering from an organisational perspective (Thewes, Sept and Richter, 2024). Since registered associations and religious organisations play a central role, especially in rural areas, the focus has been placed on these two forms of organisational structures for reasons of research economics. This approach allows us to gain a comprehensive understanding of the challenges and opportunities that rurality presents for volunteering within these specific contexts. By examining the impact of rurality on registered associations and religious organisations, we can identify the unique barriers they face in recruiting and retaining volunteers, as well as the strategies they employ to overcome them. Additionally, this research will contribute to a deeper understanding of how rural communities can harness the power of volunteering to address their specific needs and foster social cohesion.

Given the inadequacy of the existing data and the limited access to the register data, this article describes the sampling and recruitment strategy to address the above questions. The results of the survey are then presented and the differences with conventional survey methods are highlighted.

Sampling and recruitment strategy for associations and religious organisations

The main objective of the survey was to obtain an accurate picture of (rural) volunteering in the light of digitalisation. In most instances, a standard random sample would be the customary method for achieving this objective. However, drawing a simple random sample of associations and religious organisations was not expedient for a number of reasons: 1. Due to the proportionally high number of associations in metropolitan areas, a very large sample would have been necessary to obtain a sufficient number of associations from rural areas. 2. Unlike associations, there is no central register of religious organisations from which a sample could have been drawn. 3. Central indicators of rurality (Thünen classification (Küpper and Milbert, 2020), RegioStaR (Sigismund, 2018)) are available at the level of (associations of) municipalities. The spatial information required to link these data is often missing from the register of associations or is not available in a structured and automatically readable form. To address these issues, a different sampling procedure (single-stage cluster random sampling) was necessary.

As with any random sample, the accessibility of the population is crucial. Although available to the public, Germany’s central register of associations is insufficient for scientific use due to a lack of machine-readable data. There is no interface (API) to automatically retrieve the data, nor is there any way to download the data in any form. Only a small subset can be accessed via the web interface. Due to the existence of more than 600,000 registered associations in Germany, it is not possible to use this data for scientific purpose (see Rat für Sozial- und Wirtschaftsdaten, 2023). To draw attention to this deplorable situation and to provide access to this data, the Open Knowledge Foundation Deutschland e.V. (OKF) conducted web scraping in 2019 and created with data from OpenCorporates a replication of the data from the register of associations, available in a machine-readable form (Open Knowledge Foundation Deutschland e.V., 2019). This source was used for the subsequent sampling. Without the contribution of the OKF, this project would not have been possible.

Sampling strategy

The first step for a single-stage cluster random sample is drawing the clusters: Out of the 11,007 municipalities in Germany (as of 2019), 635 were randomly selected. The 16 metropolitan regions (Hamburg, Hanover, Bremen, Düsseldorf, Duisburg, Essen, Cologne, Dortmund, Frankfurt (am Main), Stuttgart, Mannheim, Munich, Nuremberg, Berlin, Dresden and Leipzig) were excluded from this stage due to the proportionally large number of associations in those areas. Missing or incorrect spatial data from the register of associations/OKF for the selected municipalities were adjusted manually. As a result, a complete survey of all the registered associations within the selected municipalities was obtained (N = 21,606). For a comparative analysis, a sample of 5 per cent of all the associations in the 16 metropolitan regions with N = 5,516 was taken (simple random sample). With a total population of 497,328 associations in non-metropolitan areas and 109,797 associations in metropolitan areas, this results in a selection probability of 4.3% for registered associations in non-metropolitan and 5.0% for those in metropolitan areas. Later analyses took into account these differences by using design-weights. In addition, for all selected municipalities, the religious organisations in each municipality were identified by means of internet research (N = 1,487). Unfortunately, a separate weighting for religious organisations could not be calculated due to a lack of population statistics. Therefore, the values of registered associations were used for the sake of simplicity.

Since the OKF data is a replication of the register of associations, the shortcomings of the official data are also present here: The district courts, of which there are over 600 in Germany, enter the data for each association. In the absence of national standards, this data varies both in terms of the amount of information and in terms of its quality. While there are some district courts that consistently list all postal addresses, there are others that document this only sporadically or not at all. A manual (non-exhaustive) investigation showed that this could especially in the context of spatial analysis result in a systematic bias (missing not at random).

The next step was therefore to manually research contact information for all 28,609 randomly selected registered associations and religious organisations. An overly resource-intensive task, but a necessity to ensure the highest possible quality of data for spatial analysis. Priority was given to obtaining an email address or a web contact (e.g. contact form). If this information was not available, an attempt was made to obtain a postal address. In 4,454 cases, no contact information could be found. Postal addresses were found in 6,416 cases and email addresses (or contact forms) in 17,739 cases.

 

Table 1: Sampling frame and response rates

Recruitment strategy

In order to ensure a sufficient number of cases, participants were invited to take part in the online survey either by email or, in cases where email was not available, by post. The fact that an online survey on digitalisation was conducted with a predominantly digital invitation is by no means unproblematic. As both spatial analysis and digitalisation played a key role in the research project, a judgement had to be made between a bias on the spatial level due to a systematic lack of postal addresses and a bias on the level of digitalisation due to the recruitment and survey mode. Given the limited budget, the choice between these two options was forced to be a simple one, and the resulting shortcomings of a predominantly online invitation and online survey had to be dealt with and taken into account in later analyses. The fact that there was also an invitation by post in addition to the invitation by e-mail helps to reduce this problem. This allowed us to reach out to those associations with a more pronounced digital presence as well as those that do not have a well-developed web presence. However, due to budget constraints, the number of letters that could be sent with invitations to participate in the online survey was limited. First, we had to reduce the number of letters to 4,070 out of the 6,416 addresses collected. Direct mailing of printed surveys, including a stamped return envelope, would have significantly reduced the number to 700 letters. Assuming a response rate of no more than 10 per cent, the number of cases would be far too small for detailed spatial analysis. Furthermore, the emphasis on rural areas resulted in the omission of any letters sent to metropolitan areas. Due to different sampling procedures and the exclusion of letters in metropolitan areas, the findings and conclusions drawn from the survey may not be as representative between metropolitan and non-metropolitan areas as originally intended.

Gross sample achieved

In November 2021, 17,739 associations and religious organisations were invited by email to participate in the online survey. 4,070 postal addresses could be used to reach organisations that could not be contacted digitally. The letters sent contained a QR code for direct participation in the survey, in addition to a simple URL with access token. A second email was sent to those online contacts who had not yet completed the survey at the beginning of January. The survey was closed in the middle of January 2022.

A total of 2,828 organisations took part in the survey. 89 per cent of these completed the entire questionnaire and 17.6 per cent of all organisations came from the 16 metropolitan areas. Only 98 (7 per cent) of the 1,392 religious organisations participated.

Survey-results

In Germany, the problem is not only that the register of associations lacks sufficient data quality and is not easy to access. There is also no official and reliable information on other characteristics of associations, such as the number of members or the organisational area. The only information available from the register is the number of registered associations per federal state. To assess the survey’s quality by comparing it with official data, the total number of registered associations per federal state provides the only viable option.

 

Figure 1 shows the proportion of associations by federal state based on official register data provided by OKF (N = 607,125), the gross sample (N = 28,609), and the observations actually reached in the net sample (N = 2,828). Notable here are Baden-Württemberg and Mecklenburg-Western Pomerania. The survey figures largely coincide with the official register data in the other federal states. While the differences between the gross sample and the net sample remain within limits everywhere, it is the gross sample that deviates from the register data in the two cases mentioned above. The way in which the sample is drawn, namely through the municipalities, can help explain some of the discrepancies. In Baden-Württemberg, for example, the sample was made up of municipalities with a slightly higher number of inhabitants and therefore a larger number of associations. The same is true for Mecklenburg-Western Pomerania, but in the opposite direction. While the largest differences are seen between the register data and the gross sample, the small differences between the gross sample and the actual associations reached (from 0.09 to 1.00 per cent) suggest a positive indication of low unit non-response, at least with respect to regional differences at the state level. The information from the register data is used in the further analyses to compensate for the differences described here between the register data and the sample obtained by applying population weighting.

With the exception of the number of associations per state, there is no official data available to further compare the results of the survey. An additional point of reference is provided in the preliminary findings of the current ZiviZ survey, to be published in full at the end of 2023. The method of data collection via an online questionnaire was identical in both surveys. It should be noted, however, that the ZiviZ survey is based on address data from the Register of Associations, so the problems mentioned in the previous section may have introduced a certain degree of selection-bias in this survey.

 

The difference is that the number of cases is significantly higher in ZiviZ and not only associations, but also non-profit companies, non-profit cooperatives and civil law foundations were asked to complete the online questionnaire. However, this was done exclusively by post and not by a combination of e-mail and post as it was the case in the AppVeL-project. Figure 2 shows the results for each organisational area in both surveys. Notable differences can be seen, in particular, in the field of education and among religious groups. The latter may be explained by the fact that this group was examined and selected separately for the AppVeL-project. In this way, many religious groups were included in the sample that could not be found in the register of associations (or in any of the other ZiviZ sources). Whether the high number of organisations in the education sector comes from non-profit companies, non-profit cooperatives or civil law foundations cannot be said without the data, but it is an obvious explanation.

It is surprising that, with the exception of the education sector, the results are quite identical in most organisational areas. Despite the differences in the number of cases, the way in which respondents were contacted and the population covered, and despite the different problems of case selection faced by both surveys, the initial results of the two surveys seem to endorse each other. Once both datasets are publicly available, this first tentative assumption needs to be further verified.

A last aspect is the question of bias due to the predominantly digital invitation, which provides a general information on the quality of the data and is an important background for further analysis. In addition to the 17,739 emails, 4,070 letters were sent. This enables a differentiation between the groups that can be reached online and those for whom it is not possible to find any digital contact information.

 

Table 2: Comparison of different characteristics with regard to contact mode

 

The characteristics presented in Table 2 are elementary for addressing the key research questions. Both the organisational structure (age and size) and the degree of digitalisation are essential for an inventory of the German volunteering sector as well as for recommendations for a targeted funding policy. Comparing those characteristics of the survey between the two groups, the differences are quite clear.

The level of digitalisation and digital skills are considerably lower among those without digital contact (For information on how the level of digitalisation was measured see Thewes, Sept and Richter, 2024). The use of e-mail as a part of internal digitalisation further emphasises this. Structural differences, such as the size of the association and the age of the staff, are not very apparent. Response rates, on the other hand, vary considerably, with a difference of 14.7 to 4 per cent. For associations without an online presence, both the subject matter and the invitation to participate in an online survey appear to be important exclusion criteria. This is exacerbated by the fact that in the case of associations where only a postal address could be found, this information often came from automated branch platforms. In the course of the research, this source of information was repeatedly found to be of poor quality.

Conclusion

This article describes an alternative sampling method for analysing voluntary organisations in Germany. While there is a large amount of survey data with long time series from an individual perspective, the situation is different when research focuses mainly on the meso level of voluntary organisations. As the lack of data made it necessary to conduct a new survey, the problems of sampling at the organisational level had to be addressed. In Germany, this poses a number of problems: access to the most important database on formal volunteering, the register of associations, is hardly possible for sampling purposes. On the other hand, the quality of the data is problematic, making structured sampling difficult. Therefore, an alternative sampling method based on a sample of municipalities had to be used. This required manual address research, which was time-consuming but unavoidable due to the lack of contact information and budgetary constraints.

The most notable shortcomings of the survey are the way in which the organisations were contacted, mostly by email, and the way in which the survey was conducted using an online questionnaire. The first problem was at least addressed to the extent that some of the organisations were also invited by post. A comparison of the two groups revealed significant differences with regard to key aspects of the research project’s questions. This is a fact that has been given too little attention in the analyses carried out so far by the research project. Unfortunately, the low response rate from the postal group limits the scope for detailed analysis regarding the form of contact.

The online format of the survey, as opposed to a paper-based questionnaire, is an important issue in survey research. As the project deals with digitalisation, the influence of the survey instrument is even more important. This problem was known from the beginning, but an (additional) survey with a paper questionnaire was not possible at any time for budgetary reasons. The limited validity of the results had to be accepted and taken into account when evaluating the results. The comparison with the results of the much larger ZiviZ survey shows that the two sampling methods produce largely identical results for the central variable of organisational area. However, as soon as the complete data are published, a detailed comparison with other parameters will be essential. The differences at the level of the federal states, which can be seen in the comparison with the register data, can be explained by the sampling at the level of the municipalities and are mostly marginal. Furthermore, these effects can be accounted for by using the size of the municipalities.

In conclusion, despite the difficult data situation regarding formalised volunteering in Germany, a valuable source of data for research on the German voluntary sector has been created. The results of the AppVeL-project and of ZiviZ show that, as long as no structured data are available from the authorities, there will always be a trade-off between time and financial commitment with regard to sampling and the form of contact. At the same time, however, research funding must also provide the resources to avoid known and avoidable sources of bias, such as the mode of survey. Not only when it comes to digitalisation should an analogue form of contacting and interviewing be carried out when surveying volunteers. Future research and survey projects must therefore have the necessary budget, but it’s also essential that access to the population, via the register of associations, is not only automated or machine-readable but that the quality of the data meets modern administrative standards.

 

References

1. Barutzki, B., H. Kribbel, A. Magin, S. Hennig and C. Thewes (2023). Zwischen Appstore und Vereinsregister. Ländliches Ehrenamt auf dem Weg ins digitale Zeitalter. https://neuland21.de/wp-content/uploads/2023/02/20230203-laendliches-ehrenamt-layout-interaktiv.pdf.
2. Braun, S. and Hansen, S. (2004). Soziale und politische Integration durch Vereine? Forschungsjournal Soziale Bewegungen, 17, 62–69.
3. Chaves, M. (2022). National Congregations Study, Cumulative Dataset (1998, 2006-2007, 2012, and 2018-2019): Open Science Framework.
4. Dederichs, K. (2023). Volunteering in the United Kingdom During the COVID-19 Pandemic: Who Started and Who Quit? Nonprofit and Voluntary Sector Quarterly, 52, 1458–1474.
5. Gilroy, P., Krimmer, H., Priemer, J., Kononykhina, O., Pereira Robledo, M. and Stratenwerth-Neunzig, F. (2018). Vereinssterben in ländlichen Regionen – Digitalisierung als Chance: ZiviZ im Stifterverband.
6. Küpper, P. and Milbert, A. (2020). Typen ländlicher Räume in Deutschland. In Krajewski, C. and Wiegandt, C.-C. (Eds.). Land in Sicht. Ländliche Räume in Deutschland zwischen Prosperität und Peripherisierung. Bonn: Bundeszentrale für Politische Bildung, pp. 82–97.
7. Lachance, E. L. (2021). COVID-19 and its Impact on Volunteering: Moving Towards Virtual Volunteering. Leisure Sciences, 43, 104–110.
8. Mook, L., Handy, F. and Quarter, J. (2007). Reporting Volunteer Labour at the Organizational Level: A Study of Canadian Nonprofits. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 18.
9. Open Knowledge Foundation Deutschland e.V. (2019). https://offeneregister.de.
10. Rat für Sozial- und Wirtschaftsdaten (2023). Positionspapier des RatSWD: Nutzung von Registerdaten für Zwecke der Forschung sichern. https://www.konsortswd.de/wp-content/uploads/RatSWD-Positionspapier-Nutzung-von-Registerdaten.pdf.
11. Schubert, P., Tahmaz, B. and Krimmer, H. (2023). Erste Befunde des ZiviZ-Survey 2023. Zivilgesellschaft in Krisenzeiten: Politisch aktiv mit geschwächten Fundamenten. Berlin: ZiviZ im Stifterverband.
12. Schwarzenberg, T., Miggelbrink, J. and Meyer, F. (2017). „Nicht für Erich Honecker früher oder heute für Angela Merkel, sondern für sich selber“ – Eine Fallstudie zu ehrenamtlichen Engagementformen im ländlichen Raum zwischen gesellschaftspolitischen Ansprüchen und individuellen Wahrnehmungen. Raumforschung und Raumordnung | Spatial Research and Planning, 75, 563–576.
13. Sigismund, M. (2018). Regionalstatistische Raumtypologie (RegioStaR) des BMVI für die Mobilitäts- und Verkehrsforschung. Arbeitspapier.
14. Thewes, C. (2023). Zwischen Appstore und Vereinsregister – Ländliches Ehrenamt auf dem Weg ins digitale Zeitalter (AppVeL): GESIS Data Archive. DOI: 10.7802/2577.
15. Thewes, C., Sept, A. and Richter, R. (2024). A Voluntary Divide? Exploring the Role of Digitalisation in German Rural Volunteering. European Countryside. 1/2024.