E-Appointments in Empirical Field Research

---

Abstract

A time-consuming part of the organization of empirical field research is the arrangement of appointments with individual participants. Often times part of the sample is hard to reach via phone. An online appointment system to facilitate the coordination of the annual family visits was tested within the scope of a German educational panel study in 2012. The participants had the opportunity to arrange their appointment online in accordance with their own scheduling needs. Roughly one quarter (n = 63) of our sample of 242 families, who had thus far been used to arranging their appointments by phone, chose the online appointment system instead. To evaluate its usability for field research and the strength of possible sample bias, the participants were asked to complete a short questionnaire as well. We can show that personal attitudes are more significant in explaining a choice in favor of the e-appointment system than socio-demographic characteristics. Overall, the e-appointment system does not increase sample selectivity bias in our survey, so that particularly for bigger samples, it is as capable as phone based arrangement systems in getting a hold of people who are hard to reach and retaining the same high process quality, while reducing the researcher’s expenditure of time and money.

Keywords

E-Appointment, field research, internet, sample selectivity, survey organization

Acknowledgement

Funding The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project described was supported by grants from the German Research Foundation (FOR 543, RO 820/12-1, RO 820/12-2, and RO 820/12-3).

1          Introduction

In empirical field research it is often necessary to arrange appointments with survey participants to perform an interview or a specific test. To this end the telephone is often the preferred means of choice. This way appointments can be arranged by allowing participants to look for free time slots in their schedule directly. Unfortunately, the participating individuals are often only reachable at times that are incompatible with the working hours of the staff arranging the appointment, especially in the evenings. The written alternative by mail, e-mail or fax requires a voluntary response of the participant, is more complicated, associated with additional costs and often times entails a high level of non-response (Diekmann, 2008).

Therefore, the ideal option for arranging appointments should not rely on the necessity of direct contact with the subjects, but allow complete flexibility and take their preferences into account, while being binding at the same time. Appropriate internet based instruments, (capable of tackling a task like this) have been on the market for quite some time (e.g. supersaas.com). However, to date the use of this option is marginal in the scientific field, while it is already practically the standard in the service sector (Döring & Schick, 2002; Schulz, 2008) and in the service industry (Creemers & Lambrecht, 2010; Li et al., 2006). If applied at all in the scientific field, such instruments have mainly been used in the field of medicine (Bend, 2004; Klischewski, 2003; Parmar, Large, Madden, & Das, 2009). In our study, the described problem of low availability of participants motivated us to use an internet-based procedure in the context of a German educational longitudinal study. In this paper we are going to report our experiences with the e-appointment method starting with our requirements for this approach in the following section.

2          A Three-Criteria-Model in E-Appointments

More than a decade ago, Döring and Schick (2002) recognized that the demand for e-appointments was increasing steadily. However, in scientific research this topic is still not very popular and seldom found in the literature. Thus, our model for an ideal e-appointment tool is based on practical experience and theoretical considerations rather than on empirical evidence. For any appointment system to be useful in survey application, we assume that three criteria have to be met, namely a minimum impact on selectivity bias, high process quality and economic efficiency (image 1). Each of these criteria will be explained briefly in the following.

 

Image 1: 3-Criteria-Model for the E-Appointment System

2.1       Selectivity

The chosen appointment method has to be appropriately designed to reach not only as many participants as possible, but every potential participant with an equal likelihood as well. If there is a systematic exclusion of a certain portion of the sample (e. g., households without a telephone or internet connection, those with a migrant background, etc.), the method would be deemed inappropriate.

The plausibility of using online communication depends on the target group’s affinity towards the medium. According to the German Federal Bureau of Statistics, 83 percent of the German population had internet access at home in 2011. Germany was therefore slightly ahead of the EU-27 average (73.2 percent) and the OECD average (74.9 percent) according to OCED statistics. Within the group most relevant to our survey, that is households with two adults and at least one child under the age of 16 years, 98% had internet access (Czajka & Jechová, 2012). Thus, the danger of a perceived discrimination against individuals who do not have internet access and therefore are excluded from this new option is negligible. In the following we will look at a few factors which might account for variance in an individual’s internet usage briefly.

Migrant background: Differences remain in internet use. According to the results of the German ARD/ZDF-survey “migrants and media 2011”, internet use by migrants is still below the national average. In the age groups relevant to our study – 30 to 39 (and 40 to 49) year old migrants – 40 (37) percent use the internet daily, whereas in the total population 58 (47) percent do this (Müller & Beisch, 2011). Differences between various ethnicities play a minor role in this situation.

According to further research (Initiative D21 & TNS Infratest GmbH, 2011) there are also education, age, gender and regional differences in general internet usage (measured over a 12-month period).

Education: About 91 percent of the population with higher education entrance qualification (Abitur) used the internet, whereas this was only true for about 51 percent of those with a lower secondary degree (Hauptschulabschluss). However, people with lower educational achievements displayed the highest increase in internet usage and are expected to catch up with the educational high achievers soon.

Age: Older people use the internet less often. Altough roughly 90 percent in the age groups relevant for our study, 30 to 39 and 40 to 49 years, use the internet, there are still differences between these two groups: In the younger group 94 percent use the internet, whereas in the older group 86 percent do so.

Gender: Furthermore there is a strong gender difference in internet usage: 80 percent of males are using the internet, but only 68 percent of females. These gender differences are found across all levels of educational achievement, but are smaller within the group of highly educated people than in the group with lower educational achievements (see also van Eimeren & Frees, 2010).

Region: Although internet access is widespread and its infrastructure expands rapidly, residence still matters when it comes to having a usable internet access point. The number of broadband internet connections increases with the number of citizens in a location, and people without an internet connection are more common in places with less than 20,000 citizens. About 30 percent of the citizens in these rural areas do not have high speed internet access, whereas in large cities with a population over 500,000 citizens, this value is 22.5 percent.

2.2       Process Quality

The consideration of possible structural selection effects is important, but at the same time the appointment method should also ensure a high process quality as this affects project costs and possibly sample selectivity as well. Dillman’s Tailored Design Method stresses the importance of cover letters and reminders during the field phase and a thank-you card afterwards for mail or telephone contact with the sample (Dillman et al., 2009). For an online approach additional criteria have to be considered. For example, the time span between contacting and actual date should not be too long, because the likelihood of postponement or cancellation of the appointment increases proportionally with its length. Furthermore, there have to be sufficient options for the users to choose from. Without this, an online appointment method would not be successful. The so-called “Choose and Book System” (http://www.chooseandbook.nhs.uk) of the British National Health Service (NHS) aimed at making it easier to choose a practitioner, serves as an example for this argument (Green, McDowall, & Potts, 2008). Another important point is the user experience in form of ease of use and flexibility. If the user experience (or other benefits) is not at least as good as with a telephone call, acceptance and thus study participation will be severely affected. From the operator’s perspective, the benefits of a phone call are to speak to the participant directly and the social expectation to subsequently make the appointment. From the participant’s perspective, no further actions are needed, they do not have to become active by themselves. There are, however, also disadvantages of telephone appointments like inconvenient timing or situational stress that could be projected to the study and might reduce the motivation to participate. This could be avoided by an online-based appointment system.

2.3       Economics

Last but not least, economic efficiency is important for any study. In most cases it is necessary to conduct a survey within a limited time frame and with a limited amount of funding. Experiences show that making appointments with study participants takes a lot of time due to their often poor availability. It does not matter if this job is done by a central study coordination or by the interviewers themselves, because more time always means higher costs. Compared to the individual and personal approach, an online booking option covers all study participants simultaneously and leaves it to them when they want to respond. Thus, incalculable multiple contacting – and additional costs – can be avoided.

Another positive aspect is an automated reminder system which can be provided by an online appointment tool. Instead of manually keeping track of participants who do not already have a valid appointment at any given deadline and sending out reminder letters with the risk of missing participants, a suitable online tool can send reminders via SMS or e-mail automatically. This saves costs and leaves less room for human error.

3          E-Appointment process in the Research Group BiKS

BiKS is a study of a German interdisciplinary research group that followed more than 4,000 Bavarian and Hessian children in two cohorts over a period of more than 7 years. The German acronym BiKS stands for “Educational Processes, Competence Development, and Selection Decisions in Preschool and School-Age Children” („Bildungsprozesse, Kompetenzentwicklung und Selektionsentscheidungen im Vorschul- und Schulalter“). Within this study, annual visits of initially 550 families from the younger cohort comprising children ages 3 through 8 were performed. These visits took place in kindergartens at first, but when the children moved to school, some of them could not be followed in schools and had to be visited at home for individual testing. Amongst others, the individual competency measurement of children and process observations took place during these visits (Lorenz, Schmitt, Lehrl, Mudiappa, & Roßbach, 2013). The expense of arranging appointments via phone and coordinating these with the interviewers was enormous, and it often took a period of more than six months to complete the field work. Therefore, in 2012, for the first time an alternative method was developed: Families were now able to book their appointment for the visit online autonomously. The whole process included the following steps:

3.1       Cover Letter to the Parents

We informed the parents about the next upcoming family visit by mail. The letter pointed out the option of booking an appointment online in particular. For that purpose every family received both the necessary log-in data including an individual code, which had to be entered when booking the appointment. Since the families could be identified by code, there was no need to enter any personal information online. This way it was easy to comply with the strict rules for protection of personal data in Germany. It was also emphasized that the usage of the online booking option was completely voluntary and if it was not used within a limited timeframe of 2.5 weeks the familiar phone appointment booking would be used instead.

3.2       Activation of the E-Appointment System and Procedure

The website for the e-appointment went live the same time the cover letters were being sent out. This required that sufficient time slots for our interviewers had already been registered by this point in time, so that parents were aware of all available interview dates. Double bookings were prohibited by the system. As soon as the appointment was booked, the system automatically sent an email to the project office, including the families’ code and the interview date, which provided all of the information necessary for the coordination of the interviewers.

3.3       Start of the Additional Telephone Appointments

After a period of 2.5 weeks with e-appointments as the only option offered, it was assumed that all families who wanted to use the online appointment would have done so within that given time frame. As our primary focus was to avoid losing any families in our longitudinal study, we started calling the remainder of the participants immediately after the period of 2.5 weeks. Parents, who had booked their appointment online, still were able to access the website and view their booking details. However, changing the booked appointment online was not possible but had to be arranged on the phone.

4          Research Questions

We specified the following questions to test the appropriateness of e-appointments for empirical field research and to allow statements regarding its effects on the survey process:

1. Selectivity: What percentage of the sample used the e-appointment service option? Are there systematic differences between these parents and those who made their appointment on the phone?

2. Process quality: Which factors influence parental appointment postponement, and in comparison, are appointments booked online postponed less often than those booked by phone?

3. Economics: What survey sample size is required to make e-appointments profitable?

5          Data

The current study is based on data collected during the German longitudinal survey BiKS-3-10 (Lorenz et al., 2013). The sample eligible for family visits included N = 242 families who were invited to arrange their appointment online. Two-fifths of the participating families had a migrant background. The important socio-demographic information was gained from parents’ telephone interviews (CATI) in the same survey wave and in former measurement points of the BiKS-3-10 panel. 45 percent of the families had at least one parent with the highest school leaving certificate (“Abitur”), whereas about 17 percent of families were only in possession of the lowest school leaving certificate (“Hauptschulabschluss”) or of no certificate at all. The age pattern of the participants was relatively homogenous, since all parents’ children attended fourth grade in school. The average age of the mothers was 42.5 years with a standard deviation of 4.5 years. Three-fifths of the sample were from rural areas and two-fifths from urban regions. This information will serve as independent variables in the analysis of selectivity and the process relevance of the e-appointment booking system (s. chapter 2.1). Although the sample size is small, apparently all attributes show a sufficient number of cases. Nearly all of our participants were mothers so that possible gender effects could not be analyzed.

To learn more about the reasons why parents decided to arrange an appointment via the internet or via phone, we asked the parents to answer an additional one-page questionnaire during the family visit. They were prompted to evaluate different aspects such as reachability, their ability to use the internet or their attitude towards appointment making. Additionally, data on available internet enabled electronic devices of every household was collected. The questionnaire was completed by 77 percent of all participants (n = 186).

In order to cope with missing data due to non-response, we applied the multiple-imputations-by-chained-equations-technique as implemented in the –ice– Software Package in Stata (Royston & White, 2011). We generated 100 imputed data sets and analyzed them according to Rubin’s rules (Rubin, 1987). The imputations were conducted using predictive mean matching in order to receive plausible values for the covariates (Schenker & Taylor, 1996).

It is important to know that in spite of the new appointment method, the level of participation in the family visits was comparable to previous measurement points. Furthermore, it has to be mentioned that in most cases participating families had been part of the panel for about six years, and had since been surveyed on an annual basis or even more frequently. Thus, it may be assumed that their motivation level is above average. Since our panel study ran without additional sample refreshments this also meant that we could not afford to lose participants due to the new (possibly worse performing) appointment method. This is why we could not run a random treatment experiment but let the participants themselves decide first whether or not to choose the online option and afterwards called all those parents who had not responded until then. Thus, our study is not adequately suited to answer the question if such an e-appointment system impacts overall study participation.

6          Outcomes

6.1       Selectivity

To use the e-appointment option at all, it is necessary to have a web-enabled device available in the household. To probe for structural exclusion factors, questions related to existing internet-enabled devices and access paths were asked in the additional questionnaire. In our sample only 2 individuals out of 186 (1.1%) stated that they were in possession of none of the internet-enabled devices listed (about 1.9% (s.e. = .01) in the imputed data), so that almost every family had the opportunity to use the online appointment method. As shown in Table 1, there are barely appreciable differences between the groups with reference to possession of internet-enabled devices.

 

Table 1: The equipment of every household with internet-enabled devices in relation to the agreed method of appointment

Device available in the household	by phone (s.e.)	online (s.e.)	p
Desktop-PC	73.7% (.04)	88.9% (.04)	.026
Laptop/Netbook	73.6% (.04)	76.4% (.06)	.686
Tablet-PC (e.g. iPad)	15.2% (.03)	19.3% (.05)	.497
Smartphone	32.9% (0.4)	43.1% (0.7)	.191

N = 242; based on m = 100 imputed datasets.

 

Next, we focused on differences between the parents who arranged their appointment by phone (n = 179) and those who did so online (n = 63) by applying logistic regression. As shown in table 2, differences in the evaluated characteristics between these two groups partly exist. Firstly, we included basic demographic characteristics (see chapter 2.1) as predictors for using e-appointment in the model (cf. table 2, model 1). Neither the residential area (urban vs. rural) nor the age of the respondents have a significant impact in this model. In contrast, we find a small but significant effect if the highest educational achievement is the “Realschulabschluss” (General Certificate of Secondary Education) compared to families with the lowest educational degree (Certificate of Secondary Education / “Hauptschulabschluss”); parents with higher education are more likely to use the e-appointment option. Beyond that, having a migrant background also diminishes the probability of choosing the e-appointment offer.

 

Table 2: Parents’ characteristics that influence the possibility of using the e-appointment system (average marginal effects of logit models with outcome: used e-appointment)

Model	(1)	(2)	(3)	(4)	(5)
	AME (S.E.)	AME (S.E.)	AME (S.E.)	AME (S.E.)	AME (S.E.)
Family is living in a city	0.061 (0.066)				0.087 (0.062)
The highest school education of the family is the … (Ref. “Hauptschulabschluss”)    … “Abitur”	0.135 (0.121)				0.140 (0.099)
… “Realschulabschluss”	0.170+ (0.098)				0.099 (0.089)
Age of the participant	-0.011 (0.007)				-0.005 (0.007)
Migrant background (yes/no)	-0.197* (0.095)				-0.160+ (0.083)
Attitudes: It is important for me …    … to know the date of the appointment as soon as possible.		-0.011 (0.074)		-0.033 (0.074)	-0.041 (0.072)
… to make an appointment without any time pressure.		0.017 (0.109)		0.051 (0.094)	0.056 (0.088)
… to choose the time of the appointment as freely as possible.		0.238 (0.178)		0.208 (0.163)	0.219 (0.152)
I am often not reachable or hard to reach via phone.		0.112+ (0.058)		0.051 (0.057)	0.059 (0.055)
I think that the e-appointment system is easier for everybody involved.		0.299** (0.086)		0.124 (0.092)	0.119 (0.087)
I prefer personal contact when I arrange an appointment.			-0.412** (0.070)	0.334** (0.077)	0.332** (0.073)
I am very worried about the security of my personal data on the internet			-0.019 (0.058)	-0.02 (0.058)	-0.003 (0.059)
Average Pseudo-R2	0.035	0.115	0.178	0.216	0.245

N = 242, multiple imputated data (m = 100); ⁺ p<0.10, * p<0.05, ** p<0.01, robust standard error in parentheses.

 

In models 2 and 3 we added all attitude variables which had been collected through the additional questionnaire to gain further insight into the reasons for the decision in favor of or against the use of the e-appointment system. In a first block (model 2) variables indicating the usage of the online booking system were integrated: (1) “It is important for me to know the date of the appointment as soon as possible”, (2) “It is important for me to make an appointment without any time pressure”, (3) “It is important for me to choose the time of the appointment as freely as possible”, (4) “I am often not reachable or hard to reach via phone” and (5) “I think that the e-appointment system is easier for everybody involved”. The last two items show a significant influence on the chance of using the online system. People who think that they were hard to reach are more likely to book online. Going into detail, 59 percent of the “onliners” agree with the statement that they were hard to reach, whereas only 36 percent of the phone preferring parents do. Interestingly, this wide difference between the groups does not match with the data we had previously collected from the telephone interviews with these parents. We had data on how often every family had to be called for the annual parental phone interview in this study until someone answered the phone. Parents who think they are hard to reach have an average call rate of 7.1, and parents who think they are easy to reach had to be called 7.4 times. Hence only the subjective feeling of being hard to reach led to a higher willingness to use our online booking system. The second significant argument for the usage of the e-appointment system is the parents’ opinion that the system is a relief for everybody involved. This includes the possibility that participants not only see an advantage for themselves, but also for the survey management team.

In a second block (model 3), variables indicating the preference for arranging an appointment by phone were integrated: (6) “I prefer personal contact when I arrange an appointment”, (7) “I am very worried about the security of my personal data in the internet”. Unsurprisingly, parents that preferred personal contact were more likely to choose the phone method.

When including all attitudes simultaneously in the models, only the preference for personal contact remains significant (model 4). This means that the wish for being contacted personally is ultimately the most important factor, even if the participant believes herself to be hard to reach or that the online booking method is a relief.

Adding – in a final step – the variables of the questionnaire in the regression analysis to the socio-demographic characteristics (model 5) covers the effect of education background, but not completely the effect of the migration background, which remains significant on a 10-percent-level. Apparently always having the same contact person is not as important for parents – in the present long-term study –, as speaking with a real person, since in most cases the contact person changed every year. Beyond this, migrants seem to be a part of the sample that is not amenable to our e-appointment method in the same way non-migrants are.

6.2       Process Quality

Beyond selectivity concerns, relevant process quality indicators are also of interest. This includes the time period between the booking, respectively the agreement on a date and time slot, and the (first) arranged appointment. In the group of “phone arrangements” this time frame had an average of almost 24 days, four days shorter than the time frame for the “online appointments” group. The maximum time frame for families who had booked online was 111 days. For parents who had arranged their appointment by phone, the maximal time difference was 96 days. These differences are not significant. However, one has to bear in mind that arrangement by phone started 2.5 weeks later by design.

As a second process outcome, we examined the number of postponements requested by the parents. Approximately one-fifth of the sample rescheduled their arranged appointment at least once. Though we found no significant (bivariate) mean difference between the two groups, cases in which the appointment was rescheduled two or three times only occurred in the group of parents who used the phone method.

Table 3 shows the results of the multivariate analysis. We estimated the probabilities of the first arranged appointment being kept using binary logistic regression analysis with booking method and basic demographics as predictors. Model 1 shows that the probability of keeping the first appointment increases with the age of the participant and with a shorter timespan between booking and appointment. Concerning the impact of the booking method, we do not find a significant difference between phoners and onliners.

In model 2 an interaction term of “booking method” and “time span between booking and interview date” was added to examine possible group differences. The statistically insignificant effect near zero also supports the assumption that there is no difference between phoners and onliners regarding the parents’ postponement of the appointments.

 

Table 3: Characteristics that influence the parents’ postponement of the appointments (Average marginal effects of logit models with outcome: first agreed appointment kept)

Model	(1)	(2)
	AME (S.E.)	AME (S.E.)
Family is living in a city	-0.077 (0.054)	-0.076 (0.055)
The highest school education of the family is the … (Ref. “Hauptschulabschluss”) … “Abitur”	-0.042 (0.098)	-0.042 (0.098)
… “Realschulabschluss”	-0.091 (0.078)	-0.092 (0.078)
Age of the participant	0.015** (0.006)	0.015** (0.006)
Migrant background (yes/no)	-0.086 (0.066)	-0.087 (0.067)
Appointment was arranged online	0.042 (0.057)	0.039 (0.059)
Time (days) until appointment (centered)	-0.003* (0.001)	-0.003* (0.002)
Interaction: online x time		0.001 (0.003)
Average Pseudo-R2	.0787	.0791

N = 242, multiple imputated data (m = 100); * p<0.05, ** p<0.01, robust standard errors in parentheses.

 

6.3       Economics

To answer the last question concerning the necessary sample size for making the e-appointment system profitable we assume the following conditions:

• In this example the usage of our used e-appointment system – for up to 1,500 bookings – costs 35.70 euros per month.
• We estimate 40 work hours for the one-time system set-up as an upper bound. Most operators certainly need less time. Re-using the system in following studies/panel waves reduces future costs.
• It takes about seven attempts on average to reach a family via telephone and every call takes up to one minute. These figures are derived from the experience of phone interviews with the same sample conducted by the BiKS team over several years.
• If a call is successful, an average time span of three minutes is needed to make an appointment with the family (average from former years).
• A (student) research assistant is paid about ten Euros per hour.

The distribution of costs shown in image 2 is calculated from this data. For the e-appointment system time and financial resources are mainly spent on the one-time software set-up, and in addition to that, prices are scaled corresponding with the number of bookings. However, these costs are relatively low. For the most part, there are no additional costs which would not also occur in arranging appointments by telephone (e. g. coordination of the appointments with the interviewers). Arranging appointments by phone does not produce initial costs, but every single call takes up time and money. Including the unsuccessful call attempts, it takes 10 minutes to schedule an appointment by phone. In a scenario with a maximum of 1,500 bookings in one month, the break-even point is reached at a sample size of about 240 subjects. While the expenditure of time and costs is rises in a linear manner with the number of parents called, for e-appointment systems it remains at virtually the same level (in our case 40 hours and below 500 Euros for online appointments and 250 hours and accordingly 2,500 Euros for telephone appointments).

 

Image 2: Comparison of costs between arranging an appointment via phone, via the internet or in a one-, two- or three-quarter-ratio (example for the duration of max. one month with max. 1,500 appointments to arrange).

 

Assuming a mix of web and phone based arrangements, costs rise with an increasing portion of phone usage. In our study, a ratio of 25 percent online and 75 percent telephone method would amortize from N = 320 onward, a 50:50 distribution between the two methods would be worthwhile with more than 480 participants, and if three quarters of all appointments were scheduled by phone, the additional use of the online tool would be economically appropriate from a sample size of 960 participants or larger. The general formula to calculate the required minimal sample size for the (additional) use of an e-appointment tool is as following:

 

 

7          Discussion

The present paper aimed to study the effects of an e-appointment system on sample selectivity, process quality, and survey economics. We tried to make a case for the practicability of the system despite of the relatively small sample.

The fact that only every fourth family (26 percent) used the e-appointment system is disillusioning at first glance, but certainly this is due to the relatively short employment time of 2.5 weeks and the fact that the parents had been used to phone based appointments for the previous seven years of the study.

Nevertheless, the e-appointment system we used provided us with important information. Taking into account the effort necessary for calling each participating family, there is already a real payoff if one quarter of a considerably sizeable sample (N » 1,000) becomes pro-active in making their appointment. The analysis has shown that there are only slight differences between the online and the phone preferring parents. The main differentiators were a migrant background and the parents’ wish to speak to a real person, which led participants to refuse the online appointment method. Finally, we were able to show that the parents’ probability of realizing the first booked appointment increases with the mother’s age and with less time having passed between booking and appointment, but does not differ between the phone-based and the online booking method. Of course, one has to be aware of the small sample size so that our findings should next be verified with larger samples.

Certainly, there is a significant difference in the use of a particular booking system, regarding the time expended and the financial resources needed. Compared to a phone based system the financial break-even point is reached at relatively small sample sizes if online booking is used exclusively. With a sample size greater than 200 participants, the e-appointment system could be cheaper and much more efficient than other methods of making appointments. Taking into account that in our case migrants tended to be not amenable to e-appointment, one possibility was to consider them separately and provide telephone calls for them while choosing the e-appointment method for all the other participants. With a sufficient sample size, the saving would be worthwhile.

To sum up, we conclude that although online appointments are selective in some ways, we can counter this selectivity and save costs (if the sample size is sufficiently large) if we combine online appointments with phone appointments. We can only recommend using this method so that further experiences can help test our assumptions on the basis of larger samples.

References

1. Bend, J. (2004). Public Value and E-Health. London: ippr.
2. Creemers, S. & Lambrecht, M. (2010). Queueing models for appointment-driven systems. Annals of Operations Research, 178(1), 155-172.
3. Czajka, S. & Jechová, P. (2012). Der Einsatz von Computer und Internet in privaten Haushalten in Deutschland. Ergebnisse der Erhebung 2011. In Statistisches Bundesamt (Ed.), Wirtschaft und Statistik, Mai 2012. Wiesbaden: Statistisches Bundesamt.
4. Diekmann, A. (2008). Empirische Sozialforschung. Grundlagen, Methoden, Anwendungen. Reinbek bei Hamburg: Rowohlt.
5. Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail, and mixed-mode Surveys: The tailored design method (3rd ed.). New Jersey: Wiley & Sons, Inc.
6. Döring, T. & Schick, A. (2002). Phasenbildung und Typisierung von Terminvereinbarung als Voraussetzung für die Entwicklung von E-Appointment-Anwendungen. Baccalaureatsarbeit, Universität Hamburg, Hamburg.
7. Forschungsgruppe Wahlen e.V. (2012). Internet-Strukturdaten. Repräsentative Umfrage – I. Quartal 2012, available at http://www.forschungsgruppe.de/Umfragen/Internet-Strukturdaten/ web_I_12.pdf.
8. Green, J., McDowall, Z., & Potts, H. W. W. (2008). Does Choose & Book fail to deliver the expected choice to patients? A survey of patients’ experience of outpatient appointment booking. BMC Medical Informatics and Decision Making, 8(36).
9. Initiative D21 & TNS Infratest GmbH (Eds.). (2011). (N)Onliner Atlas 2011. Eine Topographie des digitalen Grabens durch Deutschland. available at http://www.initiatived21.de/wp-content/uploads/2011/07/NOnliner2011.pdf.
10. Klischewski, R. (2003). The Challenges of e-Appointment: Process Modeling, Infrastructure, and Organizational Context. Paper presented at the 2003 International Business Information Management Conference, Cairo, Egypt (December 16-18, 2003).
11. Li, Y., Voudouris, C., Thompson, S. G., Owusu, G., Anim-Ansah, G., Liret, A., Lee, H., & Kern, M. (2006). Self-service reservation in the fieldforce. BT Technology Journal, 24(1), 40-47.
12. Lorenz, C., Schmitt, M., Lehrl, S., Mudiappa, M., & Roßbach, H.-G. (2013). The Bamberg BiKS Research Group. In M. Pfost, C. Artelt & S. Weinert (Eds.), The Development of Reading Literacy from Early Childhood to Adolescence. Empirical Findings from the Bamberg BiKS Longitudinal Studies. Bamberg: University of Bamberg Press.
13. Müller, T. & Beisch, N. (2011). Onlinenutzung von Migranten. Media Perspektiven, 10/2011, 493-503.
14. Parmar, V., Large, A., Madden, C., & Das, V. (2009). The online outpatient booking system ‘Choose and Book’ improves attendance rates at an audiology clinic: a comparative audit. Informatics in Primary Care, 17, 183-186.
15. Royston, P. & White, I. R. (2011). Multiple imputation by chained equations (MICE): Implementation in Stata. Journal of Statistical Software, 45(4), 1–20.
16. Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. New York; Chichester; Brisbane; Toronto; Singapore: John Wiley & Sons.
17. Schenker, N. & Taylor, J. M. G. (1996). Partially parametric techniques for multiple imputation. Computational Statistics & Data Analysis, 22, 425–446.
18. Schulz, H. (2008). E-Appointment eröffnet neue Möglichkeiten für die Terminvergabe. Unternehmenspraxis, 6, 64-66.
19. van Eimeren, B. & Frees, B. (2010). Fast 50 Millionen Deutsche online – Multimedia für alle? Media Perspektiven, 7-8/2010, 334-349.