With this increasing experience, however, also disadvantages of using vignettes appeared. A number of studies was devoted to the analysis of the appropriate number of vignette dimensions and, related to this, fatigue effects, learning effects, and order effects (Auspurg & J\u00e4ckle, 2017; Sauer et al., 2020). Other studies considered the relevance of respondents\u2019 features for response times and response consistency (Sauer et al., 2011), as well as influences of different answering scales or the presentation form of vignettes, i.e. text or table (Sauer et al., 2020; Shamon et al., 2022; Treischl & Wolbring, 2022). Most importantly, however, researchers questioned the degree of correspondence between real-life situations and presented situations in a factorial survey. Methodological studies revealed systematic differences between real-life and vignette-based measures of attitudes, intentions, or behaviours, and, more specifically, they also found evidence of socially desirable response behaviour within the framework of factorial surveys. Overall, it is still an open question whether factorial survey experiments actually help to improve measurement quality of normative judgments, subjective beliefs or behavioral intentions (Eifler & Petzold, 2019; Petzold & Wolbring, 2018).<\/p>\n
In particular, the presentation form of factorial survey experiments still needs further clarification. In principle, there are different formats of presenting vignettes within the framework of a survey: The situation can either be described in a written form or presented by visual stimuli, e.g. by videos, photos or pictures. While the majority of studies apply written vignettes (Wallander, 2009), some studies use solely video vignettes (Krysan et al., 2009) or solely photo vignettes (Golden III et al., 2001). Another study combines written and photo information within single vignettes (Havekes et al., 2013). Couper et al. (2004) found out that presenting a question with an additional photograph influences systematically the respondent\u2019s answer by clarifying the meaning of the question and simplifying its interpretation though included images didn\u2019t increase the respondent\u2019s motivation and satisfaction with the survey.<\/p>\n
To our best knowledge, only two studies compared observed responses to both written and visual stimuli focusing on video vignettes (Rashotte, 2003; Eifler, 2007). Because these studies suggested that different mechanisms apply when presenting photo vignettes and video vignettes we decided to focus on the presentation of visual stimuli in the form of photo vignettes for a first theoretical and empirical approach to analysing systematic differences between written and photo presentation formats of vignettes (see Eifler & Petzold, 2022 for details).<\/p>\n
Theoretical considerations<\/h2>\n
We start from the idea that different mechanisms apply when respondents are confronted with either verbal or visual information (Harper, 2002). McCloud (1994) put these different mechanisms in a nutshell using the idea of a smiling face as an example. Seeing<\/em> the image of a smiling face does not require formal education to decipher it. The message is received<\/em> instantaneously. Reading a text like \u2018Two eyes, a nose, a mouth with the corners of the mouth turned up…\u2019, on the contrary, requires special knowledge to decode the idea of a smiling face from the language. The message is perceived<\/em>. In particular, the dual coding theory (DCT; Paivio, 1979) allows for a systematic analysis of such processing differences regarding verbal and visual information. DCT suggests two coding systems in human memory, one of which is responsible for language or verbal information, the other for pictures or non-verbal information. Both systems overlap and can operate simultaneously in principle. Processing verbal and\/or visual information generates \u201cinternal mental images\u201d (Sadoski & Paivio, 2013, p. 29) which represent information about situations. It is assumed that mental images of situations match experiences with the same situations (e.g. Kosslyn & Pomerantz, 1977). Both written and visual information about situations is processed by both systems, but each in a different way: Written information is processed sequentially<\/em>, i.e. by the verbal coding system first and by the non-verbal coding system subsequently; visual information is processed simultaneously<\/em> by both coding systems at a time (Paivio, 1979; Sadoski & Paivio, 2013).<\/p>\n Because of the sequential processing of verbal information, written vignettes can elicit diverging encoding processes by readers, thus leading to diverging mental images between subjects. Because of the simultaneous processing of visual information, photo vignettes facilitate a more standardized perception of the concrete situation without any loop way, thus leading to corresponding mental images of the presented situations between subjects. While verbal information requires more extensive information processing by a reader and more background knowledge (perceived information), visual information presents the information directly (received information, see McCloud, 1994). It follows from these considerations, that photos can be considered to mirror real-life (Manghani, 2013; Rose, 2012) and to serve as a concrete point of reference for all who are confronted with them (Marion & Crowder, 2013).<\/p>\n With regard to factorial survey experiments, photo vignettes not only allow for a more realistic presentation of the situations under study but also for evoking the feeling of experiencing the particular situation. While written vignettes facilitate a sequential presentation of information in the form of short stories, photo vignettes present the information simultaneously in the form of pictures, thereby activating visual and verbal mental representations and leading to emotional arousal at the same time.<\/p>\n In line with these considerations, we assume that the presentation of photo vignettes facilitates processing the information presented in factorial surveys. Consistently, we expect reduced dropout rates (H1a), reduced processing time (H1b), and a more positive respondent\u2019s evaluation (H2) in a factorial survey based on photo vignettes compared to a factorial survey using written vignettes.<\/p>\n We conducted a split ballot experiment with a between-subjects design for examining presentation format differences in a factorial survey experiment. For a first empirical approach, we regarded a homogeneous sample as appropriate. The study was part of a web survey on attitudes towards safety in public places, conducted at the Martin-Luther-University Halle-Wittenberg, Germany in 2014 (Schwarzbach & Eifler, 2020), and was carried out as a full population survey of all enrolled students (N=20.000). An overall number of n=1.449 students started to answer the survey and n=1.099 completed the survey providing sufficient information. Each subject was randomly assigned to one of the two presentation formats, using either written vignettes or photo vignettes. Table 1 contains the descriptive statistics for the sample.<\/p>\n Table 1: Descriptives<\/p>\n We use written and photo vignettes that describe varying situations of everyday experience according to the broken windows theory for the prediction of fear of crime (Wilson & Kelling, 1982; Kelling & Coles, 1996; Keizer et al., 2014; Keuschnigg & Wolbring, 2015). The two factorial surveys are based on the same 24<\/sup>\u00b73-within-subjects design (see Table 1, Appendix) where the dimensions referred to signs of social and physical disorder in urban neighborhoods that had been used in previous studies (Piquero, 1999; Taylor, 1999). A total of 48 vignettes resulted from a full combination of the dimensions and their levels. These were subdivided into six sets of eight vignettes each and presented to the subjects. Because our focus here is on the differences between written and photo vignettes, we used the aggregated data referring to the respondent\u2019s level. For detailed information about the varied dimensions and levels see Table 1 in the appendix. A complete documentation of all written vignettes and their respective visual counterparts is available in the project description (Schwarzbach & Eifler, 2020). Figure 1 shows examples of photo vignettes and their respective written counterparts.<\/p>\n Figure 1: Examples of photo vignettes and written vignettes<\/p>\n To evaluate the design\u2019s internal validity, it was examined how far the randomization of subjects across the experimental conditions of the split ballot experiment for analyzing presentation format differences of the factorial survey led to a parallelization of experimental groups. As follows from Table 2 (Appendix), there are no substantial differences with regard to the distribution of the respondents\u2019 age and gender across the two levels of the split ballot experiments. Deviations in respondents\u2019 experience with the situation is minimal. Accordingly, randomization of subjects to the presentation formats, i.e. written vignettes or photo vignettes, led to mostly parallel groups.<\/p>\n The key dependent variables referred to (a) paradata collected in the course of the web survey, i.e. the dropout rate and the required processing time, and (b) evaluation data based on self-reports of the respondents concerning the level of fatigue [Item text: \u201cThis survey was tiring at times.\u201d, response scale: \u201cyes\u201d, \u201cno\u201d] when completing the respective factorial survey that they had been assigned to. We did not include, however, item-nonresponse as a dependent variable as the level of item-nonresponse was low with a very skew distribution on the vignette level [M=1.23; SD=1.16; Skewness=3.13; Kurtosis=16.91], so that analyses of unit non-response provide clearer and less biased results.<\/p>\n Estimations are controlled for respondents\u2019 subjective reports whether or not they were able to put themselves in the presented situations, thereby evaluating the vignettes as realistic [Question text: “Were you able to put yourself in the situations presented based on the descriptions?”, response scale: \u201cyes\u201d, \u201cno\u201d] and for the reported experience with the situation presented to the subjects [Question text: \u201cHave you been in a similar situation yourself?\u201d, response scale: \u201coften\u201d, \u201cone to two times\u201d, \u201cnever\u201d.] Both questions were part of our split ballot-experiment and are not used as a standard procedure in factorial survey experiments. Also, age and gender were used as control variables. The respondents\u2019 education was not introduced as a control variable because the education distribution has a high level of homogeneity within the sample.<\/p>\n A comparison between formats revealed no relevant differences between presentation formats with regard to the dependent variable [Question text: \u201cHow safe would you feel in this situation?\u201d, rating scale: \u201cvery safe\u201d, \u201csafe\u201d, \u201cunsafe\u201d, \u201cvery unsafe\u201d. This question was asked immediately after presenting each vignette.] measured by means of the factorial survey. Most subjects reported an overall low level of feelings of safety. A majority of subjects indicated that they feel very safe or safe (see Table 3, Appendix). This reflects that both formats stimulated similar responses on the aggregate level, i.e. across all vignettes.<\/p>\n In order to test our hypotheses regarding presentation format differences, three regression models with presentation format as independent variable and the dependent variables dropout rate, processing time and respondent\u2019s self-reported fatigue have been calculated using respondent\u2019s gender and experience with the presented situations as control variables. The respective average marginal effects (AMEs) for the three regression models (M1-M3) are reported in Table 2.<\/p>\n Table 2: Regression models: Effect of presentation format on dropout rate, processing time and respondents\u2019 fatigue<\/p>\n It follows from Table 2 that no differences regarding the dropout rate occur between presentation format (AME=-0.04, p=0.12, n.s.). Overall, the dropout rate is with 24,15% relatively low in both formats as can be taken from Table 1. Accordingly, our results do not support hypothesis 1a which stated that respondents of photo vignettes are less likely to dropout the survey. As for the processing time, a significant effect of the presentation format is revealed (\u03b2\/AME=-53.90, p\u2264.001). The results indicate that respondents who receive photo vignettes need about 54 seconds less time for processing the factorial survey compared to respondents who receive written vignettes. It can be concluded that photo vignettes ease the cognitive processing of the depicted information such that this information can be retrieved more quickly from memory. Accordingly, hypothesis 1b is supported by our results.<\/p>\n With regard to respondent\u2019s self-reported fatigue, hypothesis 2 referred to the assumption that respondents do easier and report less fatigue when processing photo vignettes as opposed to written vignettes. It can be taken from Table 2 that our results support this assumption. The probability that respondents get tired in the course of processing the factorial survey, is reduced by about 19% on average when respondents go through photo vignettes instead of through written vignettes (AME=-0.18, p\u2264.001).<\/p>\n Overall, we interpret our results as pointing towards the advantages of using photo vignettes in factorial surveys. This applies to the reduced processing time and to lower feelings of fatigue in the course of the survey when using photo vignettes instead of written vignettes. A lower responding time additionally should lead to a higher concentration and commitment of respondents in the course of the survey. Because no differences between the presentation formats occur with regard to dropout rates, the present study does not indicate that factorial surveys based on photo vignettes would be superior with regard to respondent\u2019s willingness to participate. Our results also leave questions of data quality when using different presentation formats of vignettes open. Furthermore, the respondents in this study were homogeneous with regard to their level of education. Therefore, the present study allowed for an isolated test of presentation format differences while keeping cognitive abilities constant during the survey. However, the explanatory power of the models for the analysis of presentation format differences was low. This means that our dependent variables might have been influenced by other features of the data collection procedure that were beyond our control. Further studies will have to clarify whether there is an interaction between the presentation form of vignettes and respondents\u2019 cognitive abilities or features of the data collection procedure. In addition, further studies will have to show whether photo vignettes might facilitate less socially desirable responding. Finally, it must be considered that for using photo vignettes the interested situation needs to be visually displayable.<\/p>\n <\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" Research Problem Factorial survey experiments in the tradition of Paul F. Lazarsfeld and Peter H. Rossi are used to measure normative judgements, subjective beliefs or behavioral intentions by respondent\u2019s answers to a number of brief descriptions of hypothetical situations, persons, or objects, called vignettes. The approach is based on the assumption that vignettes allow to […]<\/p>\n","protected":false},"author":3426,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[813,814,815,816,264],"class_list":["post-16770","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-factorial-survey-approach","tag-presentation-format-differences","tag-verbal-vignettes","tag-visual-vignettes","tag-web-survey"],"acf":[],"_links":{"self":[{"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/posts\/16770","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/users\/3426"}],"replies":[{"embeddable":true,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=16770"}],"version-history":[{"count":37,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/posts\/16770\/revisions"}],"predecessor-version":[{"id":19472,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/posts\/16770\/revisions\/19472"}],"wp:attachment":[{"href":"https:\/\/surveyinsights.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=16770"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=16770"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=16770"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Empirical Study<\/h1>\n
![\"\"](\"https:\/\/surveyinsights.org\/wp-content\/uploads\/2022\/02\/Table_1.png\")
<\/a><\/p>\n<\/a><\/p>\nResults and Conclusions<\/h1>\n
<\/a><\/em><\/p>\nAppendix: Additional material<\/a><\/h2>\n