Response Latency Measurement in Surveys. Detecting Strong Attitudes and Response Effects

Jochen Mayerl, Faculty of Social Sciences, Empirical Social Research, University of Kaiserslautern, Germany

1.03.2013
How to cite this article:

Mayerl, J. (2013). Response Latency Measurement in Surveys. Detecting Strong Attitudes and Response Effects. Survey Methods: Insights from the Field. Retrieved from https://surveyinsights.org/?p=1063

Copyright:

© the authors 2013. This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0) Creative Commons License


Abstract

The paper reports on measurement and data treatment of response latencies in computer assisted surveys. Applying response latencies as a measure of mental processes, empirical hypotheses are tested to explain the occurrence of response effects (here: acquiescence bias) and the predictive power of generalized attitudes. Theoretically, it is assumed that behavioural and other specific evaluative judgments in surveys are stronger influenced by acquiescence bias and generalized attitudes when answering in a rather automatic-spontaneous response mode, i.e. when response latencies are fast. Additionally, it is assumed that chronic attitude accessibility acts as a moderator of acquiescence effects and predictive power of attitudes within spontaneous mode processing. Empirical tests show evidence in favour of these assumptions, demonstrating the usefulness of response time measurement in surveys.

Keywords

, , , , , ,


Acknowledgement

The author would like to thank the anonymous reviewer for valuable comments and suggestions, which were very helpful to improve the paper.


Copyright

© the authors 2013. This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0) Creative Commons License


Introduction

The empirical identification and theoretical explanation of response effects and the predictive power of generalized attitudes on behavioural and other specific evaluative judgments in surveys still is a main topic of survey research – both to reach a better understanding of respondents’ behaviour and to enhance the quality of surveys.

The paper shows how response latency measurement can be used to detect (1) highly predictive generalized attitudes on specific behavioural, social or individual judgments, and (2) response effects like acquiescence bias in computer assisted surveys. Response latencies are a rather coarse method to “looking into the minds” of respondents, i.e. to analyse on-going cognitive processes while responding. In the subsequent analyses, response latencies are used as a proxy measure of spontaneous versus thoughtful responses. Theoretically, following common dual-process theories, two response modes of information processing are distinguished: answering survey questions in an automatic-spontaneous mode in contrast to answering in a deliberative-controlled mode. Methodologically, it is assumed that answering survey questions in an automatic-spontaneous manner needs less time than answering in a deliberative-controlled mode. The paper shows that response latencies can be used as a proxy measure of response mode when applying several data treatment steps to transform raw reaction times to response latencies.

In the following analyses, it is shown that the mode of information processing – i.e. response latency measurement – acts as a moderator of the predictive power of attitudes as well as of the occurrence of acquiescence effects (as a very frequent type of response effects).

The paper is structured in three main sections. Response latency measurement introduces how to apply response latency measurement in computer assisted surveys and discusses important data treatment steps. In the next section, the theoretical background of response effects and predictive power of attitudes is discussed and a dual-process response model is presented. Several hypotheses will be derived, explaining the strength of effects of social attitudes and the occurrence of response effects (here: of acquiescence bias). In Empirical analysis these hypotheses will be tested empirically by a split ballot experiment and structural equation models. Finally, the last section gives several conclusions derived from the empirical results.

Response latency measurement

Reaction time measurement is a promising method to gain data about behavioural and cognitive processes of respondent’s behaviour in survey research. Since the end of the 19th century, reaction time measurement has been a common method in (social) psychology to operationalize latent mental processes (see Bassili, 2000; Faust, Balota, Spieler, & Ferraro, 1999). “There simply isn’t much else that can be measured.“ (Pachella, 1974, p. 43). The development of computer assisted survey techniques allows measurement of response latencies in CATI, CAPI, and CASI surveys. In the context of telephone surveys especially the research results of Bassili and Fletcher have been very important for the development of response latency measurement in even large-scale representative CATI surveys (Bassili, 1993, 1995, 1996a, 1996b; Bassili & Fletcher, 1991). Several advantages are associated with reaction time measurement: it can be implemented easily and cost-effectively in computer assisted surveys, the measure is unobtrusive and thus not consciously biased by respondents and one gets the double amount of information about each answer (i.e. the substantial rating of the respondent and the time the answer needed).

In general, response latencies in surveys can be used as proxy variables for a wide range of mental processes. Many applications use response latency to measure the mode of information processing respectively the degree of elaboration during the answering process (e.g. Baxter & Hinson, 2001; Carlston & Skowronski, 1986; Gibbons& Rammsayer, 1999; Hertel, Neuhof, Theuer, & Kerr, 2000; Sheppard & Teasdale, 2000; Urban & Mayerl, 2007).[1] In this context it is assumed that response latency becomes slower when the degree of elaboration increases. Using median split methods (e.g. Fazio, 1990b; Urban& Mayerl, 2007) response latency can also be used as a proxy of the two typical modes of information processing. Thus, slow response latencies indicate activity of the deliberative-controlled mode, and fast response latencies indicate automatic-spontaneous processing.

This interpretation of response latencies is empirically supported by results showing that an inconsistent information basis leads to slower response latencies (Bassili, 1996b; Brömer, 1999; Klauer & Musch, 1999). Further, response latency increases with the amount of considered information (Bassili & Scott 1996; Houlihan et al., 1994).

To gain information about mental processes via response times, measuring and treating these data accurately is essential for avoiding a biased interpretation of results of response time analyses. Several steps are needed to gain accurate response latency data:

–          Step 1: Measurement of raw reaction times in computer assisted surveys: active by interviewer or passive;

–          Step 2: Identification and treatment of invalid measures by outlier treatment (2.1) and timer validation by interviewers (2.2);

–          Step 3: Controlling for individual baseline speed: Measurement of baseline speed and statistical control by transformation of raw reaction times to response latencies;

–          Step 4 (optional): additional data treatment to gain normal distribution (e.g. taking the logarithm, reciprocal transformation) and/or aggregation of latency data (e.g. mean of latencies, median split etc.).

Applying step 1, reaction times can be measured with any computer assisted survey method. In case of interviewer controlled surveys (CATI and CAPI), it is recommended to use active measurement of the reaction times by a “four-screens-per-question-technique”: after having read the question (screen 1), the interviewer immediately triggers the time measurement by pressing a key and a blank screen appears in the CATI/CAPI software (screen 2). The time screen 2 is visible corresponds to the response time measurement. As soon as the respondent answers substantively, the interviewer stops the time measurement by pressing a key and screen 3 appears, where the interviewer enters the given answer of the respondent. Additionally, it is possible to ask the interviewers for timer validation (screen 4, see step 2). This can easily be done for each question in a survey (see Mayerl & Urban, 2008). Passive reaction times (also known as latent reaction times), on the other hand, measure the whole process of a question in a survey, including reading time and time entering the answer. Therefore, these times are more biased and inaccurate than active times. In web surveys and other CASI systems without a present interviewer, passive times are the only way to measure reaction times. Interestingly, several studies have shown that passive times lead to the same substantial results in comparison to active times (Mayerl & Urban, 2008; Mulligan, Grant, Mockabee, & Monson, 2003). In each case reaction times should be measured in hundredth seconds or milliseconds to get enough variance of the time data.

In step 2, invalid reaction times have to be identified to gain meaningful time data. Most reaction time studies control for outliers. Usually, reaction times are defined as invalid when they are out of the range of two standard deviations above or below the mean (this method is applied in the following analyses; several alternative thresholds are proposed and applied in latency literature, see Mayerl & Urban, 2008, p. 59). This is reasonable because some times are very long and it is very likely that these respondents have been distracted while answering the survey. Additionally, interviewers can validate each response latency measurement. In our study (see Data and sample for a description of the study design), we used three codes: (1) valid measurement, (2) error by interviewer (e.g. triggered too late or too early, jumping across the questions), (3) error by respondents (e.g. asked a question, used wrong response scale, diversion, comments). Overall, 89% of all response latency measurements were coded valid by the interviewers, in 2% of all latency measurements interviewers made errors and in 9% of all cases respondents caused invalid response latency measurements. Invalid measurements (in terms of outliers and interviewer-validation) were defined as missing values in the following empirical analysis (see Mayerl & Urban, 2008 and Urban & Mayerl, 2007 for detailed information of data treatment used in this study). Alternatively, common missing value treatments can be applied in a further step (e.g. multiple imputation).

Having identified invalid reaction times, the statistical control of individual baseline speed is elementary for the analysis of response times (step 3). Baseline speed is defined as the general mental speed that a person needs to answer questions, independent of the content of the question. Without the control of baseline speed, „[…], one is simply identifying a continuum of generally fast versus slow responders.“ (Fazio, 1990b, p. 87). To control baseline speed, several transformation indices have been proposed in literature, e.g. ratio index, difference scores, z-scores, rate-amount index, and residual index (see Fazio, 1990b; Mayerl & Urban, 2008). In the present study the so called “Residual Index”, proposed by Mayerl, Sellke, & Urban (2005), will be used as transformation index to control baseline speed differences. The logic of the Residual Index is simple: the residuals of a linear regression of raw reaction times on a baseline speed measure can be used as a response latency measure that is independent of baseline speeds (more details see Mayerl et al., 2005).[2]

Ideally, baseline speed is operationalized by the mean of reaction times of questions which are not part of the analyses (so called filler latencies). In the following analysis, the mean of reaction times of three simple factual questions were used as a measure of baseline speed: confession, year of birth, and vegetarian (yes-no).[3][4]

Undoubtedly, response time measurement depends on a lot of additional biasing factors which can be distinguished into characteristics of the measurement instrument, the interview situation, and the respondents. The fourth source of response times is the ongoing mental process during answering a survey – this is what researchers are typically interested in, from a theoretical or substantial perspective (see Figure 1, for more details Mayerl & Urban, 2008, p. 32ff). It is important to note that the baseline speed measure itself depends on a lot of methodological (e.g. interviewer speed) and individual (e.g. age) biasing factors – and this means taking baseline speed into account implies controlling for a wide range of disturbing factors (see Mayerl & Urban. 2008, p. 69f). This does not mean that other factors do not have to be controlled in general, but that controlling for baseline speed is an essential and necessary step for the treatment of response time measures and their proper interpretation as a proxy measurement of mental processes.

Figure 1: Determinants of response time

The last step of data treatment (step 4) is optional and depends on types of statistical procedures which are intended to be used. Response times usually are positively skewed due to some very long times. This skewness is reduced by outlier treatment, but in most cases reaction times still are not normally distributed. Thus, if normal distribution is necessary, reaction times can be transformed by taking the logarithm (alternatively by reciprocal transformation or by taking the square root). Also, reaction times sometimes are aggregated by taking the mean of several time measures. Additionally, aggregation of response times can be achieved by recoding (e.g. to use response time as grouping variable in a extreme group comparison) via median split, tertile split, quartile split etc. (see Fazio, 1990b).

Looking at all these possibilities of operationalization and data treatment, it is obvious that comparing results of response latency studies with different data treatments can be seriously biased. When results are inconsistent, influence of data treatment has to be inspected.

In the following sections it is shown how response latencies can be used as a proxy measure of mode of information processing in the context of dual-process models to analyse the predictive power of attitudes and the occurrence of response effects (here: acquiescence bias) in surveys.

Theoretical background

Dual-process model of response behaviour

In social psychology it is very popular to model cognitive processes as the duality of two different cognitive systems or modes of information processing, assuming an automatic, spontaneous and non-conscious mode on the one hand, and a deliberative, controlled, conscious and intention-based mode on the other hand.

Most common dual-process models share these basic assumptions. Besides this fundamental distinction of two modes of information processing, a lot of dual-process models were proposed in literature, each with different terms and some compatible and some incompatible additional assumptions. Some of the most popular dual-process models are Heuristic-Systematic Model (Chaiken, 1980), Elaboration Likelihood Model (Petty & Cacioppo, 1986), MODE-model (Fazio, 1990a), satisficing vs. optimizing in survey context (Krosnick, 1991), and Reflective-Impulsive Model (Strack & Deutsch, 2004). Incompatibilities of these models will not be discussed in this paper (see Chaiken & Trope, 1999; Keller, Bohner, & Erb, 2000; Smith & DeCoster, 2000). Instead, a generic dual mode model of attitude-behavior-research (with focus on response behavior in surveys) will be used based on five assumptions which are shared by most of these dual-process theories:

Assumption 1: A qualitative distinction is made between two modes of information processing: unconscious automatic-spontaneous vs. conscious deliberative-controlled. Automatic-spontaneous processes can be characterized as top-down, using simple heuristics (e.g. response sets like acquiescence in surveys), situative or contextual cues (e.g. interviewer characteristics, question order) or overall evaluations of objects (i.e. generalized attitudes) to answer survey questions. In contrast, deliberative-controlled processes are bottom-up processes, generating a judgement upon the retrieval of “raw data“ (e.g. beliefs).

Assumption 2: The elaboration continuum is the quantitative dimension underlying the two modes: conscious deliberative-controlled mode as the pole of high elaboration and unconscious automatic-spontaneous mode as the pole of low elaboration.

Assumption 3: Motivation and opportunity are necessary and sufficient predictors of the appearance of the modes: the higher both, the motivation and the opportunity to process information in a deliberative way, the higher the degree of elaboration and thus the probability of the usage of the deliberative-controlled mode. Thus, the lower motivation and/or opportunity, the higher is the probability of the activation of the automatic-spontaneous default mode.[5]

Assumption 4: According to the MODE model, the chronic accessibility of an attitude is assumed to be relevant only in the case of spontaneous processing (Fazio, 1990a, p. 93f, 104; Fazio & Roskos-Ewoldson, 2005, p. 58).[6][7] Thus, chronic attitude accessibility is assumed to be a moderator of the attitude-response-process (i.e. that the “real“ attitude is given by the respondent) only in the case of spontaneous mode. In respect to response effects, this means that chronic attitude accessibility is a moderator of the influence of response effects within the spontaneous mode: the higher the chronic accessibility, the lower the influence of response effects.

Assumption 5: A lot of dual-process theories assume that it is possible that the two modes may co-occur. In this paper, the assumption of co-occurrence is combined with the trade-off assumption of the Elaboration Likelihood model, stating that the two modes do not influence cognitive processes independently of each other. Instead, it is assumed that the influence of the spontaneous mode decreases and the influence of the deliberative mode increases with an increasing degree of elaboration.

In sum, on the basis of this dual-process model, at least three types of response behavior can be distinguished theoretically. To simplify the model, as done in Figure 2, elaboration and chronic accessibility will be reduced to a binary scaling level to identify typical forms of response behavior:

(1a) attitude-based automatic-spontaneous responding;

(1b) cue-/heuristic-based automatic-spontaneous responding;

(2) deliberative-controlled responding.

Figure 2: Dual-process model of response behavior

As reported, three conditions can be used to differentiate these three types of response behavior: motivation, opportunity, and chronic attitude accessibility. Response type (2) occurs under the condition of high motivation and high opportunity. Type (1a) occurs in the case of low motivation and/or low opportunity and high chronic attitude accessibility. Thus, type (1b) is expected to appear in the case of low motivation and/or low opportunity and low chronic attitude accessibility.

In the subsequent empirical analysis, response latencies will be used as proxy measure of mode of information processing, thus allowing to adopt this dual-process model of response behavior.

Types of response effects

In survey research, response effects are commonly defined as influences on response behavior which prevent that a “true” answer/value is given by the respondent. These influences may come from the measurement instrument (e.g. question order, question wording), interview situation (e.g. sponsorship effects, social desirability) or individual factors of the respondents (e.g. individual tendency to acquiescence, usage of response-sets). In general, two main types of response effects can be distinguished: Firstly, response effects may be issue-related, i.e. the use of temporarily accessible information related to the question issue (e.g. question order, social desirability, wording, sponsorship effect). Secondly, response effects may also be issue-unrelated, i.e. the use of response-sets and simple decision heuristics (e.g. acquiescence, tendency to middle category).

In this paper, it is argued that the occurrence of response effects mainly depends on how much mental effort is needed to apply such response effects. Acquiescence and other simple decision heuristics can be applied rather automatic-spontaneously (e.g. simple agreeing independent of question content). Especially the tendency to agree, i.e. acquiescence bias, is an easy applicable and well-learned conversational social norm that fits to a lot of situations – just because of the fact that agreement usually needs less or no explanation in contrast to disagreement. This is also true for other response effects without demand of high cognitive efforts, e.g. assimilation (carryover) effects of question order. On the other hand, some response effects lead to high mental efforts and thoughtful processes, e.g. contrast (backfire) effects of question order, which need subtraction or comparison of contents (Strack & Martin, 1987; Sudman, Bradburn, & Schwarz, 1996; Tourangeau & Rasinski, 1988; Tourangeau, 1992). Also, wording problems of questions may lead to high mental efforts of respondents trying to understand what is asked, or trying to fit the answering categories to the question.

In literature, a wide range of predictor variables have been identified. For example, causes of the occurrence of issue-unrelated response effects can be: lack of motivation to think about the issue, lack of time to think about the issue, lack of any knowledge about the issue, lack of interest on the issue or – more general – the survey at all, and a personal psychological tendency to generally agree, disagree or abstain (e.g. Knowles & Condon, 1999; Tourangeau & Rasinski, 1988).

Response effects may also rely on situative cues. In this case, it depends on the specific situation, whether it is issue-related (e.g. social desirability initiated by characteristics of the interviewer or the sponsor of the study) or issue-unrelated (e.g. effects of the actual mood of the respondent).

Combining the shown theoretical dual-process model (Figure 2) with these response effect explanations, it is obvious that some response effects are expected to appear in automatic-spontaneous response mode and others in deliberative-controlled mode. Additionally, in case of automatic-spontaneous mode, these response effects are expected to appear only when there is no chronic attitude accessible – thus leading to a cue-/heuristic-based response behaviour (response type 1b in Figure 2). These assumptions will be tested empirically in the subsequent sections for the case of acquiescence bias.

Hypotheses

The following hypotheses can be concluded out of the presented dual-process model of response behaviour and will be tested empirically in the next section. The corresponding research question is whether it is possible to use response latencies to predict (a) strong vs. weak effects of generalized attitudes towards targets on  specific judgments in surveys (e.g. on specific personal characteristics or individual behaviour), and (b) the occurrence of acquiescence bias in surveys as a special type of response effect.

H1.1:     If respondents answer in the automatic-spontaneous mode of information processing, acquiescence effects are more likely to appear in comparison to deliberative-controlled mode processing.

H1.2:     If respondents answer in the automatic-spontaneous mode of information processing, generalized attitudes are more predictive towards specific judgments in surveys (e.g. towards judgments about specific behaviour) in comparison to deliberative-controlled mode processing.

H1.3:     If respondents answer in the deliberative-controlled mode of information processing, raw data of decision making (“beliefs”) are more predictive towards specific judgments in surveys (e.g. towards judgments about specific behaviour) in comparison to automatic-spontaneous mode processing.

H2.1:     If respondents answer in an automatic-spontaneous mode of information processing and chronic attitude accessibility is low, acquiescence effects are more likely to appear in comparison to (a) automatic-spontaneous processes with high attitude accessibility and (b) deliberative-controlled information processing.

H2.2:     If respondents answer in an automatic-spontaneous mode of information processing and chronic attitude accessibility is high, generalized attitudes are more predictive towards specific judgments in surveys (e.g. towards judgments about specific behaviour) in comparison to (a) automatic-spontaneous processes with low attitude accessibility and (b) deliberative-controlled information processing.

Hypotheses H1.1, H1.2 and H1.3 refer to the distinction between two modes of information processing (spontaneous vs. deliberative). Hypotheses H1.1 and H1.2 assume that the mode information processing acts as a moderator of effects of attitudes and response sets (here: acquiescence), where stronger effects are expected in spontaneous mode. According to H1.3, in deliberative response mode behavioural judgments should base on a broader basis of “raw data” about evaluations and expectations on possible outcomes.

Hypotheses H2.1 and H2.2 are assumptions on the role of attitude accessibility within the modes of information processing as shown in Figure 2, expecting spontaneous response effects and stronger effects of generalized attitudes to occur under special conditions only: generalized attitudes towards targets are assumed to guide specific judgments (e.g. behavioural judgments) strongest under the condition of ‘automatic mode and high attitude accessibility’, and acquiescence bias is assumed to occur under the condition of ‘automatic mode and low attitude accessibility’.

These hypotheses will be tested by the experimental variation of question wording in a split ballot design and by the estimation of structural equation models.

Empirical analysis

Data and sample

The data used for this analysis has been conducted in the DFG-founded project “Response latency measurement in survey research. Analysing the cognitive basis of attitudes and information processing”, carried out by the institute for social sciences at University of Stuttgart. The survey was conducted as a German nation-wide random sample CATI-survey[8] from 2nd February 2005 to 1st April 2005 with 2002 respondents.[9] Additionally, the study was designed experimentally with eight different questionnaire versions (regarding differences in question order and item wording). Reaction times were measured actively by interviewers during the process of interviewing for all items (in hundredth seconds). In the following analyses, reaction times were controlled for individual baseline speed by transformation to response latencies with residual index (see Response latency measurement). All latencies are controlled for outliers (cut at ±2 standard deviations above/below the mean) and timer validation by interviewers.

Subsequent empirical analyses will be done in two different ways: (1) a split ballot experiment will be used to examine acquiescence effects on aggregate level analysing judgments in survey towards individual psychological characteristics (Analysis on aggregate level); (2) structural equation models on individual level will be estimated to test hypotheses about effects of individual tendency to acquiescence, effects of generalized attitudes, and effects of beliefs predicting specific behavioural judgments (Analysis on individual level).

Analysis on aggregate level

In a split ballot experiment with two groups within our CATI study (Ni= 250 each), the wording of three items (5-point rating scale) of a psychological scale to measure need for cognition was varied in an original wording (condition A) and an opposite wording (condition B). In this sense, an acquiescence effect is expected in the way that respondents generally tend to agree independently of the content of the item (i.e. the direction of item wording). Therefore, it is assumed that a negative worded item (“not”, “none”, etc.) leads to a higher percentage of agreement in comparison to disagreement to the contrary mirror (i.e. positive worded) item. In the case of items 1 and 2 (see Table 1), the assumption is that the negation “not” of an otherwise identical expression is the exact contrary of the original meaning in terms of logical sense.[10] In case of item 3, it is more questionable whether the negated item may be the exact opposite (the negation “less” instead of “not” is used). Thus, results of item 3 should not be over-interpreted in case when no acquiescence effects should appear for the other two items.

For reasons of a better presentation in the Table 1 and 2, the 5-point rating scales were recoded into 3-point scales (1-2=agreement, 3=undecided and 4-5=disagreement). In the next step, the negative formulated items (condition B) were recoded so that “agreement” always means that there is an agreement towards the original wording.

Table 1: Acquiescence bias for three items of a need for cognition scale

  A: “original“
(in %)
B: “negative”
(recoded)
(in %)
difference
(in %)
Item1)
original: I like abstract thinking.
negative: I do not like abstract thinking.
1-2 agreement 48.4 40.7 7.7
3 undecided 24.0 20.7 3.3
4-5 disagreement 27.6 38.6 -11
∑100
N=246
∑100
N=246
\mathsf{\chi^2}=6.7, df=2, p=0.035, N=492
Item2)
original: I like to think for hours about something.
negative: I do not like hours of thinking about something.
1-2 agreement 47.6 46.8 1.2
3 undecided 26.8 24 2.8
4-5 disagreement 25.6 29.2 -3.6
∑100
N=250
∑100
N=250
\mathsf{\chi^2}=0.88, df=2, p=0.608, N=500
Item3)
original: I think it is exciting to learn new ways of thinking.
negative: I think it is less exciting to learn new ways of thinking.
1-2 agreement 63.5 55.8 7.7
3 undecided 27.7 18.1 9.6
4-5 disagreement 8.8 26.1 -17.3
∑100
N=249
∑100
N=249
\mathsf{\chi^2}=27.52, df=2, p=0.000, N=498

In Table 1, in two out of three cases (item 1 and item 3) the expected acquiescence effect is statistically significant (p ≤ 0.05). This means, the difference for both items between the two wording groups is 7.7% more agreement to the original wording compared to disagreement to the negative worded items. This is also true for disagreement to the original wording and agreement to the negative wording: in this case, the difference is 11% respective 17.3%.

To test hypothesis 1.1, assuming that acquiescence biases should appear in the spontaneous mode and not in the deliberative mode, response latencies (median split at each item) will be used as a measure of the mode of information processing. To test the hypothesis, an interaction model via ANOVA will be tested using the original 5-point rating scales to reach metric scales. Table 2 reports the results of this analysis.

Table 2: Mode of information processing and acquiescence effect

automatic-spontaneous mode
(response latency ≤ median)

deliberative-controlled mode
(response latency > median)

modera­tor effect of response mode (%)

A: “original” (in %) B: “negative” (recoded) (in %) differ- ence (in %) A: “original“ (in %) B: “negative”
(recoded) (in %)
differ- ence (in %)
Item1)
original: I like abstract thinking.
negative: I do not like abstract thinking.
1-2 agreement 59.8 31.4 28.4 30.2 46.8 -16.6 45
3 undecided 17.4 25.6 -8.2 33.3 19.4 13.9 22.1
4-5 disagreement 22.7 43.0 -20.3 36.5 33.9 2.6 22.9
ANOVA: Condition (“C“; 1: A; 0: B): p=0.470; Response latency (“RL“; 1: fast, 0: slow): p=0.243;
Interaction C
HRL: p=0.000; N=438
F=5.940; df=3; p=0.001
Item2)
original: I like to think for hours about something.
negative: I do not like hours of thinking about something.
1-2 agreement 56.4 42.5 13.9 34.0 48.8 -14.8 28.7
3 undecided 18.8 28.7 -9.9 39.2 22.0 17.2 27.1
4-5 disagreement 24.8 28.7 -3.9 26.8 29.3 -2.5 1.4
ANOVA: C: p=0.553; RL: p=0.268; CHRL: p=0.041; N=440
F=1.874; df=3; p=0.133
Item3)
original: I think it is exciting to learn new ways of thinking.
negative: I think it is less exciting to learn new ways of thinking.
1-2 agreement 70.4 57.9 15.8 48.3 51.9 -4.8 20.6
3 undecided 23.2 14.5 7.0 37.9 18.6 20.7 13.7
4-5 disagreement 6.3 27.6 -22.3 12.6 29.5 -15.9 6.4
ANOVA: C: p=0.203; RL: p=0.000; CHRL: p=0.088; N=433
F=7.495; df=3; p=0.000

In all three cases, the mode of information processing acts as a significant moderator of acquiescence bias. In the cases of items 1 and 2, the interaction effects are significant with p ≤ 0.01 (item 1) respectively p ≤ 0.05 (item 2) (it should be taken into account that the overall fit of the model of item 2 is fairly poor). In the case of item 3, the interaction effect is marginally significant with p ≤ 0.10 and response latency acts as a significant main predictor, too (fast respondents agree significantly more than slow respondents). Additionally, in all three cases, the main effect of the questionnaire version becomes non-significant. The fact that the interaction effect is only marginal significant in case of item 3 may be explained due to the less straightforward exact negation of the meaning of the negative item. But the overall result stays stable (or is even more consistent) when looking at items 1 and 2 only. Substantively speaking, strongest agreement appears when two conditions are given: short latencies (i.e. spontaneous processing mode) and a wording without negation. Interestingly, in case of item 2, it can be shown that a substantive response effect appears only when response latencies are implemented into the analysis (Table 1 versus Table 2). Thus, response latency measurement helps to identify acquiescence bias on aggregate level and acts as a moderator of this response effect. In sum, the result presented in Table 2 supports hypothesis H1.1.

Interestingly, results of Table 2 show evidence that respondents do not only tend to acquiescence in case of spontaneous responding, but in addition there seems to appear a contrary effect in case of deliberative responses: when response latencies are slow, more people (relatively speaking) disagree to the negative item in comparison to agreement to the positive item. This pattern appears for all three items. Thus, when respondent’s answer slowly in a deliberative manner, they are more “skeptical” than fast responders, and this seems to be forced by negative item wording. This finding is in line with literature showing that people with negative mood need more time to answer to survey questions (Fazio, 1995; Ruder, 2001).

Analysis on individual level

Another way of analysing acquiescence effects is to use a construct of general individual tendency towards acquiescence. To test the hypothesis whether such acquiescence effects occur in spontaneous mode (H1.1), and whether generalized attitudes towards targets are more predictive in spontaneous mode (H1.2) and beliefs are more predictive in deliberative mode (H1.3), a multiple group structural equation model of spontaneous versus deliberative responders in the domain of money donation towards welfare organizations will be reported.

In addition, a more complex model will be tested by differentiating two mode groups and two chronic attitude accessibility groups, testing the dual-process model presented in Figure 2 in a more detailed way. This allows analysing whether acquiescence bias occurs in a special case of spontaneous processing, i.e. under the condition of low chronic attitude accessibility (H2.1), and whether attitudes show strongest effects in case of spontaneous mode and high attitude accessibility (H2.2).

For these structural equation analyses, the same CATI data (see Data and sample) were used.[11] In the following model, a part of the Theory of Reasoned Action (Ajzen & Fishbein, 1980) will be applied: the explanation of an attitude towards donating money to welfare organizations (two indicators; 5-point rating scales with 1: agree to 5: disagree)[12]. According to Ajzen & Fishbein (1980, p. 84), specific attitudes towards behaviour are determined by several predictor variables. Firstly, the main predictors are behavioural beliefs, modelled in the tradition of expectancy-value models, thus assuming that it is the product of evaluation of outcomes times expectancy that behaviour leads to outcome which guides behavioural perceptions. According to dual process model of response behaviour (Figure 2), these rational outcome beliefs can be seen as “raw data”. Therefore, more beliefs should show significant effects in case of deliberative-controlled mode of information processing in contrast to automatic-spontaneous mode (H1.3). Secondly, background variables may influence specific attitudes towards behaviour. One class of background variables are generalized attitudes towards targets (see Ajzen & Fishbein 1980, p. 84). In attitude research, it is a central issue under which conditions generalized attitudes towards targets guide evaluative information processing and thus other social and behavioural judgments like specific attitudes towards behaviour.[13] Finally, individual tendency to use response styles in surveys (here: acquiescence) is another background variable which may influence behavioural judgments. According to our dual mode model, generalized attitudes towards targets and tendency to acquiescence are assumed to show stronger effects in case of automatic-spontaneous response mode (H1.1 and H1.2). In addition, in an expanded model it is assumed that attitude accessibility moderates effects of attitudes towards targets and acquiescence in case of automatic-spontaneous response mode (H2.1 and H2.2). In sum, the explanation model of specific attitudes towards behaviour (here: donating money to charity) integrates the following groups of predictors:

(a)      generalized attitude towards the target (i.e. development aid and charity organizations) (two indicators; 5-point rating scale with 1: agree to 5: disagree)[14];

(b)     four modal salient behavioural beliefs that were carried out in a preliminary study (N=120) (interaction term of evaluation (5-point rating scale) and probability (percentage rating), thus reaching a scale of values from 0 to 500 for each behavioural belief; due to scaling, negative effects are expected on attitude towards behaviour)[15];

(c)      an operative scale of tendency to agree (“acquiescence”) that was measured as the sum of answering with “1” (“totally agree”) to exact one hundred 5-point rating scales on a wide variety of contents that were conducted during the whole interview (reaching a scale of values from 0 to 100; again, due to scaling, negative effects are expected on attitude towards behaviour).[16] Thus, in contrast to the foregoing analysis, acquiescence is an individual measure now;

(d)     a measure of altruistic motivation (one item; 5-point rating scale with 1: agree to 5: disagree) as control variable.[17]

The mode of information processing is measured by median split of the mean of the response latencies of both indicators of the behavioural attitude construct. Chronic attitude accessibility of the attitude towards the target (i.e. towards charity organizations) is measured by membership in a charity organization, assuming that members do have a higher direct experience with that attitude object (membership yes: high attitude accessibility; membership no: low attitude accessibility). Thus, membership (as an indirect measure of direct experience) is a formative indicator of chronic attitude accessibility (Fazio, 1986, 1990a; Fazio, Chen, McDonel, & Sherman, 1982).[18]

Figure 3: SEM model explaining responses to behavioral attitude questions

The model shown in Figure 3 will be estimated in a multiple group structural equation model. In both groups (automatic-spontaneous mode versus deliberative-controlled mode), all factor loadings are highly significant and show standardized values above 0.5.[19] Both attitude measurement constructs were successfully externally validated.[20] Additionally, as common in multiple group analysis in SEM methodology, all unstandardized factor loadings, error variances[21] and error covariances were constrained to be equal between the groups to reach comparability of the coefficients of the groups.[22] Because all variables are nearly normal distributed (skewness <3 and kurtosis <10, see Kline 2011) a maximum likelihood estimator will be used with software Mplus.[23] Table 3 shows the results of the described structural equation model to test H1.1, H1.2, and H1.3.

Table 3: Two-group structural equation model (H1.1 and H1.2)

Automatic-spontaneous mode
(resp. latency ≤ median; N = 557)
  b SE t
Structural effects on Attitude towards behavior (AttBeh):
acquiescence -.061** .021 -2.889
attitude towards target (AttTarget) .799** .111 7.218
belief 1 -.127** .032 -3.919
belief 2 -.018 .019 -.956
belief 3 -.023 .020 -1.151
belief 4 -.018 .019 -.988
Altruism .027 .038 .706
R2 .687
Measurement models:
AttBeh → Item AttBeh-DA 1.000
AttBeh → Item AttBeh-CH 1.093** .075 14.598
AttTarget → Item AttTarget-DA 1.000
AttTarget → Item AttTarget-CH 1.371** .130 10.572
Altruism → Item ALT 1.000
Covariances of measurement errors:
AttBeh-DA ↔ AttTarget-DA .234** .024 9.708
AttBeh-CH ↔ AttTarget-CH .167** .027 6.298
Deliberative-controlled mode
(resp. latency > median; N = 557)
  b SE t
Structural effects on Attitude towards behavior (AttBeh):
acquiescence -.025 .022 -1.123
attitude towards target (AttTarget) .530** .108 4.883
belief 1 -.076* .030 -2.545
belief 2 -.014 .022 -.642
belief 3 -.043* .020 -2.203
belief 4 -.052* .021 -2.525
Altruism .069+ .035 1.952
R2 .478
Measurement models:
AttBeh → Item AttBeh-DA 1.000
AttBeh → Item AttBeh-CH 1.093** .075 14.598
AttTarget → Item AttTarget-DA 1.000
AttTarget → Item AttTarget-CH 1.371** .130 10.572
Altruism → Item ALT 1.000
Covariances of measurement errors:
AttBeh-DA ↔ AttTarget-DA .234** .024 9.708
AttBeh-CH ↔ AttTarget-CH .167** .027 6.298

Y = attitude towards donating money for charity organizations (AttBeh); DA = development aid; CH = charity; ** p ≤ 0.01; * p ≤ 0.05; + p ≤ 0.10; without marker: n.s. with p > 0.10

Overall fit: \chi^2 = 39.054; df = 28; p = 0.080; CFI = 0.993; RMSEA = 0.027 (CI0.90: 0.000 to 0.045); SRMR = 0.020

The goodness of fit of the model is good (see Table 3). The results reported in Table 3 show supporting evidence for H1.1 and H1.2: Acquiescence is only a significant predictor in the case of spontaneous processing (\chi^2-difference of 8.072, df=1, p=0.004). Additionally, the effect of generalized attitude towards target is significant stronger in spontaneous mode group (\chi^2-difference of 3.914, df=1, p=0.048).[24]

Further, the reported results show evidence for usage of response latency as an indicator of mode of information processing according to hypothesis H1.3: in case of slow response latencies, three out of four beliefs significantly affect the behavioural attitude in contrast to only one significant belief in the spontaneous mode. Thus, in line with hypothesis H1.3, slow (i.e. more thoughtful) responders indeed consider a broader information basis in contrast to fast responders.

Next, the presented model will be expanded by the implementation of chronic attitude accessibility of the attitude towards the target. By doing this, it is possible to estimate the complete dual-process model presented in Figure 2 and thus to test hypothesis H2.1 and H2.2. Table 4 reports the empirical results of this four-group structural equation model (two response mode groups and two attitude accessibility groups).

Table 4: Four-group structural equation model (H2.1 and H2.2)

Automatic-spontaneous mode
(resp. latency ≤ median; N = 557)
 

High attitude accessibility

(N = 244)

Low attitude accessibility

 (N = 313)

b SE t b SE t
Structural effects on Attitude towards behavior (AttBeh):
acquiescence -.053 .034 -1.541 -.064* .027 -2.400
attitude towards target (AttTarget) .950** .157 6.057 .715** .149 4.787
belief 1 -.097* .042 -2.285 -.157** .047 -3.358
belief 2 .008 .028 .285 -.044 .027 -1.644
belief 3 -.026 .032 -.794 -.017 .025 -.701
belief 4 -.036 .026 -1.406 -.003 .027 -.120
Altruism -.077 .066 -1.169 .092+ .049 1.885
R2 .781 .706
Measurement models:
AttBeh → Item AttBeh-DA 1.000 1.000
AttBeh → Item AttBeh-CH 1.087** .075 14.534 1.087** .075 14.534
AttTarget → Item AttTarget-DA 1.000 1.000
AttTarget → Item AttTarget-CH 1.401** .130 10.765 1.401** .130 10.765
Altruismus → Item ALT 1.000 1.000
Covariances of measurement errors:
AttBeh-DA ↔ AttTarget-DA .233** .024 9.786 .233** .024 9.786
AttBeh-CH ↔ AttTarget-CH .155** .026 5.962 .155** .026 5.962
Deliberative-controlled mode
(resp. latency > median; N = 557)
 

High attitude accessibility

(N = 244)

Low attitude accessibility

(N = 313)

  b SE t b SE t
Structural effects on Attitude towards behavior (AttBeh):
acquiescence -.052 .035 -1.487 -.003 .031 -.094
attitude towards target (AttTarget) .464* .192 2.421 .568** .130 4.378
belief 1 -.062 .046 -1.361 -.089* .039 -2.302
belief 2 -.025 .032 -.794 -.011 .030 -.370
belief 3 -.047 .029 -1.645 -.037 .026 -1.393
belief 4 -.021 .028 -.747 -.074* .031 -2.402
Altruism .017 .048 .357 .099+ .053 1.855
R2 .432     .452
Measurement models:
AttBeh → Item AttBeh-DA 1.000 1.000
AttBeh → Item AttBeh-CH 1.087** .075 14.534 1.087** .075 14.534
AttTarget → Item AttTarget-DA 1.000 1.000
AttTarget → Item AttTarget-CH 1.401** .130 10.765 1.401** .130 10.765
Altruismus → Item ALT 1.000 1.000
Covariances of measurement errors:
AttBeh-DA ↔ AttTarget-DA .233** .024 9.786 .233** .024 9.786
AttBeh-CH ↔ AttTarget-CH .155** .026 5.962 .155** .026 5.962

Y = attitude towards donating money for charity organizations (AttBeh); DA = development aid; CH = charity; ** p ≤ 0.01; * p ≤ 0.05; + p ≤ 0.10; without marker: n.s. with p > 0.10

Overall fit: \chi^2 = 104.836; df = 67; p = 0.002; CFI = 0.975; RMSEA = 0.045 (CI0.90: 0.027 to 0.061); SRMR = 0.034

Again, the goodness of fit values support the estimated model. In correspondence with the dual-process model and H2.1, acquiescence does in fact have only a significant effect under the two proposed conditions: spontaneous mode and low chronic attitude accessibility (Table 4). In the other three groups, the effect of acquiescence is weaker and non-significant (\chi^2-difference: 5.58; df=1; p=0.018).

According to the dual-process model and hypothesis H2.2, the effect of the attitude towards the target is – as expected – significantly strongest in the group of spontaneous mode and high accessibility (\chi^2-difference: 4.795; df=1; p=0.029).

In sum, the empirical model of basis of specific evaluative judgments in surveys shows that all hypotheses are supported. Thus, acquiescence and attitude effects are stronger when respondents answer fast respectively in automatic-spontaneous mode – and beliefs are more predictive in case of deliberative response mode. And looking deeper into the mechanism that is responsible for these effects within spontaneous mode, SEM analysis showed that (1) acquiescence occurs only under conditions of automatic-spontaneous processing without having a high chronic accessible attitude towards targets, and (2) that attitude effects are significant strongest in case of spontaneous processing with a high attitude accessibility.[25]

Conclusions

In this paper, it was shown how response latencies can be measured in computer assisted surveys and which data treatment steps are necessary. An important application of response latencies is their use as proxy measure of response mode, i.e. the mode of information processing which is active during answering to survey questions. To adopt this response latency approach, a dual-process model of response behaviour was proposed to explain the predictive power of attitudes and the occurrence and strength of response effects (here: acquiescence bias).

Empirical analyses using CATI data to explain response behaviour in surveys showed evidence in favor of the hypotheses assuming stronger generalized attitudinal effects and the occurrence of acquiescence bias in case of automatic-spontaneous response mode (i.e. fast latencies) in contrast to deliberative-controlled responses (i.e. slow latencies). Additionally, it was shown that responders indeed use to a broader information basis (i.e. use of more beliefs) when answering in a deliberative-controlled way (i.e. slow latencies). In an expanded model, it was expected that spontaneous response effects and attitudinal effects are moderated by chronic accessibility of attitudes within the automatic-spontaneous response mode. Empirically, indeed chronic attitude accessibility did moderate effects of acquiescence bias and attitudes in spontaneous mode only. Thus, as assumed by the dual-process model, attitude accessibility is irrelevant for the impact of attitudes and response effects on response behaviour in the case of deliberative processes. On the other hand, as theoretically expected, spontaneous response effects (here: acquiescence bias) are strongest in the case of low attitude accessibility, and attitudes are most predictive towards information processing in case of spontaneous mode with high attitude accessibility.

As theoretically argued, certain response effects are more likely to appear if response latencies are short (e.g. acquiescence, assimilation effects of question order) and other response effects are more likely in the case of long response latencies (e.g. contrast effects of question order). This casts doubt on the interpretation of slow response latency as an indicator of non-attitudes (Fazio, 1990a, 1990b). Instead, response effects have to be differentiated theoretically in terms whether they are expected to need more or less mental effort (see Mayerl & Urban, 2008 for an empirical application of response latencies on question order effects).

The presented analysis points out the usefulness of measuring response latencies in CATI surveys to apply dual-process models for a better understanding and explanation of response effects and predictive power of attitudes. Response latencies can be easily applied and should become a standard in all computer assisted surveys.


[1] A lot of studies use response latencies as a measure of chronic accessibility of social judgments like attitudes (e.g. Fazio, 1986), intentions (e.g. Bassili, 1995), beliefs (e.g. Ajzen, Nichols, & Driver, 1995), or social values (e.g. Dehue, McClintock, & Liebrand, 1993). In the case of all these applications, the central assumption is that the stronger the mental association between an object and its evaluation, the faster will this evaluation be processed and articulated. Important predictors of chronic accessibility are the frequency of elicitation and direct experience with the object (Fazio, 1986, 1990a; Fazio et al., 1982). The interpretation of response latency as an indicator of cognitive accessibility is consistent with the use as an indicator of elaboration, since high accessible judgments are mainly dominant in spontaneous mode, see Theoretical background.

[2] In case of non-linearity, the residuals can be computed by splitting the regression estimation in several subgroup analyses, e.g. by tertile split.

[3] All correlations between raw latencies and baseline speed measures are highly significant (p < 0.05) with Pearson’s r about r = 0.25 (little variation due to different target latencies; see Mayerl et al., 2005 and Mayerl & Urban, 2008). In contrary, the correlation of Residual Index and baseline speed is always non-significant (p > 0.1).

[4] The use of simple factual items to gain a basal physical-motoric baseline speed avoids that other important characteristics of respondents are controlled for too (e.g. level of education). This would be the case when applying a measure of “cognitive demanding information processing speed” (Mayerl et al., 2005, p. 3) as baseline speed. See Mayerl et al., 2005 and Mayerl & Urban, 2008 for a comparison of different baseline speed measures.

[5] In the presented dual-process model, motivation and opportunity are multidimensional constructs, see Mayerl 2009 for a detailed discussion.

[6] This is the case because of the fact that deliberative processes base on the calculation of raw data in a bottom-up way instead of the top-down processing in the spontaneous mode.
Thus, low chronic accessible attitudes (or other overall-evaluative mental constructs) can be retrieved or generated in the deliberative mode and it is irrelevant for the result of deliberative processing whether the attitude was high or low chronically accessible at the beginning of the cognitive process.

[7] Chronic accessibility of mental objects (e.g. attitudes) can be defined as the „[…] readiness to be used in information processing […]“ (Eagly & Chaiken, 1993, p. 131) and „[…] the ease with which information is retrieved.“ (Shrum & O’Guinn, 1993, p. 440). Thus, the higher accessible an attitude, the higher the probability of its automatic activation: “According to the model, the likelihood of activation of the attitude upon mere observation of the attitude object depends on the chronic accessibility of the attitude.” (Fazio, 1990a, p. 81).

[8] The population was defined as follows: German speaking resident population with telephone connection and an age of 18 years and older.

[9] According to the AAPOR Standard Definitions, the response rate is RR1=0.11. Using an alternative definition of net response rate as the proportion of completed interviews and successfully contacted households (excluding neutral non-response), this net response rate is 28.4%.

[10] This assumption holds true for a lot of expressive cases of negation, especially in a logical sense, but clearly sometimes the subjective meaning of a negated expression is not perceived as the exact opposite. In this application, we assume that respondents perceive “I do not like XYZ” as the mirror of “I like XYZ”, basing on the fact that all respondents share experiences in everyday life to “like” or “not like” a lot of things and events. So we think that respondents are both experienced with expressing the negative and the positive forms of judgment we test in our split ballot experiment and that people treat these as clear opposites.

[11] A possible effect of different questionnaire versions was controlled in two ways: firstly, all used variables in this model were not varied concerning their question order or wording. And secondly, all analyses were rerun with dummy control variables of questionnaire version. As a result, the questionnaire variations don’t have any influence on the reported results.

[12] The exact wording of the two items (own translation) (1: agree…5: disagree): “I think it is very positive if I donate money to charity organizations“; “I think it is very positive to donate money to organizations undertaking development aid”.

[13] See Ajzen & Fishbein (1980, p. 79ff), Eagly & Chaiken (1993, p. 163f), Fazio (1990a) and Mayerl (2009) for a detailed discussion of the difference between attitudes towards targets (sometimes misleading called attitudes towards objects) and more specific attitudes towards behavior.

[14] The exact wording of the two items (own translation) (1: very good…5: very bad): “How do you like charity organizations?”; “How do you like organizations undertaking development aid?“

[15] Due to a better interpretation, the scale of the beliefs was divided by 10. Substantially, the four beliefs are: “help people”, “clear conscience”, “money usage” and “religious belief”. The evaluative dimension of modelling beliefs according to Theory of Reasoned Action (based on expectancy-value theory) was measured by (own translation; 1: very important…5: very unimportant; scaling was reversed before multiplying with expectancies): “How important is it to you to help other people?”; “How important is it to you to always have a clear conscience?”; “How important is it to you to that you can be absolutely sure that the money will be used in a way it was intended to?”; “How important are religious beliefs to you?”; The according expectancies were measured with following items (percentage scale 0-100%): “What do you think, how probable is it that donating money helps other people?”; “How probable is it that donating money gives you a better clear conscience?”; “How probable is it that donated money will be used in a way it was intended to?”; “How probable is it that donating money corresponds to your religious beliefs?”

[16] In literature numerous ways to measure tendency to acquiescence as a response style are proposed (e.g. Billiet & McClendon, 2000; Kieruj & Moors, 2010; Narayan & Krosnick, 1996; Smith, 2004; Van Herk, Poortinga, & Verhallen, 2004; Watson, 1992). In SEM framework, one possibility is to model response style as a latent method factor in addition to substantive latent constructs as it is specified in SEM multitrait-multimethod models (e.g. Brown, 2006; Eid, 2000). Billiet & McClendon (2000, p. 623f) validate their latent acquiescence construct with a different and more classical way of operationalization of acquiescence by computing an acquiescence score variable. Both ways of measuring acquiescence highly correlate with r=0.9, indicating high correspondence. In our case, due to lack of enough balanced sets of items, we compute acquiescence score by adding amount of “completely agree” (rating “1”) to exact 100 items with 5-point rating scales during the survey on multiple contents. These items are about attitudes and perceived norms on health-conscious nutrition, environmental concern, religion, attitude towards surveys, and different psychological scales. Such a measure of acquiescence without balanced items has already been tested and shown to be a valid measure of acquiescence (Watson, 1992). The assumption is that the score measures “[…] a tendency to strongly agree with survey statements irrespective of content.” (Watson, 1992, p. 66). Thus, using a wide range of items with completely different contents, it allows us to compute a content-independent index score of tendency to acquiescence. To test whether our acquiescence score index indeed is independent of item content, correlations of acquiescence with behavioral intentions (percentage scale) to donate money (5 items) and to eat in a health-conscious way (6 items) were estimated. Results show that acquiescence score and behavioral intentions correlate independent of content (donation intentions: r between 0.09 and 0.13; health-conscious nutrition: r between 0.05 and 0.16; all correlations p < 0.05).

[17] Wording of the item of altruism (own translation) (1: agree…5: disagree): “Everybody should spend some time on the welfare of his city or community”.

[18] To test the external validity of this measure of direct experience and thus of chronic accessibility, the correlation of membership and yearly frequency of donating money was checked. This correlation was highly significant (p ≤ 0.01) in expected direction on a moderate level (r = 0.25 (N = 1081)).

[19] The measurement error variance of the one-indicator latent construct control variable “altruism” is fixed to “(1-Reliability)*Variance(Item)” (see Bollen, 1989, p. 168) with assumed reliability of 0.8. The results are stable when re-estimating the model with different reliability assumptions (0.7; 1.0). In addition, all results remain the same when re-estimating the complete model with acquiescence as latent one-indicator construct with different reliability assumptions.

[20] The criterion variables were “frequency of going to church” and “sum of donated money in the last 12 months (Euro, taken the logarithm)”. All correlations were highly significant (p ≤ 0.01) on a moderate level with r = 0.14 to r = 0.39.

[21] Except the error variances of the construct “attitude towards behavior” which were significantly different between the response mode groups, thus leading to partial error invariance. Reanalyzing all models without this partial error invariance showed that all reported results are stable and therefore independent of this invariance assumption.

[22] The error correlations are necessary due to the substantive nearness of both attitude constructs. But the correlations are not high (r = 0.17 and r = 0.32; all p ≤ 0.01), so that discriminant validity of both constructs is still given. Additionally, error covariances are constrained to be equal across groups so that differences between groups are not affected by possibly different error covariances.

[23] Additionally, the model was re-estimated with variance and mean robust ML (MLMV) and categorical WLSMV estimation. The results of the tests of the hypotheses are stable and thus independent of distribution of indicators and applied estimators (see Mayerl, 2009, p. 285ff for more details).

[24] All \chi^2-difference tests to compare effects between groups are applied by testing a model with equality constraints (equal effects between groups) against a model with freeing the relevant parameter.

[25] Interestingly, effects of beliefs seem to be moderated by attitude accessibility within deliberative mode in four-group model (Table 4) (we had no hypothesis on moderation of effects of beliefs within deliberative mode). According to this result, beliefs are more predictive when attitude accessibility is low. But more important and in line with dual-process model of response behavior, attitude accessibility does not moderate attitude effects in deliberative mode. Future research is needed on moderation of belief effects within deliberative mode processing and a possible role of attitude accessibility.

References

1. Ajzen, I., & Fishbein, M. (1980). Understanding Attitudes and Predicting Social Behavior. London et al.: Prentice-Hall.

2. Ajzen, I., Nichols, A. J., & Driver, B. C. (1995). Identifying Salient Beliefs About Leisure Activities: Frequency of Elicitation Versus Response Latency. Journal of Applied Social Psychology, 25(16), 1391-7710.

3. Bassili, J. N. (1993). Response latency versus certainty as indexes of the strength of voting intentions in a CATI survey. Public Opinion Quarterly, 57, 54-61.

4. Bassili, J. N. (1995). Response Latency and the Accessibility of Voting Intentions: What Contributes to Accessibility and How it Affects Vote Choice. Personality and Social Psychology Bulletin, 21, 686-695.

5. Bassili, J. N. (1996a). Meta-judgmental versus operative indexes of psychological attributes: The case of measures of attitude strength. Journal of Personality and Social Psychology, 71, 637-653.

6. Bassili, J. N. (1996b). The How and Why of Response Latency Measurement in Telephone Surveys. In N. Schwarz & S. Sudman (Eds.), Answering Questions. Methodology for Determining Cognitive and Communicative Process in Survey Research (pp. 319-346). San Francisco: Jossey-Bass.

7. Bassili, J. N. (2000). Editor’s Introduction: Reflections on Response Latency Measurement in Telephone Surveys. Political Psychology, 21(1), 1-6.

8. Bassili, J. N., Fletcher, J. F. (1991). Response-time measurement in survey research – A method for CATI and a new look at nonattitudes. Public Opinion Quarterly, 55, 331-346.

9. Bassili, J. N., & Krosnick, J. A. (2000). Do Strength-Related Attitude Properties Determine Susceptibility to Response Effects? New Evidence From Response Latency, Attitude Extremity, and Aggregate Indices. Political Psychology, 21(1), 107-132.

10. Baxter, B. W., & Hinson, R.E. (2001). Is smoking automatic? Demands of smoking behavior on attentional resources. Journal of Abnormal Psychology, 110(1), 59-66.

11. Billiet, J. B., & McClendon, M. J. (2000). Modeling Acquiescence in Measurement Models for Two Balanced Sets of Items. Structural Equation Modeling: A Multidisciplinary Journal, 7(4), 608-628.

12. Bollen, K. A. (1989). Structural Equations with Latent Variables. New York et al.: Wiley.

13. Brömer, P. (1999). Informationsverarbeitung bei ambivalenten Einstellungen. Regensburg: Roderer.

14. Brown, T. A. (2006). Confirmatory Factor Analysis for Applied Research. New York/London: Guilford.

15. Carlston, D. E., & Skowronski, J. J. (1986). Trait Memory and Behavior Memory: The Effects of Alternative Pathways on Impression Judgment Response Times. Journal of Personality and Social Psychology, 50(1), 5-13.

16. Chaiken, S. (1980). Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology, 39, 752-766.

17. Chaiken S., Trope Y. (1999). Dual process theories in social psychology. New York, London: Guilford Press.

18. Dehue, F. M. J., McClintock, C. G., & Liebrand, W. B. G. (1993). Social value related response latencies – unobtrusive evidence for individual differences in information processing. European Journal of Social Psychology, 23(3), 273-293.

19. Eagly, A. H., & Chaiken, S. (1993). The Psychology of Attitudes. Fort Worth: Harcourt Brace Jovanovich.

20. Eid, M. (2000). A multitrait-multimethod model with minimal assumptions. Psychometrica, 65(2), 241-261.

21. Faust, M. E., Balota, D. A., Spieler, D. H., & Ferraro, F. R. (1999). Individual Differences in Information-Processing Rate and Amount: Implications for Group Differences in Response Latency. Psychological Bulletin, 125(6), 777-799.

22. Fazio R. H. (1986). How do attitudes guide behavior? In R. M. Sorrentino & E. T. Higgins (Eds.), The handbook of motivation and cognition: Foundation of social behavior (pp. 204-243). New York: Guilford Press.

23. Fazio R. H. (1990a). Multiple Processes by which Attitudes guide Behavior: the MODE Model as an integrative framework. Advances in Experimental Social Psychology, 23, 75-109.

24. Fazio R. H. (1990b). A practical guide to the use of response latency in social psychological research. In C. Hendrick & M. S. Clark (Eds.), Review of Personality and Social Psychology (pp. 74-97). Vol. 11. Research Methods in Personality and Social Psychology. Newbury Park, CA: Sage Publications.

25. Fazio, R. H. (1995). Attitudes as object-evaluation associations: Determinants, consequences, and correlates of attitude accessibility. In R. E. Petty & J. A. Krosnick (Eds.), Attitude strength: Antecedents and consequences (pp. 247-282). Hillsdale, NJ: Erlbaum.

26. Fazio, R. H., Chen, J.-M., McDonel, E. C., & Sherman, S. J. (1982). Attitude Accessibility, Attitude-Behavior Consistency, and the Strength of the Object-Evaluation Association. Journal of Experimental Social Psychology, 18, 339-357.

27. Fazio, R. H., & Roskos-Ewoldsen, D. R. (2005). Acting as we feel: When and how attitudes guide behavior. In T. C. Brock & M. C. Green (Eds.), Persuasion: Psychological insights and perspectives (pp. 41-62) (2. Ed.). Thousand Oaks, CA: Sage.

28. Gibbons, H., & Rammsayer, T. (1999). Auswirkungen der Vertrautheit mit einer Reizdimension auf Entscheidungs¬prozesse: Der modulierende Einfluss kontrollierter vs. automatischer Informationsverarbeitung. In I. Wachsmuth & B. Jung (Eds.), KogWis99 (pp. 159-164). Proceedings der 4. Fachtagung der Gesellschaft für Kognitionswissenschaft, Bielefeld St.Augustin.

29. Hertel, G., Neuhof, J., Theuer, T., & Kerr, N. L. (2000). Mood effects on cooperation in small groups: Does positive mood simply lead to more cooperation? Cognition & emotion, 14(4), 441-472.

30. Houlihan, M., Campbell, K., & Stelmack, R. M. (1994). Reaction Time and Movement Time as Measures of Stimulus Evaluation and Response Processes. Intelligence, 18, 289-307.

31. Keller, J., Bohner, G., & Erb, H.-P. (2000). Intuitive und heuristische Urteilsbildung – verschiedene Prozesse? Präsentation einer deutschen Fassung des “Rational-Experiential Inventory” sowie neuer Selbstberichtskalen zur Heuristiknutzung. Zeitschrift für Sozialpsychologie, 31(2), 87-101.

32. Kieruj, N. D., & Moors, G. (2010). Variations in response style behavior by response scale format in attitude research. International Journal of Public Opinion Research, 22(3), 320-342.

33. Klauer, K. C., & Musch, J. (1999). Eine Normierung unterschiedlicher Aspekte der evaluativen Bewertung von 92 Substantiven. Zeitschrift für Sozialpsychologie, 30, 1-11.

34. Knowles, E. S., & Condon C. A. (1999). Why people say “yes”: A dual-process theory of acquiescence. Journal of Personality and Social Psychology, 77(2), 379-386.

35. Krosnick, J. A. (1991). Response Strategies for Coping with the Cognitive Demands Of Attitude Measures in Surveys. Applied Cognitive Psychology, 5, 213–236.

36. Mayerl, J. (2009). Kognitive Grundlagen sozialen Verhaltens. Framing, Einstellungen und Rationalität. Wiesbaden: VS Verlag.

37. Mayerl J., Sellke P., Urban D. (2005). Analyzing cognitive processes in CATI-Surveys with response latencies: An empirical evaluation of the consequences of using different baseline speed measures (SISS Schriftenreihe des Instituts für Sozialwissenschaften der Universität Stuttgart No. 2/2005). Stuttgart: University of Stuttgart.

38. Mayerl, J., & Urban, D. (2008). Antwortreaktionszeiten in Survey-Analysen: Messung, Auswertung und Anwendungen. Wiesbaden: VS Verlag.

39. Mulligan, K., Grant, J. T., Mockabee, S. T., & Monson, J. Q. (2003). Response latency methodology for survey research: Measurement and modeling strategies. Political Analysis, 11(3), 289-301.

40. Narayan, S., & Krosnick, J. A. (1996). Education moderates some response effects in attitude measurement. Public Opinion Quarterly, 60, 58-88.

41. Pachella, R. G. (1974). The interpretation of reaction time in information-processing research. In B. H. Kantowitz (Ed.), Human information processing: Tutorials in performance and cognition (pp. 41-82). Hillsdale: Erlbaum.

42. Petty, R. E., & Cacioppo, J. T. (1986). The Elaboration Likelihood Model of Persuasion. Advances in Experimental Social Psychology, 19, 124-205.

43. Ruder, M. (2001). Die Nutzung der Verfügbarkeitsheuristik – eine Frage der Stimmung?. Hamburg: Kovac.

44. Sheppard, L. C., & Teasdale, J. D. (2000). Dysfunctional thinking in major depressive disorder: A deficit in metacognitive monitoring? Journal of Abnormal Psychology, 109(4), 768-776.

45. Shrum, L. J., & O’Guinn, T. C. (1993). Processes and Effects in the Construction of Social Reality. Construct Accessibility as an Explanatory Variable. Communication Research, 20(3), 436-471.

46. Smith, E. R., & DeCoster, J. (2000). Dual-process models in social and cognitive psychology: Conceptual integration and links to underlying memory systems. Personality and Social Psychology Review, 4(2), 108-131.

47. Smith, P. B. (2004). Acquiescent response bias as an aspect of cultural communication style. Journal of Cross-Cultural Psychology, 35(1), 50-61.

48. Strack, F., & Deutsch, R. (2004). Reflective and Impulsive Determinants of Social Behavior. Personality and Social Psychology Review, 8(3), 220-247.

49. Strack, F., & Martin, L. L. (1987). Thinking, Judging, and Communicating: A Process Account of Context Effects in Attitude Surveys. In H.-J. Hippler, N. Schwarz & S. Sudman (Eds.), Context Effects in Social and Psychological Research (pp. 123-148). New York et al.: Springer.

50. Sudman, S., Bradburn, N. M., & Schwarz, N. (1996). Thinking about answers. The application of cognitive processes to survey methodology. San Francisco: Jossey-Bass.

51. Tourangeau, R. (1992). Context Effects on Responses to Attitude Questions: Attitudes as Memory Structures. In H.-J. Hippler, N. Schwarz & S. Sudman (Eds.), Context Effects in Social and Psychological Research (pp. 35-47). New York et al.: Springer.

52. Tourangeau, R., & Rasinski, K. A. (1988). Cognitive Processes Underlying Context Effects in Attitude Measurement. Psychological Bulletin, 103(3), 299-314.

53. Urban, D., & Mayerl, J. (2007). Antwortlatenzzeiten in der survey-basierten Verhaltensforschung. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 59(4), 692-713.

54. Van Herk, H., Poortinga, Y. H., & Verhallen, T. M. M. (2004). Response Styles in Rating Scales: Evidence of Method Bias in Data from 6 EU Countries. Journal of Cross-Cultural Psychology, 35(3), 346-360.

55. Watson, D. (1992). Correcting for Acquiescent Response Bias in the Absence of a Balanced Scale: An Application to Class Consciousness. Sociological Methods & Research, 21, 52-88.



Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 4.0 International License. Creative Commons License