{"id":10663,"date":"2019-02-22T09:00:08","date_gmt":"2019-02-22T08:00:08","guid":{"rendered":"https:\/\/surveyinsights.org\/?p=10663"},"modified":"2019-02-22T08:52:42","modified_gmt":"2019-02-22T07:52:42","slug":"trust-and-credibility-in-the-u-s-federal-statistical-system","status":"publish","type":"post","link":"https:\/\/surveyinsights.org\/?p=10663","title":{"rendered":"Trust and Credibility in the U.S. Federal Statistical System"},"content":{"rendered":"

Trust and Credibility in the Federal Statistical System<\/strong><\/h1>\n

Public trust in federal statistics has the potential to affect response rates, operational costs, and data quality. Federal statistics are vital to the federal government, policymakers, and non-profit organizations that rely on these data. These statistics help members of the government, as well as those external to the government, understand the population\u2019s composition, needs, desires and concerns so that they may be able to respond appropriately. In the midst of historically low response rates to government surveys, the United States has also seen declining levels of civic engagement across a wide range of activities, including voter turnout (Jost 2010). The U.S. Federal Statistical System (FSS) is searching for ways to ensure high quality data in ongoing surveys, given declines in response rates and the associated increase in operational costs. If there are problems with public trust, we want to identify and address them because these issues may hinder response rates and associated research efforts.<\/p>\n

This line of research is intended to increase understanding of public trust in Federal statistics, which will provide guidance for communicating with the public and for future planning of data collection activities. In this research, we set out to understand the predictors of trust to inform activities that will improve overall public trust of the Federal Statistical System and its statistical products.<\/p>\n

Measuring Trust<\/h2>\n

The issue of trust in government statistics is not unique to the U.S. Across the globe, national statistical offices are recognizing the role of public trust in ensuring high quality data, particularly in an era of constrained resources (Kim 2010; Marozzi, 2014; Banks 2014). Trust in government is an important construct to study and understand in order to ensure that buy-in from the public is present (Paliszkiewicz, 2012; Chanley, Ruduolph, & Rahn, 2000; Kampen,Van de Walle, & Bouckaert, 2006; Banks, 2014, Marrozi, 2014).<\/p>\n

Chanley et al. (2000) mentioned that establishing public trust in government is essential in helping political leaders make appropriate decisions, ensure that the goals are aligned to the society\u2019s well-being, and gain the public\u2019s compliance. The researcher further discussed that without the public\u2019s trust, the \u2018citizens withdraw support from government and become less willing to comply with governmental decisions\u2019 (Chanley, 2000, p. 240). Kampen, Van de Walle, and Bouckaert (2006) explained that trust in the government is an important factor that helps governments act without coercion and without relying on force to make decisions.<\/p>\n

Throughout the literature, multiple definitions exist to explain the trust construct (Thomas, Abts, Stroeken, & Weyden, 2015; Paliszkiewicz, Koohang, Goluchowski, and Nord, 2014; Offe, 1999; Butler, 1991; Paliszkiewicz, 2012). For example, according to Thomas et al (2015) trust is defined as people\u2019s expectations of what an organization must do based on the objectives and principles established by the cultural norm. Similarly, Offe discussed that in order for trust to be present in organizations, institutions, or government, the entity needs to be perceived by its surrounding community as useful, meaningful, and credible.<\/p>\n

Due to the difficulty and elusive meaning of trust as well as the numerous definitions that exist in literature to define trust, measuring trust in government has become difficult to accomplish. For example, Marozzi (2014) states that one of the difficulties in measuring trust is \u2018whether the concept of trust as well as the concepts of government, state and public institution are similarly interpreted by citizens\u2026\u2019 (p.880). Furthermore, both Marozzi (2014) and Cook and Granoke (2005) agreed that measuring trust can be difficult because distinguishing between active distrust and lack of trust is complicated to assess.<\/p>\n

In their paper, Thomas et al. (2015) articulated that there was no general agreement in measuring trust. However, the researchers believed the best practice to measure trust was by using multiple survey questions to assess various institutions (i.e., police, justice, and political parties), and therefore capture the variation across institutions. Thomas et al. (2015) pointed out that the majority of time this does not occur due to different methodologies, where data is often analyzed using one individual item for each particular institution. Likewise, in an earlier study Butler (1991) discussed that trust should be measured using multiple items because trust is a multidimensional construct that is influenced by various conditions. Through a factor analysis, the researcher found that availability, competence, consistency, fairness, integrity, loyalty, openness, promise fulfillment, and receptivity were closely related to public trust in government.<\/p>\n

Similar to Thomas et al. (2015) and Butler (1991), Kampen et al. (2006) used multiple variables to assess the relationship between satisfaction with service delivery and public trust in government. The researchers in the study used satisfaction items and trust items to measure public trust in the government. More specifically, four questions were asked to the Flemish community in Belgium (n<\/em>=1,250) about five public services including police, primary education, garbage collection, transportation by buses and trams, and the Belgian Post. The following items used to assess satisfaction and trust in the study were asked for each institution: 1) To what extent are you satisfied with the service delivery of [institution]?, 2) To what extent do you trust [institution]?, 3) To what extent are you satisfied with the policy with respect to [domain]?. A final item asked about service delivery in general: To what extent are you satisfied with public service delivery in general? Both sets of questions used a 5-point Likert scale. The trust item scale ranged from 1 = very little <\/em>to 5 = very much, <\/em>while the satisfaction items had responses where 1 = very dissatisfied<\/em> and 5 = very satisfied<\/em>.<\/p>\n

The researchers hypothesized that a significant correlation would be found between satisfaction and trust in government agencies. Additionally, they believed these items would significantly measure public trust in the government. The results of the study supported the researchers\u2019 hypotheses where higher disappointment leads to lower trust in the public agency, whereas higher satisfaction would lead to more trust. Based on the results, Kampen et al. (2006) suggested that when measuring or assessing public trust in the government that a government satisfaction component be included in the survey.<\/p>\n

Kampen et al. (2006) further discussed that more and more governments are realizing the importance trust has on the success and effectiveness of its implemented policies, which has led many governments to focus on programs that increase trust in their institutions. This new phenomenon called \u2018government modernization\u2019 has dominated the political agenda in several countries of the Organization for Economic Cooperation and Development (OECD; Kampen et al., 2006). The primary goal of government modernization has been to increase trust in government. Kampen et al. (2006) mentioned that many countries within the OECD are implementing surveys that assess the level of citizen satisfaction and trust.<\/p>\n

In an attempt to understand how statistical institutions and their products are perceived by respondents, an OECD working group developed a survey for measuring trust in official statistics in 2010 (Brackfield 2011). The OECD working group found that several factors affect public trust in government statistics. They summarized the factors into two constructs: 1) trust in statistical products and 2) trust in statistical institutions, which together form trust in official statistics (see Figure 1; Brackfield 2011, Fellegi 1996, 2004; OECD, 2010). The OECD emphasized and made clear that in order to improve trust in government citizens need to be engaged in the decision making process as well as adopt a policy of transparency in their procedures.<\/p>\n

The questionnaire developed by the OECD was examined for its applicability in the United States, where the statistical system is decentralized, rather than centralized like in many other countries, and it did not work well (Willson, et al. 2010). Cognitive testing of these questions by the National Center for Health Statistics (NCHS) in 2010 revealed that the survey questions did an inadequate job of conveying intent to the respondents. Most respondents did not understand the questions as asking about their knowledge and opinion of official statistics. A common interpretation was that the questions were asking about respondent knowledge and opinion of the substantive topic itself, for example unemployment or crime (Willson, 2010). When asked about their feelings on the reliability of crime and unemployment statistics, respondents tended to interpret the questions in terms of their experience with being unemployed or with being victims of crime. This pattern of question misinterpretation did not vary greatly by educational attainment. Respondents with higher and lower levels of education were both likely to misunderstand the questions.<\/p>\n

Using the OECD questionnaire as a starting point, the Census Bureau partnered with NCHS, the National Agricultural Statistics Service, the U.S. Department of Agriculture\u2019s Economic Research Service, the Statistics of Income Division of the Internal Revenue Service and the Office of Management and Budget to develop a questionnaire specifically to be administered in the United States context (Childs, Willson, Martinez, Rassmussen, and Wroblewski, 2012; Willson, 2012). This group of Federal statistical agencies make up the Federal Statistical System Working Group (FSS Working Group).<\/p>\n

\"\"<\/a><\/p>\n

Figure 1. Fellegi\u2019s Constructs to Measure Trust<\/p>\n

Based on the earlier NCHS cognitive testing, the FSS Working Group created a new set of questions to measure U.S. public opinion and trust in the FSS. Questionnaire development started with the premise that we needed to measure awareness of statistics and statistical institutions and data use first, before proceeding to questions addressing trust. We focused on factors that would be familiar to the general population, who are not likely to be sophisticated data users with a high level of knowledge of the statistical system. This included questions about general knowledge of particular statistics such as the population count, death by particular diseases, and the consumer price index. These questions draw on previous research that examined the U.S. public\u2019s knowledge of statistics (Curtin, 2007) and are designed to be understood by the general population. In developing the rest of the questions, the FSS Working Group also considered questions used by the Office of National Statistics and the National Centre for Social Research in the United Kingdom and by the Eurobarometer, a series of regularly occurring public opinion surveys of European countries. The final product, the FSS Trust Survey, was pretested in 2012. As part of the pretesting, we conducted a factor analysis to help us chose the individual items to include in our survey and to interpret how the items measuring trust in statistics work together and fit with the Fellegi model (Earp, et al. 2012). Details of this pretesting can be found elsewhere (Childs, et.al., 2012; Earp, et al., 2012; and Willson, 2012).<\/p>\n

The FSS Trust survey consisted of 19 core items that are designed to capture 1) respondents\u2019 history of data use, 2) trust in statistical products, and 3) trust in statistical institutions. (Later iterations of the survey included only 5 items, but measured these constructs as well). Trust in statistical products is measured by questions on credibility and relevance, while trust in statistical institutions is measured by questions on confidentiality and transparency. In this paper we examine whether or not someone has ever used federal statistics in addition to demographic variables as possible predictors of trust. Having a history of data use is important because trust between individuals has been shown to increase following frequent contact (Mayer, Davis and Schoorman 1995; McAllister 1995; Ferrin and Dirks 2003; Dietz and Hartog 2006). In this context, data users have formed a relationship with statistics and the statistical institutions that produce these data. Accordingly, we might expect higher levels of trust to be a result of a relationship formed between the data user and the data.<\/p>\n

Methods<\/strong><\/h2>\n

The Gallup Daily Poll is a daily survey asking U.S. adults about various political, economic, and well-being topics. The Census Bureau established a relationship with Gallup whereby the FSS Working Group added questions to the daily poll on trust, credibility, transparency and data use. From February 2012 until September 2013, we added 25 questions to a sample of cases in the ongoing survey. Approximately 19 of the 25 questions were core questions that did not vary throughout the study and approximately six were periodically rotated. Core questions focused on awareness of and attitudes towards federal statistics and federal statistical agencies. Figure 2 below shows the questions that we reference in this paper.<\/p>\n

\"\"<\/a><\/p>\n

Figure 2. Question wording for FSS Trust Survey Items<\/p>\n

Gallup\u2019s Methods<\/em><\/strong><\/h3>\n

The Gallup Daily Tracking Survey consists of computer-assisted telephone interviews with randomly sampled respondents, aged 18 and older, including cellphone users and Spanish-speaking respondents from all 50 U.S. states and the District of Columbia. Gallup conducts 1,000 surveys almost daily (350 out of 365 days per year). For this study, a random selection of 200 respondents nightly were administered the census module.<\/p>\n

Gallup Daily tracking relies on dual-frame sampling (which includes random-digit-dial (RDD) list-assisted landline interviewing and RDD wireless phone sampling to reach those in wireless-only and wireless-mostly households), and a random selection method for choosing respondents within the landline household. To compensate for disproportionalities in selection probabilities and nonresponse, Gallup weights the data daily. Gallup further post-stratifies the data using an iterative proportional fitting (i.e., raking) algorithm to account for nonrandom nonresponse by phone status, age, sex, region, education, population density, ethnicity, and race. Gallup calculates daily weights for each survey track separately and for the combined data. Gallup computes state weights twice per year and congressional district and MSA weights once per year. The American Association for Public Opinion Research\u2019s AAPOR- III response rate averaged 7 percent for the Wellbeing track during the time period reported on in this study.<\/p>\n

Although the Gallup Daily Tracking Survey is nationally representative, it does not meet Census Bureau quality standards for use as official national estimates. The Census Bureau and the Federal Statistical System used the results from this survey to monitor awareness and attitudes, as an indicator of the impact of potential negative events, and as an indicator for potential changes in communication campaigns.<\/p>\n

Results<\/strong><\/h2>\n

Using logistic regression, we modeled trust in statistics based on approximately 23,000 responses. \u00a0In the model, we predicted the general measure of \u2018Trust in Statistical Products\u2019 (the dependent variable) using independent variables indicating whether the respondent had ever used federal statistics for school or work purposes (\u2018Data User\u2019), belief in \u2018Credibility,\u2019 belief in \u2018Transparency,\u2019 and demographics, as well as interaction effects. We attempt to control for the complex design of the survey using the frame data that was available in variance estimates. We created two variance strata to account for whether a respondent completed the interview on a landline phone or on a cell phone. In addition, we randomly assigned each response to one of four variance PSUs, so that we could implement Jack Knife variance estimation.<\/p>\n

Data Use <\/em><\/strong><\/p>\n

Given the low response rate of the survey, results are intended to be used for tactical decision-making and not for estimating the population. The findings are presented in simple visualizations but are based on multivariate models predicting \u2018trust\u2019 and \u2018credibility\u2019 controlling on key covariates. Being a Data User increases the likelihood of \u2018Trust in Statistical Products\u2019 (see Table 1). Respondents who have used statistics for work or study tend to report trusting federal statistics more than those who have not. The relationship between trust and data use suggests that a distinction between interpersonal and institutional trust is useful (Bachmann 2011). Interpersonal trust is based on contact between individuals while institutional trust depends on rules of behavior and sanctions for violating those rules. Data users have had contact with statistics and the trust they have in statistics can be interpreted as similar to interpersonal trust. Respondents who have used the statistics trust them more than those who have not. In preliminary analysis of these data, we found that Data Usage is more influential than merely knowing what agency produces the statistics (data not presented here).<\/p>\n

\"\"<\/a><\/p>\n

Source: FSS Analysis Files, *** p<=0.01, ** p<=0.05, * p<=0.10<\/p>\n

Description of the variables used in the logistic regression models:<\/p>\n