{"id":7794,"date":"2016-07-01T09:30:14","date_gmt":"2016-07-01T08:30:14","guid":{"rendered":"http:\/\/surveyinsights.org\/?p=7794"},"modified":"2016-07-25T16:19:44","modified_gmt":"2016-07-25T15:19:44","slug":"a-case-study-of-error-in-survey-reports-of-move-month","status":"publish","type":"post","link":"https:\/\/surveyinsights.org\/?p=7794","title":{"rendered":"A Case Study of Error in Survey Reports of Move Month Using the U.S. Postal Service Change of Address Records"},"content":{"rendered":"
\n

1. \u00a0 \u00a0 \u00a0Introduction<\/strong><\/h2>\n<\/div>\n

In the 2010 Census, all persons living in the U.S. were counted at the place they were living as of Census Day, April 1, 2010. The data collection for that census took place over the course of a few months: February to August, with some evaluation operations occurring up to 11 months after Census Day. Respondents reported information for their households by mail or by speaking with an interviewer over the telephone or in person. For data collections occurring after Census Day, respondents often had to rely on their memory to determine where they were living on April 1. In part, the success of the census depended upon respondents recalling moves (for both themselves and others in their household) and move dates accurately because 11.6% of the population changed residences between March 2010 and February 2011 (U.S. Census Bureau, 2011). For the 2010 Census, the U.S. Census Bureau made the assumption that respondents could accurately remember moves and move dates on and around April 1 up to 11 months after April 1.<\/p>\n

1.1 Memory for Moves<\/strong><\/h3>\n

Memory of events is influenced by the recency of the event, the importance of the event to the respondent, whether the respondent has had to recall the event before, in addition to other biological factors such as the clarity of memory or the age of the respondent (Bradburn, Huttenlocher, and Hedges, 1994, Tourangeau, Rips, and Rasinski, 2000). These factors can influence a respondent\u2019s tendency to forget the event or telescope the event.<\/p>\n

Telescoping is a type of recall error where events are reported as happening either more recently than they actually happened (forward telescoping) or farther back than they actually occurred (backwards telescoping). Studies aimed at determining whether the net effect of recall errors is zero or tends toward backward or forward telescoping have frequently concluded that although there can be backwards telescoping, the net effect is most often forward telescoping of events (Neter and Waksberg 1964, Rubin and Baddeley 1989, Huttenlocher, Hedges, and Bradburn 1990, Janssen, Chessa, and Murre 2006). This tendency is due to a variety of factors including weak bounding criteria and the inability to backward telescope future events. Many of these studies asked for recall periods of 2 months, or 4 months or slightly longer. Janssen et al.<\/em> (2006) found that dates of events in the news that occurred 1000 days or more prior to the study were more likely to be telescoped forward. However, events that occurred between 100 to 1000 days prior to the study were more likely to be backwards telescoped and no telescoping effect occurred for reporting dates of recent events occurring less than 100 days ago.<\/p>\n

The census and its evaluative operations need to accurately identify who lived at each address on April 1st<\/sup>. If telescoping the move month or forgetting to recall a move increases as the interview date gets farther away from the move, then data collected months after April 1st<\/sup> may be problematic. The study of recall error associated with move dates is limited. Aurait (1993) examined the accuracy of reporting residential mobility using a study of 500 married couples aged 41 to 57 years old in Belgium. Comparing their residential mobility reports to Belgium\u2019s National Population Register data, Aurait found dating errors for moves occurred in years more so than months. She also found that the birth of a child within a year or two of the move decreased dating errors for moves. Aurait found no net effect of telescoping of the move events in her analysis. Other survey methodologists have made similar assumptions about the quality of respondents\u2019 recall of move dates. Move dates and places of residence often serve as anchors to aid memory of other events, particularly as part of the survey research technique that creates an event history or timeline (Belli, 1998; van der Vaart, 2004).<\/p>\n

1.2 Data Quality in Proxy Reports <\/strong><\/h3>\n

There is evidence in the literature that reports may also differ by the type of respondent \u2013 whether a self-report or proxy report. Miller, Massagli,\u00a0and Clarridge (1986) examined quality of proxy versus self reports to twenty health questions asking specifically about health complaints, finding that proxy reports typically underestimated the health complaints even in serious situations. In studies of proxy data following the Census 2000, item non-response was lower when responses were provided by household members versus proxies such as neighbors, postal workers or landlords (Chesnut, 2005; Wolfgang, Byrne & Spratt, 2003). In a study looking specifically at coverage issues, Martin (1999) examined proxy-reporting errors for the concept of “usual residence.\u201d She found that proxy reports of “usual residence” increased undercoverage for young black males in particular. Surrounding the 2010 Census, King, Cook and Childs (2012) examined self versus proxy reporting of living situation information, finding that self-report respondents provided more complete information than proxy respondents did. Though they reacted in different ways (self-reports refused questions and sometimes failed to provide requested information and proxies commented on sensitivity of questions), both showed evidence of privacy sensitivities that often revealed itself in item non-response.<\/p>\n

\u00a01.3 Current Study<\/h3>\n

The present study investigates recall error regarding move month and reports of a move with thoughts to planning for the timing of census and evaluative operations for the 2020 Census.<\/p>\n

The study compares independent survey responses of moves and move dates with the U.S. Postal Service National Change of Address (NCOA) files of requests to forward mail in March and April of 2010 (U.S. Postal Service 2015). The approach uses binomial and multinomial analyses to investigate whether recall error increases as the length of time from the move increases while controlling for the type of respondent. We investigated three research questions:<\/p>\n

    \n
  1. whether reporting a move is affected as the time lapse between the move and the interview increases;<\/li>\n
  2. whether the accuracy of the reported move month and the direction of the error is affected as the time lapse between the move and the interview increases; and<\/li>\n
  3. whether the accuracy of the reported residence where someone should be counted for the Census is affected as the time lapse between the move and the interview increases.<\/li>\n<\/ol>\n

    <\/h2>\n

    2. Research methodology<\/strong><\/h2>\n

    To study recall errors in reporting moves, we started with a sample of 13,500 households selected from a frame of movers. We then conducted an independent U.S. Census Bureau telephone survey with these movers over different time periods to determine whether memory of the move and the move date decayed over time.<\/p>\n

    The original frame of movers consisted of records from an extract of the National Change of Address (NCOA) file dated May 1, 2010. This file only contained records that had reported a change of address in either March or April of 2010 submitted by May 1, 2010. While it is possible that for some of these records, the person submitted a \u201cchange of address\u201d filing simply to forward mail to another address for a reason other than a physical move, for our purposes they were all classified as moves. Those who moved in March or April of 2010 but completed the United States Postal Service form after May 1 are not part of this analysis.<\/p>\n

    A systematic sample of sorted NCOA records was matched to a commercial database in May 2010 to obtain a telephone number for the address where the mail was forwarded. Then, addresses for which telephone numbers were found were randomly divided into three equal subsamples. Each subsample was contacted by telephone at a different interval after the move to determine whether they would report the move and if they reported the move whether there were recall errors in the reported move month. Appendix 1 contains more details of the sampling.<\/p>\n

    The Census Bureau survey was called the Recall Bias Study (RBS) (Linse, Pape, Rosenberger, and Contreras 2012). The interview timing for the three subsamples approximated the timing of 2010 Census and evaluative operations: June 2010, September 2010, and February 2011. Using AAPOR Response Rate 2 (American Association of Public Opinion Research 2011) that includes sample units of unknown eligibility in the denominator, response rates ranged from 63 to 69 percent.<\/p>\n

    The RBS instrument collected a roster of people currently living in the contacted housing units, other addresses where a person could have been counted around Census Day, April 1, 2010, and move dates associated with those other addresses, including month, day and year. The respondent was a member of the current household who was 18 years or older. This may or may not have been the person in the household whose name was on the NCOA form. Thus, respondents could self-report a move, or could report a move of another household member.<\/p>\n

    Our research strategy relies on assuming that the month to begin forwarding mail on the NCOA form is the \u2018true\u2019 move month. While this assumption is also a limitation in that there are reasons a person would forward mail to an address prior to, or after, the actual move month, this failure of the underlying assumption should affect the estimation for all levels of variables at the same rate. With this assumption, we are able to study recall bias in the RBS. We study forgetting to report a move by comparing the absence of a move report in the RBS to those moves reported in the NCOA. We study telescoping and the accuracy of move reports by comparing the reported month of the move in RBS to the NCOA month. The difference in the reported month and the NCOA month provides data to look for patterns of forward or backwards telescoping. Since many moves occur on the first of a month, we built in a tolerance by considering a RBS response of one month and the NCOA record having the last day of the previous month or the first day of the next month as agreeing. We use binomial and multinomial analyses to study the effects of the length of time between the \u201ctrue\u201d move and the survey interview, the respondent, and other characteristics of the move. These characteristics include whether the move was for a family or an individual (collected on the NCOA form) and, in a multi-person household, whether the report was a self or proxy response. These variables are of interest because they could yield practical information interviewers (or survey designers) could use to identify which respondent to interview to maximize the accuracy of move data. Appendix 2 contains detailed methodology.<\/p>\n

    Although the initial sample size was 13,500 households, we had to restrict our analyses to the datasets of size 3,424 households who reported the address and name that was on the NCOA form and 1,740 households who reported a move to the NCOA address and a date of that move. Appendix 1 discusses in detail the reasons for the loss of cases for the analysis.<\/p>\n

     <\/p>\n

    3. Results <\/strong><\/h2>\n

    For our analysis, we explore the differences in the observed probabilities of accurately reporting a move or move month between the levels of the variables Interview Month <\/em>and Respondent Type<\/em> to examine the net effect on recall error. The variable I<\/em>nterview month<\/em> reflects length of time since the move and could be June, August, or February, corresponding to length of time from Census Day \u2013 roughly, 2, 5 and 10 months from Census Day. Respondent Type <\/em>is <\/em>a three-level variable that combines the respondent type and the move type:<\/p>\n