{"id":20208,"date":"2025-02-25T15:56:11","date_gmt":"2025-02-25T14:56:11","guid":{"rendered":"https:\/\/surveyinsights.org\/?p=20208"},"modified":"2025-03-18T15:36:02","modified_gmt":"2025-03-18T14:36:02","slug":"who-chooses-a-qr-code-over-a-url-to-access-a-web-screener-in-a-national-probability-survey-of-older-adults-and-the-impact-on-data-quality","status":"publish","type":"post","link":"https:\/\/surveyinsights.org\/?p=20208","title":{"rendered":"Who chooses a QR code over a URL to access a web screener in a national probability survey of older adults, and the impact on data quality"},"content":{"rendered":"

Introduction<\/b><\/h1>\n

As survey response rates continue to decline and data collection costs continue to rise, researchers are under continuous pressure to look for innovative ways to combat those trends (Williams and Brick, 2018). One strategy researchers often introduce is a self-administered web option for respondents, either as the only mode, or in a mixed-mode design, which may be appealing to some sampled cases. Obtaining completed surveys from willing cases using self-administered web frees up resources for more challenging cases that require more expensive data collection modes that involve an interviewer, such as telephone or in-person. One way to invite households is via a push-to-web design (Dillman, 2017), implemented with a national address-based probability sample. Area probability-based sampling frames do not typically include email addresses; therefore, a mailed letter is generally used to invite a household (or \u201cpush\u201d the household) to participate in a web survey. The mailed invitation letter usually includes a URL and login credentials that need to be entered into a web browser by the respondent to access the survey. Typing in a URL and login credentials, which can be error prone, can be considered burdensome and a barrier for participation for some respondents. One possible way to ease the burden of typing these in is by taking advantage of quick response (QR) codes.<\/p>\n

Quick Response (QR) Codes in Surveys<\/b><\/h2>\n

QR codes have existed for decades. A QR code is a two-dimensional matrix barcode developed in 1994 by Denso Wave, a Toyota subsidiary, to track parts in vehicle manufacturing (Shin et al., 2012). A QR code encodes information, including text, a URL, or other data. QR code usage has increased in recent years, with expected growth in the future (eMarketer, 2022). Reasons for this increase are likely due to QR codes working seamlessly with native smartphone technology (i.e., the camera), and the increase in smartphone penetration, which was up to 91% in the U.S. in 2023 (Pew, 2024). The COVID-19 pandemic also increased QR code use (Davies et al., 2023; Gostin, 2021). Further, their ubiquity has broadened the context in which they are being used (Gostin, 2021). For example, they may be used to obtain information, access restaurant menus or bus schedules, recruit for jobs, enable digital payments, or facilitate survey research.<\/p>\n

Researchers have considered the use of QR codes in the survey research context (AAPOR, 2014; Dillman, Smyth, and Christian, 2014; Faggiano and Carugo, 2020), and an increasing number of studies are reporting their actual use and subsequent impact on data collection. QR codes have been introduced in several studies and the overall response to their use has been mixed. Response rates for studies that introduce QR codes as a possible method for accessing a web survey have been in the low single digits for some studies, both pre-pandemic (Gluck, 2012; Allen, Marlar, and Steele, 2016; Lugtig and Luiten, 2021) and post-pandemic (Patrick et al., 2022; Endres et al., 2023). However, several studies that occurred after the pandemic report a higher response rate from their use (Leary et al., 2023, Olson and Ganshert, 2024). Even when the impact of QR codes on increasing response is low, the inclusion of a QR code still has a modest, but positive effect on the response rate (Endres et al., 2023, Marlar and Schreiner, 2024), and increases the proportion of web respondents who complete via a smartphone (Endres et al., 2023; Lugtig and Luiten, 2021; Olson and Ganshert, 2024).<\/p>\n

The aforementioned literature suggests that including a QR code along with a URL may have benefits, but some studies report that having a QR code as the lone access method could be detrimental. Several studies show that when both a URL and QR code are included in an invitation, there are significantly more web completes (Marlar and Schreiner, 2024). Further, more respondents use a smartphone to complete the survey, with no observed negative effect on login or break-off rates (Lugtig and Luiten, 2021). One experimental study, conducted before the pandemic, found that when a QR code is the lone access method it led to significantly lower completion in a web survey (Allen, Marlar, and Steele, 2016). However, we note that QR code usage changed after the pandemic and this finding may be less relevant post-pandemic.<\/p>\n

Some studies have reported heterogeneity of the impact of QR codes among demographic subgroups. Younger adults tend to prefer using QR codes (Allen, Marlar, and Steele, 2016; Endres et al., 2023; Leary et al., 2023; Lee et al., 2023; McDaniel et al., 2023), compared to their older counterparts who tend to prefer using a URL to access a survey. However, a more recent small usability pilot of those 55 and older found that all users were familiar with what a QR code was and how to use it, with half of them having previously used a QR code (Rivas and Schulzetenberg, 2023). In addition to age, QR codes seem to attract females (McDaniel et al., 2023), those who are not currently married (Endres et al., 2023), Hispanics (Lee et al., 2023; Marlar and Schreiner, 2024), and non-whites (Endres et al., 2023; Lee et al., 2023).<\/p>\n

The previous research in this area has consistently focused on the effects of offering a QR code, but not necessarily on the actual use of QR codes. Our interest is to further this body of research by looking at the characteristics of those who used the QR code when it is offered alongside a URL. By using an additional parameter embedded on the URL string, we are able to determine when a QR code versus a URL was used.<\/p>\n

We are not aware of any research on the impact of including a QR code in invitation materials on the quality of the data collected from the survey. However, a study that compares data quality between web screening and in-person screening found that web screening is a viable option for enumerating households in population surveys (Graber et al., 2022). Web screening generally identified the same persons as in-person screenings, and full names were provided in 95% of web cases, compared to 37% of in-person cases. We hypothesize that QR codes may push more people to the web, leading to increases in web response, resulting in better data quality.<\/p>\n

Study Background<\/b><\/p>\n

The analysis presented in this paper focuses on web screening data from the Health and Retirement Study (HRS) to add to this growing literature. The HRS (https:\/\/hrs.isr.umich.edu) is a longitudinal panel study that began in 1992 (Fisher and Ryan, 2018). It is a national sample of adults over 50 and their spouses in the United States. The HRS is conducted every other year with a national probability sample of approximately 20,000 men and women. Every six years a new birth cohort of adults aged 51 to 56 are recruited into the panel to replenish the sample and ensure national representation of adults over age 50. In 2022, in addition to recruiting the next six-year birth cohort (those born between 1966-1971), the HRS recruited Hispanic and Black\/African Americans ages 57 and older (born prior to 1966) and identified households with an age eligible person for the next recruitment cycle in 2028 (those born between 1972-1977).<\/p>\n

In this 2022 recruitment effort, households were randomly assigned to one of two data collection protocols: 1) web first with in-person follow-up with non-responding households, or 2) in-person first, with nonresponding cases being offered the web. Web first households were mailed an invitation packet containing a letter inviting them to complete a web screener via a URL with login credentials alongside a fully authenticated QR code (see Figure 1), a Survey Research Center brochure, and a two-dollar bill. Two weeks later, nonresponding households were sent a reminder packet. Four weeks later, nonresponding cases were followed up in-person by a field interviewer. The in-person first group received up to 10 interviewer-administered contact attempts. Non-responders were invited to the web screener and received a similar packet as the web first group, but without a two-dollar bill or brochure.<\/p>\n

Figure 1: Example Web Invitation Letter<\/strong>\u00a0
\n\"\"<\/a><\/p>\n

Households completed a short screening instrument programmed in Blaise (5.10.6). The screener asked a set of questions to determine eligibility and confirmed the screener respondent both lived at the selected address and was at least 18 years old. The screener then asked how many people in the household were 18-44, 45-59, and 60 and older. If everyone in the household was under the age of 45, then the screener was complete with no one identified as eligible (see Figure 2). If anyone in the household was 45 or older, then a household roster was collected that gathered detailed information about each person including their name, race, ethnicity, and year of birth to determine eligibility. Eligible individuals were then asked to complete an in-person baseline interview that was approximately 3 hours in length.<\/p>\n

Figure 2: HRS Screening Eligibility Pathway<\/b>
\n<\/a>\"\"<\/a><\/p>\n

The introduction of the two options for accessing the web screener was not done experimentally. All mail-based communications included a URL alongside a QR code, therefore giving the potential screening respondent the choice of accessing the screener via the URL or the QR code. Therefore, this analysis is purely observational. While this work is not experimental, it offers valuable descriptive insight into what access methods different population subgroups prefer to use. We focus our investigation on the participation preferences among those who elected to screen in via the web.<\/p>\n

Research Questions<\/b><\/p>\n

This analysis seeks answers to the following research questions:\u00a0<\/span><\/p>\n

    \n
  1. Among screener respondents that participated via the web, how many chose to use the QR code to access the web screener?<\/span><\/li>\n
  2. Among screener respondents that participated via the web, are there particular demographic subgroups that chose to use the QR code to access the web screener?<\/span><\/li>\n
  3. Are there differences in screener data quality by access method (QR code vs URL) or mode (web vs. in-person)?<\/span><\/li>\n<\/ol>\n

    <\/h1>\n

    Methods<\/b><\/h1>\n

    QR code implementation<\/b><\/h2>\n

    The URL string underlying the QR code in the HRS screening invitation and reminder communications was a fully authenticated URL string containing an additional parameter that identified when a QR code had been used to access the screening instrument. Additionally, the QR code for each communication had a unique value for that parameter that allowed us to identify the communication (i.e., invitation or reminder) that was being used. The printing and mailing vendor was provided with the unique fully authenticated URLs and they generated the corresponding unique QR codes for each mailing.<\/p>\n

    Analytic methods<\/b><\/h2>\n

    This analysis focuses on the first five sample releases of the HRS 2022 recruitment effort, and includes screening effort outcomes from April 2022 through August 2023. Our analytic dataset includes households invited to the web screener from both protocol treatment arms who were offered a QR code alongside a URL either in an invitation letter or a non-response follow up letter. We did a sensitivity analysis including only those cases in the web first protocol group and the results did not change because just a small number of cases come from the non-response letter. A total of 1,468 households completed a screener via the web instrument, and only 12 (0.8%) came from a non-response follow-up letter to the field first cases.<\/p>\n

    Our analyses use demographic information collected from screener respondents age 18 and older. Per HRS project protocols, the exact age and race\/ethnicity of household members is not captured from screened households where all household members are 44 or younger. Therefore, race\/ethnicity subgroup analyses are limited to households that include at least one person 45 or older. For percentages of QR code choice within race\/ethnicity subgroups, age is dichotomized to screener respondents who are 18 – 56 or 57 and older to coincide with HRS-specific eligibility criteria. We use Pearson chi-square tests to test the association between dichotomized age and access method choice within race\/ethnicity groups. For the data quality analyses we use two sided z-tests for difference in proportions to compare the incomplete\/missing data percentages on first name, last name, year of birth, race, and ethnicity between QR code and URL users and between web and in-person modes. All analyses were conducted using SAS 9.4, and the analytic code can be found in the supplemental materials.<\/p>\n

    Results<\/b><\/h1>\n

    Research Question 1: How many screener respondents chose to use the QR code to access the web screener?<\/b><\/p>\n

    There were 34,313 sampled households invited to complete the screening instrument to determine HRS household eligibility. A total of 26,841 of those were invited to complete the screener via the web. A total of 13,501 households (AAPOR RR1 = 39%) completed a screener. This includes 1,468 by web (11% of all completed screeners, and 5.5% of those invited to complete on the web). Focusing on the 1,468 that completed on the web, 917 (62%) accessed the web screener via a QR code and 551 (38%) via the URL. Overall, 3.4% of the 26,841 sampled cases completed the screener via a QR code, and 2.1% completed via a URL. Among the 917 who used the QR code, 85% used the QR code associated with the invitation letter, with the remaining 15% using the nonresponse follow-up letter.<\/p>\n

    Research Question 2: Are there particular demographic subgroups that chose to use the QR code to access the web screener?\u00a0<\/b><\/p>\n

    To minimize the time spent at households found to be ineligible for the 2022 HRS, households where all members are under the age of 45 are not asked to complete a full household roster. Among households that accessed the web screener via a QR code, 38% did not meet this initial eligibility criteria. Among households that accessed the web screener via a URL, this ineligibility rate was 25%.<\/p>\n

    Considering next those households with at least one person 45 or older, Table 1 shows that 47% of screener respondents who chose a QR code to complete the web screener are 18-44, and 21% are 57 or older. Among screener respondents that chose to access the web screener via a URL, 31% were 18-44 and 46% were 57 or older. A total of 16% of QR code users were Hispanic, and 8% of URL users were Hispanic.<\/p>\n

    Eligibility for the HRS relies on a combination of age and race\/ethnicity. Overall, 40% of households were eligible for the HRS among those who chose to use a QR code, and 35% of households were HRS-eligible among those who chose to use a URL. Among those who chose to use a QR code, 12% are eligible for a future wave of HRS (born 1972-1977), 22% are eligible for the current wave (born 1966-1971) of HRS, and 6% are eligible for the current wave (born 1965 or before and minority). Among screener respondents who chose to use a URL, only 7% were eligible for a future wave, 16% for the current wave, and 12% for the current wave as older minorities.<\/p>\n

    \"\"<\/a><\/p>\n

    Looking within each race\/ethnicity category in more detail, we see consistent overall patterns among web screener respondents. A total of 75% of Hispanics, 61% of non-Hispanic Blacks, and 55% of non-Hispanic White\/Others chose a QR code to access the web screener; see Table 2. When we look within race\/ethnicity groups, the overall preference for QR codes is driven by the younger members of the group. Among Hispanic screener respondents, 81% aged 56 and younger chose to use a QR code, compared to 56% of those 57 and older (Pearson chi-square=8.4, p < .01). Among non-Hispanic Black screener respondents, 76% aged 56 and younger chose to use a QR code, compared to 42% of those 57 and older (Pearson chi-square=15.6, p < .0001). Among non-Hispanic White\/Other screener respondents, 67% aged 56 and younger chose to use a QR code, compared to 42% of those 57 and older (Pearson chi-square=44.2, p < .0001).<\/p>\n

    \"\"<\/a><\/p>\n

    Research Question 3: Are there differences in screener data quality by access method (QR code vs. URL) or mode (web vs. in-person)?<\/b><\/p>\n

    We did not observe any significant differences in any of the data quality metrics we looked at between the QR code and URL access methods; see Table 3. We see significantly higher rates of incomplete name data (first and last) when the screener is administered by an interviewer in-person. The first name is incomplete 29% of the time when the screening is done in-person, compared to 22.2% (z=5.13, p < .01) of the time on the web. The last name is incomplete 37% of the time when the screening is done in-person, compared to 21.7% of the time on the web (z=11.15, p < .0001). The collection of ethnicity, race, and year of birth was consistent in terms of item-missing data across all access methods and modes.<\/p>\n

    \"\"<\/a><\/p>\n

    <\/h1>\n

    Discussion<\/b><\/h1>\n

    When sampled cases for the 2022 HRS were invited to complete a screening instrument online and presented with a QR code alongside a URL, more cases chose to use a QR code to access the web screener. Since QR codes were not experimentally introduced, we cannot make direct comparisons of response rates engendered by these access methods. However, this analysis offers valuable descriptive insight into what different age and race\/ethnicity subgroups chose to use the QR codes to access the web screener. About half of those who chose to use a QR code were in our youngest (18-44) age group, although we saw usage in all age groups, with at least one QR code user over the age of 90.<\/p>\n

    Similar to what Lee et al. (2023) and Marlar and Schreiner et al. (2024) found, we found that Hispanics preferred to use QR codes across all age groups. This preference may be attributable to the 20% of Hispanic U.S. adults who are smartphone dependent (i.e., they have a smartphone but do not have home broadband), which is second only to Black U.S. adults at 21% (Pew, 2024), leaving their smartphone as their major means of regularly accessing the Internet. Regardless of the reason, this is an important finding as Hispanics and other minorities are often underrepresented in research, and lowering the barriers for their participation is beneficial.<\/p>\n

    Most importantly from this analysis, we found that data quality of the self-administered web screener was equivalent or better than an interviewer-administered in-person screener. Given that an interviewer needs to follow-up with any eligible cases from a web screener, we were happy to see that the data quality seems to be as good or better when the screener is completed via the web. Having just as much or more information to follow-up with during the first in-person visit for the baseline interview will hopefully make the interviewers\u2019 jobs easier. Furthermore, if the inclusion of QR codes helps to increase the percentage of sampled cases responding by web, there may be data quality benefits overall introduced by providing this access option.<\/p>\n

    A primary limitation of our study is that we did not offer a QR code experimentally. Everyone invited to complete the web screener was able to choose between the URL and the QR code. While this may provide some insight into who prefers different access methods, it does not allow for formal comparisons between the two access methods. As noted earlier, researchers are including QR codes as an access method for their surveys more frequently, and having insight into how they are implemented and used is valuable. QR codes are an inexpensive, easy way of making it more convenient to access a survey, but QR codes do have potential downsides. They can be used for quishing (i.e., QR code phishing), or they can be a barrier to participation for those without a device that can read or interpret them. Those who chose to use a QR code will also be using mobile devices, so a researcher will need to make sure their surveys are optimized for viewing on smaller screens. Future research into QR code implementation and usage is needed to determine the optimal way in which to maximize their potential benefit.<\/p>\n

    <\/h1>\n

     <\/p>\n","protected":false},"excerpt":{"rendered":"

    Introduction As survey response rates continue to decline and data collection costs continue to rise, researchers are under continuous pressure to look for innovative ways to combat those trends (Williams and Brick, 2018). One strategy researchers often introduce is a self-administered web option for respondents, either as the only mode, or in a mixed-mode design, […]<\/p>\n","protected":false},"author":5004,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[90,1086,1083,1084,1085],"class_list":["post-20208","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-data-quality","tag-health-and-retirement-study","tag-qr-code","tag-qr-code-choice","tag-web-screening-survey"],"acf":[],"_links":{"self":[{"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/posts\/20208","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/users\/5004"}],"replies":[{"embeddable":true,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=20208"}],"version-history":[{"count":68,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/posts\/20208\/revisions"}],"predecessor-version":[{"id":20680,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=\/wp\/v2\/posts\/20208\/revisions\/20680"}],"wp:attachment":[{"href":"https:\/\/surveyinsights.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=20208"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=20208"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/surveyinsights.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=20208"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}