Optical Mark Read Technology and Response Rates By Berwood Yost et al.

Research suggests that one of the easiest ways to increase mail survey response rates is to use a well-designed, attractive, and easy-to-complete questionnaire. Although a great deal of research has examined the response rate and quality of response to traditional mail surveys (TMS), little has been published regarding the use and effectiveness of Optical Mark Read (OMR) technology in mail surveys. OMR forms contain small circles, referred to as “bubbles,” that are filled in by the respondent. Some of the supposed advantages of OMR include ease of administration, increased speed and reliability of data input, and cost-effectiveness.

In a 2000 study, Don Dillman addressed the use of optical scanning and imaging on self-administered surveys, but was able to offer little empirical evidence about its impact on response rate. We found only one published study on this topic: Researchers Allen Klose and A. Dwayne Ball suggested in 1995 that OMR surveys could yield a response rate equal to that of the traditional mail survey (TMS); however, they found that response quality suffered. Respondents were more likely to omit answers or incorrectly complete the OMR survey form, resulting in unusable data or requiring the researchers to correct the forms by properly filling in the bubbles. The OMR survey was also found to be more expensive than the TMS for samples with less than 1,000 respondents.

Klose and Ball concluded that even though OMR surveys provide for cost-effective, high-speed, error-free data entry, the advantages might not be worth the problems associated with response quality. Additionally, because OMR surveys obtain a lower quality of response, the researchers concluded it should only be used when measuring simple items where response errors would be less likely to occur. They also concluded that, because OMR generates a higher cost per unit, it should only be used with a large sample.

In 2004, we conducted a study to attempt to answer some questions about OMR: Are response rates between traditional mail surveys and OMR surveys comparable? Does response quality differ between OMR and TMS formats? Which survey type is most cost-efficient? And do OMR surveys appear well-designed, attractive, and easy to complete?

Study participants represented a random sample of 3,321 employees from a local health care system. Every six months, this organization surveys a random sample of its employees about issues affecting their quality of work life and satisfaction.

For data collection conducted between January 12 and February 27, 2004, Franklin and Marshall College’s Center for Opinion Research prepared two versions of the survey, one using traditional mail survey software and one using OMR technology. Each questionnaire was formatted to fit on one double-sided 8½” by 11” sheet of paper. Respondents were asked about their satisfaction with workplace programs and policies such as benefits, work flexibility, and how the organization keeps them involved and informed. The survey also included questions regarding the organization’s efforts to study employee satisfaction. Two of these measured the survey’s visual appeal and its ease of use.

Survey packets were mailed to participants’ home addresses using the health care system’s letterhead and envelopes. Each packet contained one of the two formatted survey instruments, a cover letter detailing the use of the survey, an explanation of the tracking number, a guarantee of confidentiality, and a business reply envelope. A total of 1,665 participants received the TMS, and a total of 1,656 participants received the OMR survey. Respondents were randomly assigned to one or the other. Two weeks after the first mailing, a follow-up postcard was sent. A second mailing was conducted one week after the postcard reminder.

Worth The Cost? A Research Note on Optical Mark Read Technology and Response Rates