Is a Web Survey as Effective as a Mail Survey? A Field Experiment Among Computer Users

Abstract

With the exponential increase in Web access, program evaluators need to understand the methodological benefits and barriers of using the Web to collect survey data from program participants. In this experimental study, the authors examined whether a Web survey can be as effective as the more established mail survey on three measures of survey effectiveness: response rate, question completion rate, and the lack of evaluative bias. Community- and university-based educators ( n = 274) attending a 2-day program were randomly assigned to receive a Web or mail survey evaluating the program. Among those participants successfully solicited by e-mail, Web survey participants were more likely to respond (95%) than mail survey participants (79%). Web survey participants completed similarly high numbers of quantitative questions as mail survey participants, provided longer and more substantive responses to qualitative questions, and did not demonstrate evidence of evaluative bias. These results suggest that program evaluators could expand their use of Web surveys among computer users.