Myths and Realities of Respondent Engagement in Online Surveys

Abstract

This paper describes an experiment in which a single questionnaire was fielded in four different styles of presentation: Text Only, Decoratively Visual, Functionally Visual and Gamified. Respondents were randomly assigned to only one presentation version. To understand the effect of presentation style on survey experience and data quality, we compared response distributions, respondent behaviour (such as time to complete), and self-reports regarding the survey experience and level of engagement across the four experimental presentations. While the functionally visual and gamified treatments produced higher satisfaction scores from respondents, we found no real differences in respondent engagement measures. We also found few differences in response patterns.