Methods to increase response to postal and electronic questionnaires
Top Cited Papers
- 8 July 2009
- journal article
- research article
- Published by Wiley in Emergencias
- Vol. 2010 (1), MR000008
- https://doi.org/10.1002/14651858.mr000008.pub4
Abstract
Postal and electronic questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of health research. To identify effective strategies to increase response to postal and electronic questionnaires. We searched 14 electronic databases to February 2008 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, we also contacted authors to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria. Randomised controlled trials of methods to increase response to postal or electronic questionnaires. We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity among trial odds ratios using a Chi2 test and the degree of inconsistency between trial results was quantified using the I2 statistic. Postal We found 481 eligible trials. The trials evaluated 110 different ways of increasing response to postal questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.87; 95% CI 1.73 to 2.04; heterogeneity P < 0.00001, I2 = 84%), recorded delivery (1.76; 95% CI 1.43 to 2.18; P = 0.0001, I2 = 71%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08; 95% CI 1.27 to 7.44) and a more interesting questionnaire topic (2.00; 95% CI 1.32 to 3.04; P = 0.06, I2 = 80%). The odds of response were substantially higher with pre-notification (1.45; 95% CI 1.29 to 1.63; P < 0.00001, I2 = 89%), follow-up contact (1.35; 95% CI 1.18 to 1.55; P < 0.00001, I2 = 76%), unconditional incentives (1.61; 1.36 to 1.89; P < 0.00001, I2 = 88%), shorter questionnaires (1.64; 95% CI 1.43 to 1.87; P < 0.00001, I2 = 91%), providing a second copy of the questionnaire at follow up (1.46; 95% CI 1.13 to 1.90; P < 0.00001, I2 = 82%), mentioning an obligation to respond (1.61; 95% CI 1.16 to 2.22; P = 0.98, I2 = 0%) and university sponsorship (1.32; 95% CI 1.13 to 1.54; P < 0.00001, I2 = 83%). The odds of response were also increased with non-monetary incentives (1.15; 95% CI 1.08 to 1.22; P < 0.00001, I2 = 79%), personalised questionnaires (1.14; 95% CI 1.07 to 1.22; P < 0.00001, I2 = 63%), use of hand-written addresses (1.25; 95% CI 1.08 to 1.45; P = 0.32, I2 = 14%), use of stamped return envelopes as opposed to franked return envelopes (1.24; 95% CI 1.14 to 1.35; P < 0.00001, I2 = 69%), an assurance of confidentiality (1.33; 95% CI 1.24 to 1.42) and first class outward mailing (1.11; 95% CI 1.02 to 1.21; P = 0.78, I2 = 0%). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94; 95% CI 0.88 to 1.00; P = 0.51, I2 = 0%). Electronic We found 32 eligible trials. The trials evaluated 27 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were increased by more than a half using non-monetary incentives (1.72; 95% CI 1.09 to 2.72; heterogeneity P < 0.00001, I2 = 95%), shorter e-questionnaires (1.73; 1.40 to 2.13; P = 0.08, I2 = 68%), including a statement that others had responded (1.52; 95% CI 1.36 to 1.70), and a more interesting topic (1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third using a lottery with immediate notification of results (1.37; 95% CI 1.13 to 1.65), an offer of survey results (1.36; 95% CI 1.15 to 1.61), and using a white background (1.31; 95% CI 1.10 to 1.56). The odds of response were also increased with personalised e-questionnaires (1.24; 95% CI 1.17 to 1.32; P = 0.07, I2 = 41%), using a simple header (1.23; 95% CI 1.03 to 1.48), using textual representation of response categories (1.19; 95% CI 1.05 to 1.36), and giving a deadline (1.18; 95% CI 1.03 to 1.34). The odds of response tripled when a picture was included in an e-mail (3.05; 95% CI 1.84 to 5.06; P = 0.27, I2 = 19%). The odds of response were reduced when "Survey" was mentioned in the e-mail subject line (0.81; 95% CI 0.67 to 0.97; P = 0.33, I2 = 0%), and when the e-mail included a male signature (0.55; 95% CI 0.38 to 0.80; P = 0.96, I2 = 0%). Health researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this systematic review.Keywords
This publication has 106 references indexed in Scilit:
- Effect of Incentives and Mailing Features on Online Health Program EnrollmentAmerican Journal of Preventive Medicine, 2008
- Effects of personalization and envelope color on response rate, speed and quality among a business populationIndustrial Marketing Management, 2007
- The effect of visual stimuli on mail survey response ratesIndustrial Marketing Management, 1995
- Believe it or not! longer questionnaires have lower response ratesJournal of Business and Psychology, 1990
- An examination of the effects of questionnaire factors on response to an industrial mail surveyInternational Journal of Research in Marketing, 1989
- The effect of two variables on industrial mail survey returnsIndustrial Marketing Management, 1985
- Effects of signed versus unsigned internally administered questionnaires for managersJournal of Business Research, 1978
- Methodological considerations in corporate mail surveys: A research noteJournal of Business Research, 1976
- An experimental investigation of mail-survey responsesJournal of Business Research, 1975
- Initial returns on mail questionnaires: A literature review and research noteResearch in Higher Education, 1974