Data Quality and Cost-effectiveness Analyses of Electronic and Paper-Based Interviewer-Administered Public Health Surveys: Systematic Review

Abstract

Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Population-level survey (PLS) is an essential standard method used in public health research. It supports to quantify sociodemographic events and support public health policy development and intervention designs with evidence. Though all steps in the survey can contribute to the data quality parameters, data collection mechanisms seem the most determinant to avoid mistakes before they happen. The use of electronic devices such as smartphones and tablet computers improve the quality and cost-effectiveness of public health surveys. However, there is a lack of systematically analyzed evidence to show the potential impact of electronic-based data collection tools on data quality and cost reduction in interviewer-administered surveys compared to the standard paper-based data collection system. Objective: This systematic review aims to evaluate the impact of interviewer-administered electronic device data collection methods concerning data quality and cost reduction in PLS compared to the traditional paper-based methods. Methods: A systematic search was conducted in MEDLINE, CINAHL, PsycINFO, the Web of Science, EconLit and Cochrane CENTRAL, and CDSR to identify relevant studies from 2008 to 2018. We included randomized and non-randomized studies that examine data quality and cost reduction outcomes. Moreover, usability, user experience, and usage parameters from the same studies were included. Two independent authors screened the title, abstract, and finally extracted data from the included papers. A third author mediated in case of disagreement. The review authors used EndNote for de-duplication and Rayyan to screen and note the reasons for inclusion and exclusion based on the protocol. Meta-analysis was planned if the studies were considered combinable with minimal heterogeneity. Results: The search strategy from the electronic databases found 3,817 articles. After de-duplication, 2,533 articles were screened, and 14 articles fulfilled the inclusion criteria. None of the studies was designed as a randomized control trial. Most of the studies have a quasi-experimental design, like comparative experimental evaluation studies nested on the other ongoing cross-sectional surveys. 4 comparative evaluations, 2 pre-post intervention comparative evaluation, 2 retrospectives comparative evaluation, and 4 one arm non-comparative studies were included in our review. Meta-analysis was not possible because of the heterogeneity in study design, the type, and level of outcome measurements and the study settings. Individual article synthesis showed that data from electronic data collection systems possessed good quality data and delivered faster when compared to the paper-based data collection system. Only two studies linked the cost and data quality outcomes to describe the cost-effectiveness of electronic-based data collection systems. The majority of the reported costs are partial or differential cost estimations with extrapolated cost information from small-scale surveys. Despite the poor economic evaluation qualities, most of the reported results were in favor of EDC for the large-scale surveys. The field data collectors reported that an electronic data collection system was a feasible, acceptable and preferable tool for their work. Onsite data error prevention, fast data submission, and easy to handle devices were the comparative advantages of electronic data collection systems. Technical difficulties, accidental data loss, device theft, security concerns, power surges, and internet connection problems were reported as challenges during the implementation. Conclusions: Though positive evidence existed about the comparative advantage of electronic data capture over paper-based tools, the included studies were not methodologically rigorous enough to combine. We need more rigorous studies that demonstrate the comparative evidence of paper and electronic-based data collection systems in public health surveys on data quality, work efficiency, and cost reduction. The review protocol is registered in the International Prospective Register for Systematic Reviews (PROSPERO) CRD42018092259. The protocol of this article was also pre-published (JMIR Res Protoc 2019;8(1): e10678 doi:10.2196/10678).