Finding a Comparison Group
- 18 June 2013
- journal article
- research article
- Published by SAGE Publications in American Journal of Evaluation
- Vol. 34 (3), 372-384
- https://doi.org/10.1177/1098214013490223
Abstract
This article explores the viability of online crowdsourcing for creating matched-comparison groups. This exploratory study compares survey results from a randomized control group to survey results from a matched-comparison group created from Amazon.com ’s MTurk crowdsourcing service to determine their comparability. Study findings indicate that online crowdsourcing, a process that allows access to many participants to complete specific tasks, is a potentially viable resource for evaluation designs where access to comparison groups, large budgets, and/or time are limited. The article highlights the strengths and limitations of the online crowdsourcing approach and describes ways that it could potentially be used in evaluation practice.Keywords
This publication has 15 references indexed in Scilit:
- Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical TurkPolitical Analysis, 2012
- The viability of crowdsourcing for survey researchBehavior Research Methods, 2011
- Amazon's Mechanical TurkPerspectives on Psychological Science, 2011
- The labor economics of paid crowdsourcingPublished by Association for Computing Machinery (ACM) ,2010
- The national survey of student engagement: Conceptual and empirical foundationsNew Directions for Institutional Research, 2009
- Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within‐study comparisonsJournal of Policy Analysis and Management, 2008
- Psychological Research Online: Report of Board of Scientific Affairs' Advisory Group on the Conduct of Research on the Internet.American Psychologist, 2004
- Causal Effects in Nonexperimental Studies: Reevaluating the Evaluation of Training ProgramsJournal of the American Statistical Association, 1999
- Simulation Study of Nonequivalent Group-Matching and Regression-Discontinuity Designs: Evaluations of Gifted and Talented ProgramsThe Journal of Experimental Education, 1998
- The central role of the propensity score in observational studies for causal effectsBiometrika, 1983