The viability of crowdsourcing for survey research
Top Cited Papers
Open Access
- 25 March 2011
- journal article
- Published by Springer Science and Business Media LLC in Behavior Research Methods
- Vol. 43 (3), 800-813
- https://doi.org/10.3758/s13428-011-0081-0
Abstract
Online contract labor portals (i.e., crowdsourcing) have recently emerged as attractive alternatives to university participant pools for the purposes of collecting survey data for behavioral research. However, prior research has not provided a thorough examination of crowdsourced data for organizational psychology research. We found that, as compared with a traditional university participant pool, crowdsourcing respondents were older, were more ethnically diverse, and had more work experience. Additionally, the reliability of the data from the crowdsourcing sample was as good as or better than the corresponding university sample. Moreover, measurement invariance generally held across these groups. We conclude that the use of these labor portals is an efficient and appropriate alternative to a university participant pool, despite small differences in personality and socially desirable responding across the samples. The risks and advantages of crowdsourcing are outlined, and an overview of practical and ethical guidelines is provided.Keywords
This publication has 39 references indexed in Scilit:
- Measurement Equivalence of Paper‐and‐Pencil and Internet Organisational Surveys: A Large Scale Examination in 16 CountriesApplied Psychology, 2009
- AlphaTestApplied Psychological Measurement, 2008
- Employee Surveys Administered OnlineOrganizational Research Methods, 2007
- The Measurement Equivalence of Web-Based and Paper-and-Pencil Measures of Transformational LeadershipOrganizational Research Methods, 2006
- Ascertaining the validity of individual protocols from Web-based personality inventoriesJournal of Research in Personality, 2004
- A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews.Journal of Applied Psychology, 1999
- The Computer Understanding and Experience Scale: a self-report measure of computer experienceComputers in Human Behavior, 1998
- Generalizability of Psychological Research from Undergraduates to Employed AdultsThe Journal of Social Psychology, 1993
- Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference?Journal of Applied Psychology, 1992
- The volunteer subject revisitedAustralian Journal of Psychology, 1976