Improving Survey Methods With Cognitive Interviews in Small- and Medium-Scale Evaluations
Open Access
- 16 March 2012
- journal article
- research article
- Published by SAGE Publications in American Journal of Evaluation
- Vol. 33 (3), 414-430
- https://doi.org/10.1177/1098214012441499
Abstract
Findings derived from self-reported, structured survey questionnaires are commonly used in evaluation and applied research to inform policy-making and program decisions. Although there are a variety of issues related to the quality of survey evidence (e.g., sampling precision), the validity of response processes–how respondents process thoughts and perceptions when answering questionnaires–is crucial. We assess the extent to which cognitive interviews, used to test survey questions as part of the process of questionnaire design and refinement, can strengthen the quality of survey evidence for small- and medium-scale evaluations. To illustrate how cognitive interviews can reveal respondents' response processes and improved questionnaire interpretations, we present excerpts from two types of evaluations (e.g., needs assessment) in two domains (e.g., education). The paper concludes with a brief summary about how the use of cognitive interviews can be augmented in survey development, refinement, and adaptation to improve survey questionnaire interpretations in evaluations with limited resources.Keywords
This publication has 25 references indexed in Scilit:
- Sample Size for Cognitive Interview PretestingPublic Opinion Quarterly, 2011
- Sources of Error in Cognitive InterviewsPublic Opinion Quarterly, 2009
- Research Synthesis: The Practice of Cognitive InterviewingPublic Opinion Quarterly, 2007
- The Keys to Effective Schools: Educational Reform as Continuous ImprovementPublished by SAGE Publications ,2007
- An Assessment of Evaluation DesignsAmerican Journal of Evaluation, 2006
- Designing SurveysPublished by SAGE Publications ,2005
- Do Different Cognitive Interview Techniques Produce Different Results?Published by Wiley ,2004
- Are We Asking the Right Questions? Using Cognitive Interviews to Improve Surveys in Education ResearchEducational Evaluation and Policy Analysis, 2004
- Quality of School-Based Prevention Programs: Results from a National SurveyJournal of Research in Crime and Delinquency, 2002
- Protocol AnalysisPublished by MIT Press ,1993