Cognitive Processing of Self-Report Items in Educational Research: Do They Think What We Mean?
- 27 July 2007
- journal article
- research article
- Published by Informa UK Limited in Educational Psychologist
- Vol. 42 (3), 139-151
- https://doi.org/10.1080/00461520701416231
Abstract
Techniques emerging from the considerable research on cognitive aspects of survey methodology include various forms of probing and cognitive interviewing. These techniques are used to examine whether respondents' interpretations of self-report items are consistent with researchers' assumptions and intended meanings given the constructs the items are designed to measure. However, although informal procedures are common, such developments have not been systematically applied in educational research. We describe how information derived from the systematic application of cognitive pretesting can contribute to determining the validity—designated cognitive validity—of self-report items. Examples are presented from prominent motivation-related instruments that assess real-world instructional practices, mastery classroom goal structure, and student self-efficacy. The implications and pragmatics of adopting this approach are discussed.Keywords
This publication has 36 references indexed in Scilit:
- A Historical Perspective and Current Views on the Standards for Educational and Psychological TestingEducational Measurement: Issues and Practice, 2006
- The Development and Evaluation of Procedures to Assess Child Self-Report Item Validity Educational and Psychological MeasurementEducational and Psychological Measurement, 2006
- Classroom Goal Structure, Student Motivation, and Academic AchievementAnnual Review of Psychology, 2006
- Are We Asking the Right Questions? Using Cognitive Interviews to Improve Surveys in Education ResearchEducational Evaluation and Policy Analysis, 2004
- Using Multiple Methods to Assess Students' Perceptions of Classroom Goal StructuresEuropean Psychologist, 2004
- Learning environment, motivation, and achievement in high school scienceJournal of Research in Science Teaching, 2003
- The Classroom Social Environment and Changes in Adolescents’ Motivation and Engagement During Middle SchoolAmerican Educational Research Journal, 2001
- Assessing Student Perceptions of Classroom Methods and Activities in the Context of an Outcomes-Based EvaluationEvaluation Review, 2000
- An Empirical Evaluation of In-Depth Probes Used to Pretest Survey QuestionsSociological Methods & Research, 1998
- The Value (And Convergence) of Practices Suggested by Motivation Research and Promoted by Mathematics Education ReformersJournal for Research in Mathematics Education, 1998