Peer and self assessment in massive online classes
Top Cited Papers
Open Access
- 1 December 2013
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer-Human Interaction
- Vol. 20 (6), 1-31
- https://doi.org/10.1145/2505057
Abstract
Peer and self-assessment offer an opportunity to scale both assessment and learning to global classrooms. This article reports our experiences with two iterations of the first large online class to use peer and self-assessment. In this class, peer grades correlated highly with staff-assigned grades. The second iteration had 42.9% of students’ grades within 5% of the staff grade, and 65.5% within 10%. On average, students assessed their work 7% higher than staff did. Students also rated peers’ work from their own country 3.6% higher than those from elsewhere. We performed three experiments to improve grading accuracy. We found that giving students feedback about their grading bias increased subsequent accuracy. We introduce short, customizable feedback snippets that cover common issues with assignments, providing students more qualitative peer feedback. Finally, we introduce a data-driven approach that highlights high-variance items for improvement. We find that rubrics that use a parallel sentence structure, unambiguous wording, and well-specified dimensions have lower variance. After revising rubrics, median grading error decreased from 12.4% to 9.9%.Keywords
Funding Information
- Division of Information and Intelligent Systems
- Hasso Plattner Design Thinking Program
This publication has 53 references indexed in Scilit:
- Promoting creativity in the computer science design studioPublished by Association for Computing Machinery (ACM) ,2011
- The Social Psychological Effects of Feedback on the Production of Internet Information PoolsJournal of Computer-Mediated Communication, 2008
- Exiting the Cleanroom: On Ecological Validity and Ubiquitous ComputingHuman–Computer Interaction, 2008
- Teaching With Rubrics: The Good, the Bad, and the UglyCollege Teaching, 2005
- Distance learning through distributed collaborative video viewingPublished by Association for Computing Machinery (ACM) ,2000
- Sustainable Assessment: Rethinking assessment for the learning societyStudies in Continuing Education, 2000
- Evaluating an Automatically Scorable, Open‐Ended Response Type for Measuring Mathematical Reasoning in Computer‐Adaptive TestsJournal of Educational Measurement, 1997
- Interactions, 1996
- Social psychology of creativity: A consensual assessment technique.Journal of Personality and Social Psychology, 1982
- Recognition and retrieval processes in free recall.Psychological Review, 1972