Multimodal Student Engagement Recognition in Prosocial Games
Open Access
- 23 August 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Games
- Vol. 10 (3), 292-303
- https://doi.org/10.1109/tciaig.2017.2743341
Abstract
In this paper we address the problem of recognizing student engagement in prosocial games by exploiting engagement cues from different input modalities. Since engagement is a multifaceted phenomenon with different dimensions, i.e., behavioral, cognitive and affective, we propose the modeling of student engagement using real-time data from both the students and the game. More specifically, we apply body motion and facial expression analysis to identify the affective state of students, while we extract features related to their cognitive and behavioral engagement based on the analysis of their interaction with the game. For the automatic recognition of engagement, we adopt a machine learning approach based on artificial neural networks, while for the annotation of the engagement data, we introduce a novel approach based on the use of games with different degrees of challenge in conjunction with a retrospective self-reporting method. To evaluate the proposed methodology, we conducted real-life experiments in four classes, in three primary schools, with 72 students and 144 gameplay recordings in total. Experimental results show the great potential of the proposed methodology, which improves the classification accuracy of the three distinct dimensions with a detection rate of 85%. A detailed analysis of the role of each component of the Game Engagement Questionnaire (GEQ), i.e., immersion, presence, flow and absorption, in the classification process is also presented in this paper.Keywords
Funding Information
- EU Horizon 2020 Framework Programme (644204)
This publication has 39 references indexed in Scilit:
- Detecting Driver Drowsiness Based on Sensors: A ReviewSensors, 2012
- An Automated Optimal Engagement and Attention Detection System Using ElectrocardiogramComputational and Mathematical Methods in Medicine, 2012
- Assessing twenty‐first century skills through a teacher created video game for high school biology studentsResearch in Science & Technological Education, 2010
- When helping helps: Autonomous motivation for prosocial behavior and its influence on well-being for the helper and recipient.Journal of Personality and Social Psychology, 2010
- The development and evaluation of a survey to measure user engagementJournal of the American Society for Information Science and Technology, 2009
- Junior Secondary Students' Perceptions of Influences on Their Engagement with SchoolingAustralian Journal of Education, 2009
- Affect-aware tutors: recognising and responding to student affectInternational Journal of Learning Technology, 2009
- Multimethod assessment of affective experience and expression during deep learningInternational Journal of Learning Technology, 2009
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- The motivation of gameplayOn the Horizon, 2002