Outcomes assessment pitfalls: challenges to quantifying knowledge gain in a sex education game

Abstract

Background: We describe challenges associated with incorporating knowledge assessment into an educational game on a sensitive topic and discuss possible motivations for, and solutions to, these challenges. Methods: The My Future Family Game (MFF) is a tool for collecting data about family planning intentions. The game was expanded to include information about human anatomy and sexual reproduction. To assess the efficacy of the game as a tool for teaching sexual education, we designed a pre-post study with assessments before and after the game which was deployed in three schools in and around Chennai, India in summer of 2018. Results: The pre-post process did not effectively assess knowledge gain and made the game less enjoyable. Although all participants completed the pre-test because it was required to access the main game, many did not complete the post test. As a result, the post-test scores are of limited use in assessing the efficacy of the intervention as an educational tool. This deployment demonstrated that pre-post testing has to be integrated in a way that motivates players to improve their scores in the post-test. The pre-test results did provide useful information about players’ knowledge of human anatomy and mechanisms of human reproduction prior to gameplay and validated the tool as a means of data collection. Conclusion: Adding outcomes assessment required asking players questions about sexual anatomy and function with little or no introduction. This process undermined elements of the initial game design and made the process less enjoyable for participants. Understanding these failures has been a vital step in the process of iterative game design. Modifications were made to the pre-post test process for future deployments so that the process of assessment does not diminish enthusiasm for game play or enjoyment and motivates completion of the post-test as part of gameplay.