Digital Badges for Stealth Assessment

Introduction

Stealth assessment in games has been shown to be a valid method of assessment. For example, Shute et al. (2021) conducted a study of 264 high school students who played a physics game. Stealth assessment estimates of physics understanding were significantly correlated with external physics test scores. In games, stealth assessment enables instructors to identify a learner’s current skills, competencies, and knowledge, which can be used to support student learning by, for example, adjusting difficulty or which topics are presented next. While traditional assessment can also perform these functions, stealth assessment limits their intrusiveness, may improve accuracy, and enables more frequent and dynamic measurement and adjustment due to its unobtrusive nature (Shute, 2011).

Assessment can be an exhausting and stressful process for the person being assessed, and oftentimes it is not important for the person to be aware of the assessment. For example, consider an adaptive learning system – a system that creates learning experiences that are unique to each individual learner’s needs and interests (Monova-Zhelava, 2005; Rosmalen et al., 2006). Adaptive learning systems are those systems that adjust the type, content, or presentation of information to best suit the learner’s current learning needs. In order to tailor the information in this way, the system must first understand what those needs are. The more frequently the system can take measure of a learner’s needs, the more frequently the system can adapt, leading to a system that reacts to learner needs in a manner that is both timely and sensitive to the current context. Consider two students struggling to pass the same lesson in an introductory algebra class. While they may appear to be struggling with the same material, the reasons for their struggle may be different; perhaps, student A does not yet understand variables, while student B is not adept with division. Here, student A is struggling with a fundamental algebra concept and student B cannot apply the concepts due to insufficient pre-requisite knowledge. In this situation, a stealth assessment system can use results from each student to identify the precise knowledge or skill set that is underdeveloped and make personalized recommendations for additional learning modules that will help each student catch up and succeed. Sometimes, automated systems are unable to understand the problem with enough granularity to make the correct recommendations; the human touch is necessary. Now, consider the same situation that combines hidden assessment and badges. Designers can first identify the knowledge required to succeed in each lesson. They may then create badges for each bit of prerequisite knowledge, as well as assessments to measure them at regular intervals. When a teacher sees that a student is struggling, they can review the student’s earned and unearned badges to better understand the student’s capabilities. From here, they may use that understanding to create personalized lessons and resources for the struggling student. The utility of such an approach is magnified when we consider large classrooms with hundreds or thousands of learners, where the instructor may be unable to understand learner capabilities at such a fine level of granularity without the use of a computer-aided system.

Likewise, instructional scaffolding is a process through which instructional support is provided to a learner based on their current needs (Wood et al., 1976). The support is gradually removed as the learner advances in certain topics, and support is added for more advanced topics in which the learner will next acquire mastery. Here, too, understanding the learner’s current abilities is critical for adding and removing support at the right times, in the right ways, and for the right content. While frequent assessment is useful for purposes like scaffolding and adaptive learning, at some point, traditional assessment will become overbearing due to its intrusive nature. To address this concern, stealth assessment can happen behind the scenes, in real-time (Shute, Ventura, Zapata-Rivera, & Bauer, 2009; Shute & Ventura, 2013) assessing the learner as frequently as necessary while reducing or eliminating negative assessment effects (e.g., test anxiety) while maintaining engagement (Wang, Shute, & Moore, 2015; Shute, Hansen, & Almond, 2008).