Design, Technology, and Measurement Considerations in Virtual-Reality Assessment

Introduction

In online competency-based higher education (OCBHE), students move through their degree programs one course at a time by demonstrating mastery of various content and professional domains. As students work within an individual domain of knowledge, they are assessed on one or more topics, each consisting of a series of competencies with associated test objectives and performance tasks (Gyll & Ragland, 2018). As the popularity of OCBHE programs continues to grow, their integrity will be scrutinized by students and employers alike, and their credibility will largely depend on the quality of the assessments used (McClarty & Gaertner, 2015).

To meet this demand, institutions of higher learning are beginning to realize the importance of real-world, performance-based measures like those found in Virtual Reality (VR) environments, where demonstrations of competence are crucial to success. Central to the development of VR assessments is the need for high-fidelity demonstrations of learner aptitudes and competencies that include a significant emphasis on cognitive abilities (i.e., knowing what) and the performance of those abilities (i.e., demonstrating how). Assessment fidelity describes the reality of function and how closely the environment imitates the real-life counterpart. The most significant advantage of this testing mode is the real-world relevance that can be incorporated into the assessment, and technological innovations continue to expand these opportunities.

Competency-based skills education follows a system of assessing needs, designing processes, developing materials, and evaluating them against specific learning outcomes. Within this context, learning is no longer viewed as transmitting knowledge from instructor to student (e.g., sitting passively, listening to a lecture, taking notes, and applying concepts) but as an active process acquired through various instructional and media types. Compared to traditional instructor-led methods, research shows that competency-based skills are improved when technology integrates learning into the educational experience rather than being delivered in a compartmentalized fashion (Ford & Gopher, 2015; Hoogveld, 2003; Van Merrienboer & Kirschner, 2013; Merrill, 2002). As a result, students' capacity to develop a particular domain's competency and then transfer that information to learning outcomes improves (Gyll & Hayes, in press).

Some have suggested that skills development as a first step to acquiring competency can be better taught with high-fidelity VR simulation because VR allows for a more active and immersive learning experience (Kyaw et al., 2019; Birrenbach et al., 2021). For example, VR is often used in medical education for developing skills of varying complexity, ranging from simple nursing skills to laparoscopic/endoscopic/endovascular skills (Pantelidis et al., 2017; Khan et al., 2019) or complex surgical skills (Hooper et al., 2019; Grantcharov, 2008). Additionally, VR simulation using head-mounted devices (HMDs) offers a multisensory, 3-D, fully immersive experience. Through immersion, sense of presence, and interaction with the virtual environment in a real-time and realistic manner, VR simulation can create emotional experiences that facilitate experiential learning, exceeding other 2-D learning modalities (Babini et al., 2020). However, while these studies have demonstrated the utility of VR simulation as a teaching tool, scant literature exists on its utility as an assessment instrument. Furthermore, fewer studies exist on meaningful development guidelines to aid assessment professionals in the development process.