This contribution explores the connections between gameful elements, bodily interactions, and musical features in the context of three case studies of music learning applications, demonstrating the role of these components in accomplishing the application's task. The critical essential element common to the three case studies is engagement due to gameful elements and other design characteristics such as the choice of music materials and the link between their structural elements and full body/gestural interaction. Assessment and experimental results demonstrate that these characteristics deeply influence users' involvement, memory, and learning.
TopIntroduction
The great diffusion of electronic devices and ICT technologies have made gamification pervasive in everyday digital life. Gameful elements, video games and game-inspired design seem to be intimately connected to a great deal of computer activities such as work (Oravec, 2015), entertainment (Sharma & Somani, 2019), advertising (Terlutter & Capella, 2013), behavior modeling (Volkswagen, 2009), business management (Lim & Rubasundram, 2018) and education (Nordby et al., 2016). There are probably many reasons to explain this phenomenon, the most obvious of which are the involvement, the fun, the role of challenges and the power of attraction demonstrated by gamification. Previously employed to improve market strategies or for military training, gamification has spread to many sectors of activity, including education. Since the dawn of computer-assisted education, programmers have always sought new ways to grab students' attention leveraging the power of multimedia content through the use of plasma displays, bitmapped graphics and sound (Hofstetter, 1981). Nowadays, gamification seems to respond to the same needs, but reinterpreting the multimedia paradigm in the light of the new possibilities offered by human-computer interaction. However, while the use of gamification brings certain advantages when used inside its original domains, its employment in educational contexts raises some problems, especially in relation to educational strategies and game mechanics. Arnab et al. (2015) define the serious game mechanic as the element that actually realizes effectiveness in the instructional design as it links the gameplay to pedagogical practices. Particularly, just the lack of reflection about the relationships between the gameful elements and the learning process has been addressed by the Game Object Model (GOM) proposed by Amory et al. (1999). In this model the idea of abstract and concrete interfaces is introduced. Abstract interfaces relate to the pedagogical approach, while concrete interfaces represent the actual design elements (gameplay and software). Leveraging the Object-oriented programming paradigm1, the GOM connects these interfaces to objects organized according to encapsulation, inheritance and polymorphism, concepts which are used to guarantee the interrelationships between the pedagogical and the gamification domain.
In this book chapter the same problem is tackled from the point of view of music serious games based on gameful elements and bodily interaction. The contribution does not focus on the theory but rather on the analysis of the design elements and the critical evaluation of three case studies. The first one is “Harmonic Walk”, a large-space interactive space where users' movements are tracked by a camera hanging from the ceiling. Six interactive landmarks trigger six connected musical chords to be played according to the position of the user. This allows to link the movements of the user to the timed harmonic changes implied in a tonal melody, thus creating a dance movement which follows the changes of the musical chords in accompaniment to the melody2.
A desktop version of “Harmonic Walk” is “Harmonic Touch”, a web platform for the study and practice of tonal harmony. The platform leverages interactive maps, graphical representations, score calculation, audio elements, and positive/negative reinforcement to engage students in three experiences related to fundamental aspects of the musical language such as implicit harmony, harmonic rhythm and melody harmonization. Using the platform through their customized accounts allowed teachers to control the activities of their students and prepare targeted assignments for them during the Covid-19 pandemic. Thus teachers could experiment the effects of engagement in the music learning process of elementary school children during the period of school closure3.
The third case study is “Good or Bad”, a two-players game based on full-body interaction. The two players have to rebuild track by track a musical piece by accepting or rejecting music excerpts belonging to two different songs. This happens in a large-scale interactive environment where the landmarks of the gameplay appear as floor projections4.