Abstract
This chapter explores how learning analytics can enhance learning and teaching in large scale, introductory programming courses. More specifically, it examines analytical approaches to identify at-risk students, personalize learning experiences, and make informed decisions about instructional content and delivery. Case examples drawn from empirical research are outlined to warrant a conceptual framework for best practice in analyzing data for these purposes. In this chapter, the authors review the benefits of temporal data, such as late assignment submission times, in terms of early detection of at-risk students. They also highlight the use of clustering algorithms in differentiating amongst the specific needs of different students using multidimensional data, allowing for tailoring instruction in an optimal manner. Finally, they discuss challenges in aligning data to gain insights into skill acquisition as a result of study habits to inform instructional decision making.
TopIntroduction
In recent years, there has been an increasing shift in computing education due to the coronavirus pandemic. Students now have the opportunity to gain experience with a mix of online, blended, and hybrid environments, which exposes them to various instructional technologies and pedagogies (Siegel et al., 2021). From pre- to post-pandemic, undergraduate students in large-scale CS1 classrooms have expressed interest in more flexible modalities that cater to their specific needs and interests. They seek a combination of traditional face-to-face instruction and the accessibility of online content when required. Many students turn to online and distance education not only for its convenience but also for the comfort of learning in a setting of their choice and seeking help online. However, this approach also presents challenges for both students and instructors. With fewer opportunities to communicate in a face-to-face setting and receive help in a timely manner, students may feel frustrated or confused. While online technologies facilitate interactive learning experiences, students may not always recognize when it is necessary to seek assistance or make the best of every opportunity to practice and refine their skills through feedback (Medeiros, Ramalho, Falcão, 2018).
To assist students in becoming more proficient in programming, it is crucial to understand the idealized trajectories or paths taken by students within these instructional contexts. Cognitive skill acquisition involves the ability to solve problems and perform specific tasks by acquiring, retaining, and applying interconnected pieces of knowledge over time, with improvements in speed and accuracy (VanLehn, 1996). Information processing theories in computing education allow us to identify, structure, and sequence the development of programming skills in students (Xie et al., 2019). Although there are significant differences among theories and models, students learning a programming language are generally described as making continuous efforts to translate information in their working memory. The execution of cognitive tasks involves interpreting the meaning of syntactic elements in a programming language and explaining their function, whether for specific statements or how the resulting operations interact with one another (Schulte, 2008). Theories provide a foundation for differentiating diverse types of skills, such as the ability to trace code execution, write syntax correctly, comprehend reusable abstractions of programming knowledge, and purposefully write code while relying on this knowledge.
Recent research has made significant strides in understanding the interdependence of skills. The tightly coupled nature of these skills may support an ideal sequencing of practice and feedback opportunities (Fowler, Smith, Hassan, Poulsen, West, & Zilles, 2022). However, there are still important questions regarding which programming concept are most challenging for students to understand, what types of practice opportunities are most effective, how to order these exercises, as well as when and how to provide feedback (Becker & Quille, 2019; Denny, Becker, Craig, Wilson, & Banaszkiewicz, 2019; Luxton-Reilly et al., 2018; Scott, Sheard, Szabo, 2018). The difficulties inherent to learning programming languages are well documented, given the cognitive demands of reading, writing, and revising code (Qian & Lehman, 2017), as well as the need for students to effectively regulate their efforts while problem solving (Loksa & Ko, 2016). The rationale for this chapter is to contribute to our understanding of this phenomenon using Learning Analytics (LA) (Smith, 2022a, b).
Key Terms in this Chapter
Multimodal Data: Multimodal data refers to data that is composed of multiple modalities or types of information. Each modality represents a distinct aspect or channel of data, such as formative or summative assessments, whether recorded in the form of structured or unstructured textual data, video code replay, and so on. These typically include different skills being assessed, such as reading, writing, or revising code.
Learning Analytics: The collection, analysis, and interpretation of data generated from educational activities and digital learning environments to gain insights and inform decision-making. It involves applying data science techniques and statistical methods to understand and improve the learning process and outcomes.
Data Alignment: Data alignment refers to the process of arranging or synchronizing data elements in a structured manner so that they correspond or match with each other based on a specific criterion or relationship. It involves organizing and adjusting data to ensure consistency, coherence, and compatibility across different data sources or data sets, such as taking into consideration prerequisite knowledge or skill as well as when assessments are administered over time.