Using Written Responses to Reflection Questions to Improve Online Student Retention: A Text Analysis Approach

Introduction

Online learning has been growing in both popularity and practice over the past thirty years, with a drastic acceleration since the onset of the COVID-19 pandemic (Ginder, Kelly-Reid, & Mann, 2019; National Student Clearinghouse Research Center, 2022; Seaman, Allen, & Seaman, 2018; Xu & Xu, 2019).While online education has traditionally been viewed as an alternative pathway for nontraditional students seeking higher education opportunities, the emergence of the COVID-19 pandemic has forced students and educators across all levels of education to rapidly adapt to online learning. The pandemic has created the largest disruption of education systems in history, affecting nearly 1.6 billion learners in more than 190 countries, forcing K-20 education to shift to online learning (United Nations, 2020). Today, learners engage in online or blended learning in primary, secondary, post-secondary, workforce training, and workplace learning like never before in modern history.

Concomitant with the boom in online learning are escalating concerns about academic accountability, specifically with respect to student outcomes as measured by persistence and success. These concerns emerge from research indicating that dropout rates in online courses are significantly higher than in traditional brick-and-mortar learning environments (Soffer & Cohen, 2019). A 2019 report by Xu and Xu found that online students are up to 15 percent more likely to drop out than are students in face-to-face classes at community colleges in North America (Xu & Xu, 2019). This finding is very problematic, as lower retention rates among online students have been connected to overall lack of academic success in higher education (Xu & Jaggars, 2011). Xu and Jaggars’ study points to the importance of online course success, particularly in view of evidence suggesting that failing online college courses early in one’s college career may impede progression towards graduation (Xu & Jaggars, 2011).

While online learning is growing in popularity, many students struggle in such settings, especially students lacking certain personal attributes - such as being goal-oriented and self-disciplined - that are essential for online success. Learners need to be able to navigate, engage, and persist in such environments with more autonomy and self-directedness than was otherwise needed in traditional brick-and-mortar learning environments. This, in turn, requires learners to adopt, practice, and internalize self-regulated learning strategies. Students who self-regulate are more likely to improve their academic performance, find value in their learning process, and continue to be effective lifelong learners (Zimmerman, 2002). However, online students often struggle to self-regulate, which may contribute to lower academic performance. It is important, therefore, to be able to identify early in the course students who lack self-regulation skills in order to improve retention and graduation.

A growing body of research is seeking to respond to this challenge by employing learning analytics techniques to analyze student online learning behavior, log files, and clickstream data (Cohen et al., 2019). However, research examining other types of analytics (e.g., text data mining, text analysis, etc.) is scarce. Therefore, the purpose of this study is to employ text analysis techniques to analyze student reflective responses to a set of skill-building videos incorporated into online courses. Specifically, this study aims to examine whether persistence and performance of online Latin American students may be predicted by evidence of significant learning found in student written responses to post-video reflection questions.