Using Dr. Scratch as a Formative Feedback Tool to Assess Computational Thinking

Introduction

School systems around the world have been adopting and even requiring that computer science (CS) or computational thinking (CT) become part of their curriculum. In Europe, coding integration in the curriculum has seen a large adoption rate, “at the national, regional or local level” (Balanskat & Engelhardt, 2015, p. 9), including 16 countries with two more having plans to integrate coding into their core curriculum by 2020. CT and the fundamentals of coding are also starting to be introduced in K–12 schools throughout the United States (Rich, Bly, & Leatham, 2014; Elahi, 2016; Grover & Pea, 2013; K–12 Computer Science Framework, 2016; Repenning, Webb, & Ioannidou, 2010; Smith, 2016). With so many schools adopting coding at earlier ages, there is a need to better understand more and less effective methods to assess the computational ability of younger students.

While assessment in itself is important to measure progress and goal attainment, formative assessment can be used by learners themselves to measure the growth of their own CT ability over time and across projects. Educational research has demonstrated that feedback is an important and effective learning intervention. According to Hattie (2015), performance feedback measures are one of the most effective forms of intervention. Hattie’s research data is pulled from nearly 1200 meta-analyses and his list has grown to 195 influencers on student achievement. Feedback consistently ranks highly significant in Hattie’s meta analyses, currently ranking 15th largest in effect size on student achievement (ES = 0.73; Hattie, 2015; Visible Learning, 2016).

Feedback may be defined as: “the means by which the learner, or any other agent directing the learning process, ascertains whether or not progress is being made toward the end goal, and whether or not the goal has been reached” (Weibell, 2011, p. 361). The goal in measuring progress and providing feedback in CT can be looked at in two ways: (a) the completion of or progress toward solving the problem to which computational thinking processes are applied, or (b) the ability of the student/learner to apply CT processes correctly to any given task/problem. Feedback towards the goal of solving the problem can be given by teachers and peers, or even by the way the learner’s program interacts with the problem (e.g., student’s program does not solve or partially solves the given problem). Feedback towards the goal of applying CT processes correctly would most likely require a way to understand how the learner was thinking during the creation process. One way to do this would be to analyze the artifact(s) a student/learner creates and what CT processes would have been needed to create those artifacts.