Abstract
This chapter presents the results of a preliminary study which consisted of (1) the design, and (2) preliminary testing of a series of automated assessment tools, which were used in a virtual course of English for Tourism, level B2, to promote students' learning autonomy and self-evaluation. These rubrics were used to assess two activities: a writing task, where students had to create an audio description script of a touristic video, and an oral task, where students had to record themselves and produce their own audio described sequence of the selected clip. Then, for each task they had to assess themselves by filling in two corresponding rubrics. These rubrics were also used by the tutors to assess the same tasks. After that, the results of the rubrics were compared. The preliminary results show that the rubrics helped students obtain better feedback and develop a more accurate sense of their own progress.
TopIntroduction
In the past decades rubrics have become popular in education as an effective means to assess students’ learning and progress. In this sense, Jönsson and Svingby (2007) noticed that rubrics have the potential of promoting learning, and of improving instruction, because students explicitly know what is expected from them. Also, they provide detailed feedback, and can be a very good tool for self-assessment. However, they also noted that it was not possible to draw any conclusions about student improvement since the results were mixed. Indeed, during the first decade of the XXIst century there was a general shift from traditional assessment to assessment for learning (Dochy et al., 2006), which had already started with Brown et al. (1997) and Dochy et al.’s (1999) -among others- proposals for more specific and detailed higher education students’ evaluation. However, the use of rubrics was welcomed with reasonable caution and reservations, as Kohn (2006, p. 12) remarked:
Consistent and uniform standards are admirable, and maybe even workable, when we’re talking about, say, the manufacture of DVD players. The process of trying to gauge children’s understanding of ideas is a very different matter, however. It necessarily entails the exercise of human judgment, which is an imprecise, subjective affair. Rubrics are, above all, a tool to promote standardization, to turn teachers into grading machines or at least allow them to pretend that what they are doing is exact and objective.
In this sense, also Panadero and Jönsson (2013) call for caution by means of a meta-analysis of 21 studies on the use of rubrics for formative assessment. They state that, even if scoring rubrics are assumed to be an effective tool for the summative aspect of assessment, the use of rubrics for formative purposes is yet to be fully proved. Again, they found out that although rubrics may have the potential to influence students learning positively, there are factors that may moderate the effects of using rubrics formatively (metacognitive activities, intervention length & gender) as well as factors that need further investigation (effects of personal differences): “…we do not know how the use of rubrics may facilitate in improving student performance or which factors are important in moderating the potential effect” (Panadero & Jönsson, 2013, p. 130).
Despite the initial reticence on the use of rubrics in both summative and formative assessment, this shift towards standardized, explicit and detailed criteria of evaluation was especially noticeable in second language learning studies, where rubrics were also suggested in the Common European Framework of References for Languages (CEFRL, 2001) under the term descriptors, and which in the companion volume (CEFRL, 2018, p. 1) are further exhaustively elaborated:
The Common European Framework of Reference for Languages: Learning, teaching, assessment – Companion volume broadens the scope of language education, reflecting academic and societal developments since the CEFR publication in 2001. It presents the key aspects of the CEFR for teaching and learning in a user-friendly form and contains the complete set of extended CEFR descriptors, replacing the 2001 set. These now include descriptors for mediation, online interaction, plurilingual/pluricultural competence, and sign language competences. The illustrative descriptors have been adapted with modality-inclusive formulations for sign languages and all descriptors are now gender-neutral.
The companion volume includes specific grids for self-assessment on the different competences, which now also focus on mediation and interaction. With all this in mind, in the present study our main aim was to see whether rubrics actually contribute to an effective formative evaluation of Spanish students of English for Specific Purposes (more specifically, English for Tourism) in an online course, as well as to a better self-assessment of their writing and oral skills, combined with mediation skills.
Key Terms in this Chapter
English for Tourism: English for Tourism is one type of English for Specific Purposes, which focus on the specialized domain of tourism.
Task: In the language learning setting, a task is a communicative activity which emulated an activity that can take place in the real world. A task can take very different formats and can be developed in combination with several other strategies (a) focus-on-form tasks; (b) focus-on-meaning tasks, leading to communication, and the most used in communicative approaches, as well as in this study.
MALL (Mobile Assisted Language Learning): Mobile learning is a continuation of e-learning, of learning with computers, something that took in the past century place only in schools, colleges and universities and in corporations, institutions with the expertise and resources necessary for working with scarce, expensive, fragile and difficult devices, to enhance and extend the existing curriculum. Nowadays, MALL is a continuation of CALL. It consists of learning languages with the aid of any mobile device, such as a tablet or a smart phone. In some cases, even mini portable computers are considered mobile devices, although we prefer to focus on actually mobile devices with a different operating system (Android or Apple), since the use of apps is the most distinguishing feature of this type of learning.
Formative Evaluation: Formative evaluation is a kind of evaluation that rate of progress of students during the language course. Therefore, it can also be called continuous evaluation , because it takes place during the learning process, to help students check their progress and correct or improve the areas that need further work. It also helps students to work during the course and prevents them from leaving all efforts of study for the summative evaluation, which is normally the final test or exam.
Audio Description: Intersemiotic translation activity which consists of translating images into words by introducing descriptions of what is happening on the screen in the gaps with no sound or music. It is intended to make audiovisual products accessible to the visually impaired or challenged audiences.
CALL (Computer Assisted Language Learning): Levy (1997) AU25: The in-text citation "Levy (1997)" is not in the reference list. Please correct the citation, add the reference to the list, or delete the citation. defined computer assisted language learning (CALL) as a way of using computers to teach and learn languages. Nowadays CALL offers a great opportunity to develop digital literacy skills as the ones proposed by Dudeney (2014) AU26: The in-text citation "Dudeney (2014)" is not in the reference list. Please correct the citation, add the reference to the list, or delete the citation. : a. Critical Thinking & Problem Solving, b. Collaboration & Communication, c. Creativity & Imagination, d. Citizenship, e. Student Leadership & Personal Development. Computers are used in combination with language learning, such as looking for possibilities to use the language in real situations and thus be better prepared for the forthcoming real experiences.
Rubric: In language learning, or in the educational setting in general, a rubric is set of standardized instructions or items that allow to assess or evaluate different learners’ outputs (such as a task or assignment) in a detailed, pedagogical and standardized way.
Self-Assessment: Self-assessment is a way of assessment in which students observe their own learning outputs, and we believe it can help them identify when they need help and when they can learn independently. Self-assessment is part of an active learning process, making students engage with learning and achieve their goals.