Previous fluency studies typically used small datasets to analyze L2 learners' oral performance with objective fluency measures, and found positive correlation between objective measures and subjective ratings. Findings from a small dataset are difficult to generalize. However, it takes a great deal of time and effort to build a large dataset with various measures. To help facilitate this process, CAF Calculator, which outputs 50 fluency measures, has been developed. In this chapter, CAF Calculator and a workflow to compute fluency measures are introduced along with a study investigating utterance fluency of L2 learners of Japanese at two proficiency levels performing two tasks. The study found significant differences in speed, breakdown, and composite fluency measures between the two groups in both tasks. It also found that task type affects pause locations. It is hoped that the research tools introduced in this chapter will encourage more research on fluency.
TopIntroduction
Foreign language learners regard speaking fluently as the most important skill to acquire (Harlow & Muyskens, 1994; Houston 2005; Rivera & Matsuzawa, 2007; Tse, 2000). Second Language (L2) instructors have, therefore, focused on improving learners’ communicative competence and have increasingly paid attention to measuring oral proficiency. The Oral Proficiency Interview (OPI) of ACTFL (The American Council on the Teaching of Foreign languages) is well-known in the U.S., but the test has practical difficulties. The test requires a time-consuming individual interview process conducted by a certified tester. The results of the test are broad categories, such as novice, intermediate, and advanced with three subcategories, low, mid, and high. It is, therefore, hard to see small amounts of growth in speaking proficiency. Typically, speaking tests administered at many institutions are subjectively rated based on local standards that instructors decide, and their inter-rater reliability may not be consistently high, which would be problematic in terms of fairness. It goes without saying that evaluating L2 learners’ oral proficiency on a fair basis as well as tracking their proficiency development in detail are important for both L2 research and language instruction.
Fluency research using objective measures (e.g., speech rate) started in the 80s and it attempted to measure L2 learners’ oral proficiency with objective measures of oral fluency. These objective measures are widely used in current second language acquisition research (e.g., de Jong & Mora, 2017; Freed, 1995; Ginther, Dimova, & Yang, 2010; Iwashita, Brown, McNamara, & O’Hagan, 2008; Tavakoli, 2016; Tavakoli, Campbell, & McCormack, 2016; Tavakoli, Nakatsuhara, & Hunter, 2017). Previous research has opened the door to the use of objective measures of oral proficiency as a supplemental tool to subjective rating. Significant positive correlation has been found consistently between objective measures of oral fluency and subjective ratings (Bosker, Pinget, Quene, Sanders, & de Jong, 2012; Freed, 1995; Ginther et al., 2010; Iwashita et al., 2008; Kormos & Denes, 2004; Prefontaine, Kormos & Johnson, 2015; Tavakoli et al., 2017; Towell, Hawkins, and Bazergui, 1996). Thus, objective fluency measures may be useful to include in the assessment of L2 learners’ oral production for better understanding of their oral performance and development.
However, such objective measures of fluency have not been widely used for assessments in educational settings due to the cumbersomeness in obtaining the measurements. Computing objective measures manually indeed requires large amounts of time and effort. In order to assist with this process, CAF1 Calculator2 (Fukada, Hirotani, & Masumoto, 2019), formerly known as Fluency Calculator (Fukada, Hirotani, Matsumoto, & Huston, 2015a; 2015b) has been developed. In this article, CAF Calculator and a workflow to compute fluency measures will be introduced to demonstrate easy access to the measures for L2 instructors and researchers. Along with the introduction of CAF Calculator, a study which examines oral samples using CAF Calculator will be described as an example. The findings of the study will be discussed to reveal characteristic of L2 learners’ oral performance.