Article Preview
TopIntroduction
The world's current population of about 7 billion speaks almost 7,000 languages (Lewis, 2009). Language barriers often cause inconvenience but when medical issues are involved they cease to be mere inconvenience and can become life-threatening. Anyone who has had the misfortune to fall ill in a country where they do not speak the local language will be painfully aware of this from their own experience. Moreover, quantitative studies have shown that limited English proficiency correlates with a vastly increased probability of negative outcomes across a variety of objective indicators: poor access to medical care (Brach & Chevarley, 2008; DuBard & Gizlice, 2008; Pippins, Alegría, & Haas, 2007), excess hospitalizations, medical errors, and drug complications (Bard et al., 2004; Flores, 2005; Hampers, Cha, Gutglass, Binns, & Krug, 1999; Waxman & Levitt, 2000), and poor satisfaction with the provided care (Flores, 2005).
Unfortunately, trained medical translators are both scarce and expensive. Even if an interpreter is available, health-care practitioner expressed dissatisfaction with both their own methods of working with them and with the interpreter's qualifications (Kale, 2010). Although a universal speech-to-speech translator still seems an insurmountable problem, the substantial gap between the need for and availability of language services in health care could be bridged through effective medical speech translation systems (Bouillon et al., 2007; Fluential, 2012); such a system would be far more useful to users if it was available on a hand-held device, as exemplified in (Figure 1). Indeed, different systems already moved towards the deployment of mobile speech-to-speech translation applications in academia (Tsourakis, Bouillon, & Rayner, 2009; Zhang & Vogel, 2008), and commercially (Google, 2012; Prasad et al., 2011; Voxtec, 2012).
Figure 1. Doctor interacting through the mobile device
During the development lifecycle of these systems authors have provided evaluation results leveraging various computer and human centered metrics. Despite some early efforts towards a common evaluation framework (Rayner et al., 2008) we argue that there is a lack of such a methodology that would provide a fair comparison framework for different mobile medical translation systems. Additionally, the lack of appropriate quality assessment techniques can deteriorate user satisfaction. In the US for example, user dissatisfaction even caused legislative actions governing software quality and suppliers' responsibilities (Kaner, 2000). The purpose of this paper is to provide a common evaluation methodology in order to compare mobile medical speech translators and also to help developers focus on those aspects of quality that users deem important.