Engaging Faculty in Examining the Validity of Locally Developed Performance-Based Assessments

Introduction

Some state departments of education and educator preparation providers (EPPs) in the United States are adopting the use of standardized performance-based assessments, such as edTPA. Other EPPs have discretion with using proprietary or EPP-created (i.e., locally developed) assessments. For example, the Council for the Accreditation of Educator Preparation (CAEP), a national accrediting agency, allows EPPs to use both proprietary and locally developed assessments as evidence of teacher candidate competency in relation to building a case to demonstrate that the EPP is meeting CAEP Standards. Proprietary instruments are “created and/or administered by states, research organizations, or commercial test organizations. Typically information about the design of the assessments, and their validation, scoring, and other attributes, is available” (CAEP, 2018, p. 115). With locally developed assessments, “EPPs take responsibility for design, administration and validation of these assessments” (CAEP, 2018, p. 109). Proprietary and locally developed performance-based assessment instruments offer a more relevant measure of students’ knowledge, skills, and dispositions; and thus, are more representative of real-world teaching compared to multiple choice tests and traditional types of assessments (Darling-Hammond, 2010). Since performance-based assessments are generally embedded into coursework, students are actively engaged in their own learning (Suskie, 2009). However, the development of effective performance-based assessment instruments that can consistently provide useful formative and summative feedback for students and program improvement is difficult.

CAEP also asks EPPs to provide evidence for the validity and consistency of data used in their accreditation self-study. Specifically, the CAEP Handbook Initial-Level Programs 2018 states, “for all EPP-created evidence measures, providers should demonstrate the quality of the data, including their validity and reliability in the context of the CAEP Standards” (p. 9). This relates to CAEP Standard 5.2, “the provider’s quality assurance system relies on relevant, cumulative and actionable measures, and produces empirical evidence that interpretations of data are valid and consistent” (CAEP, 2018, p. 22). CAEP reviews locally developed assessments using the CAEP Evaluation Framework for EPP-Created Assessments (2017).

As performance-based assessments become more prevalent in higher education, challenges have emerged. Faculty are not always familiar with procedures for writing learning outcomes, aligning outcomes through a curriculum map and assessment plan, and developing high quality performance-based assessments. Faculty often question the reliability and validity of assessment results. They also may be unsure of how to use results for program improvement. Additionally, faculty may oppose accountability efforts dictated by administration or external entities, or they do not believe they have the time to prioritize assessment responsibilities in their overall workload (Ewell, 2002; Hutchings, 2011). However, well-designed performance-based assessments provide students with a “transparent” understanding of the purpose, the alignment to outcomes or standards, the expected tasks, and the criteria established for rating of the responses (Winkelmes et al., 2015, p. 5). Green and Hutchings (2018) propose the need to better connect assessment with teaching and learning. These connections can be accomplished through “conversations, collaborations, and habits that support ongoing improvements” (p. 39). This chapter describes a facilitated process and provides specific guidance to support teacher education faculty in this work.