Abstract
In the field of Education, computerized data systems are used to manage, retrieve, and analyze information. Educators view this information in the form of data reports, which educators use to inform decisions that impact students. These decisions are frequently undermined by misunderstandings concerning the data and its implications. Yet data systems and their reports typically display data without any guidance concerning the data's proper analysis. In a quantitative study, medicine labeling conventions were applied to data systems to embed guidance in the proper use of contents. Among 211 educators of varied backgrounds and roles, data analyses were found to be 307% more accurate when a report label/footer was present, 205% more accurate when a 1-page reference sheet was present, and 273% more accurate when a reference guide was present. Findings hold implications for those who provide or use tools for high-stakes information retrieval, analysis, and/or management, particularly in Education.
TopBackground
Given the many complexities of education data, particularly assessment data, educators’ data-informed decisions are frequently undermined by misunderstandings concerning the data and its implications. Yet data systems and their reports typically display data without any guidance concerning the data’s proper analysis. Pharmaceutical label conventions can result in improved understanding on non-medication products, as well (Hampton, 2007; Qin et al., 2011). Thus, in the way over-the-counter medicine’s proper use is communicated with a thorough label and sometimes with added documentation, the researcher hypothesized a data system used to analyze student performance could include components to help users better comprehend the data it contains.
A data system, also referred to in education as a student data system, is software that provides student data to educators in a digestible, report-based format (Wayman, 2005), and educators use data systems to make decisions that impact students (VanWinkle, Vezzu, & Zapata-Rivera, 2011). No or poor medication labels have resulted in many tragic errors (Brown-Brumfield & DeLeon, 2010), and the researcher hypothesized that data systems’ lack of proper labeling for educators was similarly contributing to poor data understanding and use. Feedback is one of the most powerful influences on student learning and achievement, but this impact can be negative if the performance feedback is not provided in the best way (Hattie & Timperley, 2007).
Key Terms in this Chapter
Over-the-Counter Data: Data that is accompanied by embedded usage guidance just as over-the-counter products are accompanied by embedded usage guidance. Data systems and reports can feature over-the-counter data by adhering to Over-the-Counter Data Standards ( www.overthecounterdata.com/s/OTCDStandards.pdf AU63: The URL http://www.overthecounterdata.com/s/OTCDStandards.pdf has been redirected to http://overthecounterdata.com/s/OTCDStandards.pdf. Please verify the URL. ) when reporting data to educators and other education stakeholders in order to improve their understanding and use of the data. The standards involve implementation of effective labels, supplemental documentation, a help system, package/display, and content.
Help System: A computer-based, online collection of task-based lessons that walk data system users through sequential steps to accomplish tasks within the data system, as well as topic-based lessons that help users understand and use the data housed in the data system.
Footer: A means for labeling a data report with data usage guidance, involving brief (such as one to three lines of) text at the bottom of a data report.
Professional Development (PD): Educators’ recommended continual process of learning, which can involve a range of improvement strategies such as online networks, embedded videos, on-the-job training, traditional workshops, weekly collaboration sessions, and more.
Reference Sheet: Accompanies each report to help users understand that specific report’s data. It is often called an abstract or summary. This single page contains the report’s title, description, image, focus (content reported), and warning (vital, cautionary information an educator would need to avoid the most common analysis errors made when analyzing the particular data being displayed). It can also communicate the report’s purpose (key questions the report will help answer) and additional focus information (intended audience, and format in which data is reported).
Data-Informed Decision-Making: The process of using data to guide decisions. In the field of Education, these decisions ultimately impact students and often impact other stakeholders, such as teachers and administrators. While data- driven decision-making is a more common term, data- informed decision-making is a preferable term since decisions should not be based solely on quantitative data ( Knapp, Swinnerton, Copland, & Monpas-Hubar, 2006 ; USDEOPEPD, 2009 ).
Reference Guide: Accompanies each report to help users understand and use that specific report’s data. It is often called a data guide, interpretation guide, or interpretive guide. The report’s reference sheet (described below) functions as the guide’s first page, and subsequent pages contain the report’s instructions (how to read the report), essential questions (showing the user where to look on this report – and what to look for – to answer each question listed in the purpose area of the guide’s 1st page), and a “more info” section (offering where to get additional information on related topics).
Report: An arrangement of data around a particular topic and purpose in order to communicate the data to a particular audience through graphs/graphics, tables, text, and/or other means. Reports can take online forms such as data dashboards or can appear as traditional printed pages.
User: Any person using a data system. In the field of Education, this is likely an educator (including classified staff), but in some cases can also be a student, parent, or other stakeholder.
Data System (within the Context of Education): A computerized system, often accessible online, that houses and displays student, educator, and school information and allows users to retrieve, manage, and analyze the data. Data systems go by many names and there is much overlap in terms of what different systems do. Examples include the assessment system, data and assessment management system, data mart, data warehouse, decision support system (DSS), information system, instructional management system (IMS), learning management system (LMS), special education system (SES), student information system (SIS), and other educational technology product types that contains a significant feedback component.