Designing Learning Analytics Tools for Teachers with Teachers - A Design-Based Research Study in a Blended Higher Education Context

Abstract

The overall aim of this doctoral thesis is to explore how Learning analytics (LA) uptake by and relevance to teachers can be improved by their engagement in the planning, design, and implementation of theoretically grounded LA tools and interventions in authentic teaching practices. The questions addressed in this thesis are as follows: What is the current practice of learning design among teachers, and what is the state of teachers’ awareness, acceptance, needs, challenges, and beliefs regarding applying LA to support their learning design decisions? What is the potential for different forms of analytics to provide insights into students’ learning that teachers can then use to make pedagogical decisions? How can we design and implement empirical and theoretically based LA tools together with teachers to meet teachers’ pedagogical expectations?

Article 1 explores the challenges facing teachers’ adoption of LA and the existing strategies to overcome them. The findings show that teachers struggle to make sense of analytics, and analytics systems are usually atheoretical and not aligned with their pedagogical practice.

Article 2 explores teachers’ course design practices and their perceptions of LA as a potential tool for supporting their practice. The findings show that a number of factors, including situational factors, summative and formative assessments, and teachers’ intuition and experience, underpins teachers’ course design practices. Teachers also appreciate the formative and normative value of LA in providing more objective evidence of students’ learning patterns and shaping learning processes. The findings from this study were synthesized to form a bi-directional LA course design conceptual framework.

Article 3 examines the potential of different forms of analytics (checklist and process analytics) to support teachers’ learning design decisions. The findings show that if shared in a simple and timely manner and integrated within the same teaching environment, LA visualizations can provide insights into students’ online learning processes, which teachers can use to make learning design changes. Moreover, v the different data sources and analytical techniques used in this article show how researchers can move from low levels of abstraction (such as counting page views) toward higher-level constructs (such as identifying discourse patterns).

Article 4 is an intervention in seven courses using the Canvas Analytics Dashboard (CADA), which was designed by the researcher together with the teachers. The findings reveal that engaging teachers in the design of and giving them control over LA tools can favor teachers’ adoption of LA in their everyday practice. The findings further show that teachers are able to make timely learning design changes based on the insights they gain from the dashboard.