formularioHidden
formularioRDF
Login

Sign up

 

Learning Designs and Learning Analytics

InProceedings

The teaching demands and complexities evolving from recent government and university reforms, linked to quality assurance and accountability, emphasise the importance of developing replicable, scalable design and evaluative approaches. In this context learning design and learning analytics have emerged as two fields of research that can begin to address ongoing educational issues related to professional development, student and teacher evaluations and the development of personalized learning experiences. Learning designs are ways of describing an educational experience such that it can be applied across a range of disciplinary contexts. Learning analytics offers new approaches to investigating the data associated with a learner’s experience. This ‘ideas and innovations’ paper explores the relationship between learning designs and learning analytics.

"1 Introduction. Across the Higher Education sector internationally, there is mounting interest in developing replicable and scalable exemplars and templates of effective teaching practice. This in part has arisen from increased accountability and quality assurance measures associated with the administration of education practice and its associated impact on the demonstration of teaching quality. Regardless of the drivers, the field of learning design has received much attention in the education sphere, largely due to its perceived capacity for addressing these concerns. Learning designs, or pedagogical models or patterns, are captured representations of teaching and learning practice that can act as a model or template replicable across alternate educational contexts. In essence, these learning design articulations begin to initiate and frame the intent of and process for the pedagogical experience. Learning design research and its up-take in educational practice has come in many forms. At the basis are ways of describing an educational activity (regardless of scope) in such a way that the activity can be applied across a range of disciplinary contexts. Essentially, the focus here is on the design, planning and, to some extent, implementation aspects of the educational experience. In practice, learning designs are predominately directed at the designer/teacher of the educational experience in order to provide accessible templates for developing learning actions. As much as learning design has tended to focus on the teacher for the initial design and implementation process, learning analytics largely centers on the users (learners) interactions with the developed learning artifacts and activities. In essence learning analytics seeks to use the data associated with a learner’s interactions to make pedagogically informed decisions and evaluations. This emerging interdisciplinary field draws on and integrates research related to, for example: data mining, social networks, data visualisation, machine learning, semantics, e-learning and educational theory and practice. Seemingly, learning analytics takes up where learning design finishes in the educational experience continuum – implementation and outcomes. This ‘ideas and innovations’ paper explores the relationship between learning design and learning analytic approaches. The paper considers the context in which learning design research emerged, and highlights how it has developed - with a focus on the work in Australia. It also considers work in social network analysis as an example subset of learning analytics to demonstrate how this field complements learning design. 2 Background. Although the terminology and the scope of research and application of both learning design and learning analytics approaches have evolved in recent years, their underlying basis is not new. Essentially these research fields are concerned with documenting and reporting on educational practice and experience. The interest in both learning designs and learning analytics has gained momentum within the context of two compatible worldwide trends within the higher education environment – calls for improved teaching and increasing use of technology. In the early 90s, commentators and policymakers began to argue for increased expectations of the quality of the teaching that was delivered in universities, and accountability for that quality[1]. This was met, in many countries around the world, with a wave of professional development programs aimed to improve the quality of teaching and funding for ‘innovation’ in teaching (such as the Australian Learning and Teaching Council, Higher Education Funding Council for England). Soon after this focus on quality teaching began, the increasingly accessible and user-friendly Web provided universities with an opportunity to expand their student markets and offer flexible forms of programs. There was increasing expectations for, and interest among, university teachers to integrate technology within their teaching practice. From those early days of the Web, research and teaching scholarship explored the opportunities that technology provided education – particularly in terms of improving pedagogy [2]. The increasing emphasis on and expectations towards technology integration placed added complexity to the teaching environment – its design, implementation and evaluation process. As a result, professional development and teaching innovation funding became, in the main, directed toward improving teaching using technology. Underlying the support for these projects was the premise that the outcomes, resources and products would be disseminated amongst the higher education community and adopted at the institutional and individual teacher levels. Developments in learning designs exemplify these principles of resource development, sharing and reuse. Research in Learning Designs. In Australia, learning designs first emerged through an Australian Universities Teaching Committee (a predecessor to the current the Australian Learning and Teaching Council) project, funded in 2000, entitled Information and Communication Technologies and Their Role in Flexible Learning. The project established a framework for evaluating high quality learning in higher education [3], identified and evaluated educational examples against the framework, documented these examples and generic versions as resources, and disseminated these resources through the project website (http://www.learningdesigns.uow.edu.au/). These initial resources were termed “learning designs” and comprised a visual representation with supporting textual descriptions (see Figure 1 for example) [4]. Figure 1: Example of a representation of a learning design from http://learningdesigns.uow.edu.au/exemplars/info/LD1/more/03Context.html Concurrently, around the world, researchers have been interested in the educational and technical issues associated with how teachers and instructional designers could create document, share, adapt and implement, educational design ideas. This involved developing a technical language (IMS LD [5]), developing tools in which learning designs could be created and/or used [6] [7], and investigating how teachers interpret and use different learning design representations [8, 9]. The generic and contextual learning designs developed through the Australian project have been reported to be used by staff in educational development units and university teachers as resources to support the process for designing courses as well reflecting on the implementation once a course has been delivered [10]. Studies exploring the utility of the learning design approach have demonstrated positive results for a framework to integrate learning objects [11], usefulness in interpretation and application of representations by teachers [12] and as a tool that that allows extraction of teaching and learning processes for analysis [13]. 3 The Potential of Learning Analytics. The underlying driver associated with teaching quality assurances and processes is that it will lead to a better learning experience for students and thus, improved learning outcomes. While learning designs may provide theoretical, practice-based, and/or evidence-based examples of sound educational design, learning analytics allow us to test those assumptions with actual student interaction data in lieu of self-report measures such as post hoc surveys. In particular, learning analytics provides us with the necessary data and tools to support the accountability that has been called for in higher education. While learning designs facilitate the documentation of the original pedagogical intent learning analytics assists in measuring the effectiveness of the design approach in achieving the desired outcomes As with learning designs, learning analytics takes a range of forms and foci. One such sub-set of the learning analytics domain is social network analysis (SNA). SNA provides a way to understand the educational experience and outcomes for learners engaged in online communication activities. Much of the aforementioned take-up of technology in higher education has been the associated with the the discussion forum – particularly within learning management system course sites [14, 15] (. There are countless examples in the literature that describe the design and implementation of educational programs that use discussion forums. Where these cases are aligned with a theoretical premise, it is of the social cognitive aspect of learning and the benefits of collaboration (e.g., [16]). Thus it follows that, social network analysis provides a methodology for teachers to begin to understand the impact of their implemented learning design on the student learning experience and outcomes. However, this analysis needs to be readily accessible and interpretable for all educators regardless of expertise in network analyses; information and communication technologies or educational theory. To address this issue of user uptake and interpretation, Dawson [17] and colleagues developed an analytical tool named Social Networks Adapting Pedagogical Practice (SNAPP – available at http://research.uow.edu.au/learningnetworks/seeing/snapp/index.html). SNAPP seamlessly integrates with learning management system discussion forums to extract data and provide teachers with real-time visualizations of the discussion forum activity (see figure 2). Recent evaluations of the use of SNAPP by university teachers demonstrated that the tool was seen to be particularly effective for retrospective analysis of an education implementation after a course was completed [18]. However, despite the ease in generating ongoing visualizations of student discussion interactions and relationship development, teachers did not take the opportunity to analyze data during actual course implementation. This process would have allowed for modification of the learning design ‘on the fly’ if the intended experience or outcomes were not being realized. While this learning analytics tool theoretically supports the implementation stage of that educational continuum - to date it has been largely used as a reflective tool. Further research is required to investigate how educators can be better prompted when student interactions indicate a deviation from the intended learning outcomes. For example, an educator observes minimal studentto-student interactions arising despite the implementation of learning activities designed to establish a learning community. Figure 2: SNAPP generated sociogram of the student network relationships evolving from an online discussion forum (student names have been removed). 4 The Challenge. Essentially, the learning design approach is founded on a case-based reasoning premise. A documented learning design provides a case that is abstract enough such that teachers can imagine how it might be applied to their own context. However, investigation of the use of learning designs suggests that teachers also benefit from understanding the context of others and the environments in which they teach [12] and thus cases should not be so abstract as to lose a connection with the original context [19]. Regardless of the level of abstraction, the notion is that teachers learn or refine their ideas about design from the examples of other teachers. The dissemination and adoption of innovative teaching ideas in higher education – whether those ideas are expressed through learning design representations or other types of tools or resources, recognizes the social processes of being a teacher. University teachers report that their ideas about teaching come from interacting with colleagues and other teachers [20, 21]. In this context, teachers are less likely to seek information from external resources or online sources [20]. This research (and that of others investigating educational change) frequently highlights time as a barrier to the adoption of new ideas into ones teaching practice. Academics in higher education institutions are stretched across their teaching and research responsibilities. They are more likely to invest time to try new ideas if they are convinced, through evidence, that the innovation will have an impact for them and their learners that is both time and workload efficient. Learning analytics has the potential to provide this evidence. The research challenge is identifying effective and efficient ways to make learning design and learning analytics useful and relevant for teachers. The integration of research related to both learning design and learning analytics provides the necessary contextual overlay to better understand observed student behaviour and provide the necessary pedagogical recommendations where learning behavior deviates from pedagogical intention. Acknowledgments. The authors wishes to acknowledge collaborators in Learning Design research, Dr Shirley Agostinho, Associate Professor Sue Bennett, and Emeritus Professor Barry Harper."

About this resource...

Visits 157

0 comments

Do you want to comment? Sign up or Sign in