"1. Introduction. According to George Siemens, learning analytics “is the use of intelligent data, learner-produced data, and analysis models to discover information and social connections, and to predict and advise on learning (1).” (1) http://www.elearnspace.org/blog/2010/08/25/what-are-learning-analytics The LAK 2011 conference call for papers defines learning analytics as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” In this paper, we present our vision of leveraging learning analytics tools and techniques to support teachers’ dynamic diagnostic pedagogical decision-making in actual K-12 classroom settings. Our vision seeks to extend the current state-of-the-art in learning analytics in at least four directions, to apply learning analytics in the primary and secondary education formal classroom settings compared to tertiary education settings, focus on real-time use of learning analytics by teachers for technology enhanced formative assessment, apply an extended version of the pair analytics method in visual analytics, and finally, to review and build on current work in the learning sciences and the method of design-based research. The primary contribution of our paper is the presentation of the preliminary triadic model of teaching analytics (TMTA). The remainder of this paper is organized as follows. In section 2, we briefly review two strands of research on analyzing learning data from computer supported collaborative learning (CSCL) and higher education. In section 3, based on the Next Generation Teaching Education and Learning for Life (NEXT-TELL) European Union integrating project proposal, we present the new demands faced by teachers in classrooms of the 21st century. Section 4 introduces the concept of teaching analytics and presents the preliminary triadic model of teaching analytics (TMTA). In section 5, we conclude the paper with the identification of several challenges and directions for future work. 2. Related Work. We present below two selective reviews of recent empirical work in learning analytics from computer supported collaborative learning (CSCL) and the Learning Analytics in higher education. 2.1 Computer Supported Collaborative Learning (CSCL). Various researchers [33, 34] in CSCL have considered the role of ""productive multivocality"" in the analysis of collaborative learning2. “Multivocality” refers to fact that CSCL researchers take diverse theoretical, methodological, and analytical approaches to the empirical study of how technology enhanced interaction supports learning processes and leads to learning outcomes. Multivocality can either be a source of strength (in the diversity of perspectives on a complex phenomenon such as learning) or a symptom of weakness (incoherency, divergence of empirical findings, incommensurability of perspectives and so on). “Productive multivocality” can be achieved only “if the “voices” share sufficient objects to reach some degree of coherence in the discourse of the field3.” Through a series of workshops, a group of CSCL scholars have sought to bring together different researchers, who brought with them a variety of data sets and analytic tools and approaches. Part of the motivation in doing so was to determine the degree to which there was commonality to support dialog between the various players and reach some degree of coherence in their discourse. (2) http://engaged.hnlc.org/story_comments/list/13 (3) CSCL 2009 Workshop: Common Objects for Productive Multivocality in Analysis http://engaged.hnlc.org/story_comments/list/13 The workshop participants were asked to consider analytic efforts across five dimensions: purpose of analysis, unit of interaction, data and analytic representations used, analytic manipulations, and theoretical orientation. Suthers et al. [32] have extended some of these ideas further by developing what they call an ""uptake analysis framework"" to help conceptualize, represent, visualize, analyze and interpret distributed interactions. However, in CSCL, one side effect of the symmetrical socio-technical configurations of students, equitable division of labor, shared conception of the problem, and shared task goals is the displacement of the teacher from the analytical center and a delimitation of the teacher’s role to that of a facilitator at worst and a curriculum designer/learning architect at best. Moreover, there exists a gulf of relevance between the emerging results of learning analytics work in CSCL and the professional practice of teachers. Creating solutions and generating implications for the professional practice of teachers has been a topic of interest and importance within CSCL [e.g., 21, 22, 26] and we seek to re-engage with that. 2.2 Learning Analytics in Higher Education. Networked learning analytics were first proposed for Learning Management Systems (LMS) such as Blackboard and Moodle with the objective of collecting data from learners in a non-intrusive, unobtrusive and automatic ways in order to trace the trajectory of the learning process and for appraisal and assessment of the effectiveness of online and blended courses [27]. Emerging empirical results indicate that Learning Analytics can help predict student performances with respect to learning across a variety of courses and academic programs in higher education. The use of academic analytics generated actionable intelligence for designing early interventions for freshman students at-risk of not returning for the sophomore year at the University of Alabama from 1999-2001 [7]. Another example is the Signals program [2] at the Purdue university mined institutional data from campus IT systems, analyzed the collected data, identified atrisk students and generated actionable information for designing educational interventions. Results show a significant improvement of student learning performance and subjective satisfaction [2]. Prior findings also show that monitoring and predicting the key performance indicators (KPI) of students with the help of learning analytics can help in designing, tailoring and targeting highly effective student interventions [10, 14]. Further, current results show the benefits of using learning analytics for performance monitoring and outcomes prediction for student populations in general at higher education institutes beyond the at-risk student segment [11, 23, 30, 42, 44] 2.3 Summary and Critique. An overarching observation is that the voice of the researchers and administrators in many of these approaches and studies comes through loud and clear. What is less prominent is the voice of the teacher or practitioner. We have evidence that the voice of the teacher can be very powerful when it comes to learning analytics. Some studies [35] have suggested that the sorts of detailed information that have typified analytic feedback have been useful to researchers, a more intuitive, user-friendly, and visually sophisticated representation is more powerful for use by teachers for just-in-time assessment. Knowledge building systems, with formative assessment, can be conceptualized as a cybernetic system with feedback loops serving to drive the system in new directions [28]. To optimize performance, feedback must be relevant and timely. Analysis of discourse from computer-supported collaborative learning environments is common but, as noted in [20], relatively little attention has been paid to the “formative, embedded, and transformative aspects of assessment in collaborative inquiry.” We offer two scenarios based on real anecdotes suggesting new ways in which teachers, researchers, and analysts can interact to support rapid feedback. 2.3.1 Scenario #1. Students engaged in online knowledge building often appear to be collaborating but the extent to which they are doing so is not often apparent. Are students really working together to build knowledge? What evidence can we garner that that is happening? One fourth-grade teacher was facing exactly those questions, and she was able to use a graphical social network analysis tool to show the sociograms that resulted from looking at who was interacting with whom in the online database. She used this tool to help her understand the extent to which students were interacting. At one point, a group of teachers from another school district visited her classroom and posed similar questions. She immediately started the social network analysis tool, and showed the visitors what she thought were unimpressive results: the data showed that all students were interacting. Of course, the visitors were anything but unimpressed. They were stunned by four things: that the students were interacting to such an extent, that the data to support such a claim were readily available; that the tools existed to provide simple representations of complex phenomena, and that she was able to use and demonstrate the tool so effectively. 2.3.2 Scenario #2. An experienced teacher was working with her 10-12 year old students on a module about electricity. The students were very engaged and had spent considerable time working through interesting problems. They had contributed a considerable number of notes to the online database that they used to track their inquiries and the unit had already gone on for several weeks. But were they covering the mandated curriculum topics? How could she obtain objective verification that her students had covered the curriculum even if she believed they had? A visual analytics expert had devised a tool that allowed a user to visualize the degree to which the curriculum had been covered. By literally lining up the curricular expectations on one side of the screen and the students' traces on the other side and examining the links between them the visual analytics expert was able not only to reassure her that her students were well on track, but to also allow her to see the few remaining curricular expectations that needed to be covered. The teachers' feedback on the visualization led the visual analytics expert to improve the visualization tool to make the same sorts of comparisons easier in the future. Though these may seem far-fetched or perhaps, unique scenarios, we argue that they are both representative of learning and teaching situations encountered in formal learning settings. Particularly, when we consider the new demands being made on teachers in the 21st century classroom. 3. NEXT-TELL: New Demands on Teachers in the 21st Century Classrooms According to Peter Reimann and colleagues of the Next Generation Education, Teaching and Learning for Life (NEXT-TELL)4 integrating project recently funded under the European Commission’s Seventh Framework Programme, the following are the new demands that teachers face in the 21st century classrooms (NEXT-TELL Consortium, 2010). Develop 21st Century competencies in addition to subject-matter specific Knowledge, Skills and Attitudes (KSAs)5 Personalize learning by planning lessons and learning activities for the individual student6 Teach adaptively in the classroom, making good use of ICT [9, 25] Provide evidence-based accounts for selected learning activities and assessments Be accountable towards stakeholders (students, parents, policy makers). As the NEXT-TELL project consortium says: In order to deal with these demands, teachers need to rapidly capture an ever-increasing amount of information about students’ learning, interpret this diverse body of information in the light of students’ development, appraise it in light of curricular goals, and make reasoned decisions about next learning steps. However, in comparison with most other professionals from whom clients expect rapid decisions in a dynamically changing environment, presently teachers often do not get the information they need for decision making in a timely fashion and in an 'actionable' format. This is particularly a challenge in technology-rich settings (the school computer lab, the laptop classroom) with high content and communicative density, where students engage with learning software and tools that teachers can only partially follow at any point in time. However, as technology increasingly is permeating all schools and all classrooms, the challenge is there for all to face. (Peter Reimann et al., 2010) Drawing on this, we propose that learning analytics research should focus on providing both computational and methodological support for teachers in real-time and in-situ classroom settings. Towards this end, we sought to integrate emerging developments in visual analytics and the established methodological approach of design-based research (DBR) in the learning sciences. The results of this integrative exercise are the approach called “Teaching Analytics” and a model of teaching analytics, termed “triadic model of teaching analytics (TMTA)”, discussed next. (4) Peter Reimann et.al, www.next-tell.eu (5) European Reference Framework: Key competences for lifelong learning. (6) Harnessing Technology for Next Generation Learning: Children, schools and families Implementation Plan 2009-2012. Downloadable from BECTA: http://publications.becta.org.uk/display.cfm?resID=39547 4. Triadic Model of Teaching Analytics (TMTA). Our model of teaching analytics seeks to adopt and extend the model of pair programming from the software engineering paradigm of Extreme programming. We propose an extensible triadic model. More specifically, teaching analytics adapts the Pair Analytics method [1] in visual analytics [36]. The Pair Analytics method was inspired by the Pair Programming7 model in the Extreme Programming8 software engineering approach. In pair programming, “all code to be sent into production is created by two people working together at a single computer8.” Our vision can be outlined as below: To empirically explore the effectiveness, efficiency and satisfaction in fundamentally transforming the teaching profession from a “lone ranger” model to the collaborative model where teachers, analysts and researchers with complementary expertise collaboratively leverage their knowledge, skills and aptitudes towards enhancing learning in high-performance/high-bandwidth/highdensity classrooms of the 21st century. However, the dyadic configuration of “driver” and “navigator” in pair programming and pair analytics creates a bootstrapping problem for learning settings: can we really throw a Visual Analytics Expert (VAE) and Teaching Expert (TE) together into a classroom setting and expect them to work productively without explicit facilitation, intelligent scaffolding, and guided design? Facilitating interaction is a role that can be fulfilled by a Design-Based Research Expert (DBRE). As such, we adapt and extend the dyadic model of pair analytics in visual analytics to a Triadic Model of Teaching Analytics (TMTA) as shown in Figure 1: Figure 1: Triadic Model of Teaching Analytics (TMTA). At its core, our model sees collaborative knowledge building between teachers, analysts and researchers. Each has a complementary role in the teaching analytics setting. (7) http://www.extremeprogramming.org/rules/pair.html (8) http://www.extremeprogramming.org/ Eliciting criteria for Teaching Analytics involves a collocated collaborative triad of a Teaching Expert (TE), a Visual Analytics Expert (VAE), and a Design-Based Research Expert (DBRE) analyzing, interpreting and acting upon real-time data being generated by students’ learning activities by using a range of visual analytics tools. We think of the relationships between the TE, VAE and DBRE as a dynamic socio-technical system. The design considerations are about creating feedback loops between the three individuals, such that each one drives the other two to higher levels of performance on the positive side (with the cost of anxiety in the negative case). That is, feedback from the teacher inspires the VAE to create new, better visualizations and for the researcher to better understand the ongoing teaching and learning processes while feedback from the VAE – perhaps in the form of visualization artifacts – allows the teachers to better understand what is going on in the classroom from a learning activity design perspective and the research to hypothesize, test and predict student learning trajectories and performance outcomes. All in all, these feedback loops should culminate in the teacher providing timely, meaningful actionable, customized and personalized feedback to students. The key point here is that each member of the triumvirate of TE, VAE, and DBRE can gain from the other two, not that each partner's role is to highlight deficiencies of the other two. Therefore, TMTA involves a close collaboration between the TE, VAE, and the DBRE. It includes teaching practitioners in the design process and invites them to contribute significantly to the innovation of the visual analytics tools. This allows these learning analytics tools to address pedagogical issues as they arise and evolve in real classrooms. In the next section, we outline an approach to TMTA based on open learner models (OLM). 5. TMTA and Open Learner Models. An obvious starting point for developing the TMTA approach is to base it around the existing work in Artificial Intelligence in Education, on open learner models. A learner model holds information (usually) about an individual learner, and the model is automatically and dynamically updated during the user's interaction with a computer-based/online educational environment. The learner model typically includes data about the learner's knowledge state, which may include specific difficulties and misconceptions; and it can also have data on other aspects of the learning process (e.g. representation, content, teaching style preferences; motivational, social, affective attributes). The learner model is then used by the educational environment to adapt its teaching to the specific needs of the individual learner (the environment 'understands' the user's understanding). An ""open learner model"" is a learner model that can also be externalised to the user [4]. This externalised (open) learner model may be simple or complex in format using, for example: text, skill meters, concept maps, hierarchical structures, animations [3]. Normally the user who accesses the learner model is the learner. Common purposes of externalising the learner model to learners are to promote metacognitive activity such as awareness-raising, reflection, self-assessment and planning [5]. Some learner models have, however, also been made available to teachers [6, 13, 43]. Teacher access to the learner models of their students can help them to better understand learners' needs as individuals and as a group, and can therefore enable teachers to adapt their teaching. Of particular interest in NEXT-TELL is the possibility of open learner models to support the routine but dynamic decision-making that teachers need to perform in the classroom. While the above describes the typical situation of open learner models, it is easy to envisage this being extended for use in TMTA. A range of visualisations or externalisations of the learner model have been explored (e.g. Bull et al., 2010), and these could be further extended to support the synthesis of work between teaching experts, visual analytics experts and design-based research experts, as required for the proposed TMTA approach. 6. Discussion. As mentioned in the prior section, we conceive of the Triadic Model of Teaching Analytics (TMTA) as a socio-technical system. Such systems are characterized by socio-technical interactions. The design considerations are to develop, deploy and evaluate the use and impact of the perception and appropriation of socio-technical affordances in the TMTA socio-technical system. Affordances are action-taking possibilities and meaning-making opportunities in an actor-environment system relative to the competencies of the actor and the capabilities of the system [39]. Based on the theory of socio-technical interactions in technology enhanced learning environments developed in [38, 39], we propose that design dimensions based on affordance classes [39] can help inform realize the idea of TMTA. Future work will consist of a systematic exploration and exploitation of the affordance classes in different socio-technical configurations of TMTA. In conclusion, we would like to highlight the similarity between the TMTA and the productive multivocality framework mentioned in the introduction. Whereas the productive multivocality framework focuses on relationships between researchers, the TMTA extends that multivocality to include teachers, design-based researchers, and visual analytics experts. Each voice in the system shares the goal for sustained innovation in leveraging the design of affordances of visual analytic tools to support teachers' dynamic diagnostic pedagogical decision making. Acknowledgement. This work is supported by the NEXT-TELL - Next Generation Teaching, Education and Learning for Life integrated project co-funded by the European Union under the ICT theme of the 7th Framework Programme for R&D (FP7)."