"1. INTRODUCTION. A concern with improving learning is foundational within the field of learning analytics. It was there in Campbell and Oblinger’s early work [4] and is there in the definition of learning analytics from the First International Conference on Learning Analytics and Knowledge (LAK11) [22]: ""the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs."" The importance of interventions in learning analytics to close the feedback loop has been clear in the literature (if not always the practice) from the birth of the field. Analytics seeks to produce ‘actionable intelligence’ [5]; the key is that action is taken. Campbell and Oblinger [4] thus set out five steps in learning analytics: Capture, Report, Predict, Act, Refine. ‘Act’ explicitly includes making appropriate interventions, and this is echoed across the literature (e.g. [3, 9, 10, 13]). This paper builds on these ideas to articulate a Learning Analytics Cycle that makes the necessity of closing the feedback loop through appropriate interventions unmistakable. It also draws on the wider educational literature, seeking to place learning analytics on an established theoretical base, and develops a number of insights for learning analytics practice. 2. THE LEARNING ANALYTICS CYCLE. Figure 1, the Learning Analytics Cycle. The cycle, shown in figure 1, starts with learners. They may be students studying a course at a university, or informal learners taking part in a MOOC (a Massive Open Online Course, where the learners and materials are distributed across the web), participants at a research conference, or casual learners browsing Open Educational Resources (OER). The next step is the generation and capture of data about or by the learners – for instance, demographic information about a potential student logged during a phone call enquiry about study at a university; login and clickstream data generated in a VLE/LMS; postings to a forum; assessment results; or even alumni status. Some can be generated automatically; some requires a large multidisciplinary team to expend significant effort. The third step is the processing of this data in to metrics or analytics, which provide some insight in to the learning process. These include visualisations, dashboards, lists of ‘at risk’ students, comparisons of outcome measures with benchmarks or previous cohorts, aggregations, and so on. Again, some can be generated automatically, but others may take significant effort. This stage is the heart of most learning analytics projects, and has been the focus of great innovation in tools, methods and methodologies – e.g. dashboards, predictive modelling, social network analysis, recommenders, and so on. However, the cycle is not complete until these metrics are used to drive one or more interventions that have some effect on learners. This might be a dashboard for learners that enables them to compare their activity with their peers or previous cohorts, or a tutor making personal contact with a student that a model has identified to be at very high risk of dropping out. The cycle can be complete even where the intervention does not reach the learners who originally generated the data. To take a very simple example, a teacher reviewing the final grades for a course and using that to inform how to teach it with the following cohort is an example of the cycle in action. Learning analytics does not necessarily include all fours steps. A project that created reports about learners, but without any mechanism to feed this back in to an improved learning experience, would still be a learning analytics project, but not a very effective one. 3. LEARNING THEORY. The Learning Analytics Cycle has so far been presented as a development of previous theorisations of learning analytics. However, it is also more fundamentally, a development of much older learning theory. 3.1 Kolb. One of the most prevalent learning theories is Kolb’s Experiential Learning Cycle [11], which builds on Dewey and Piaget, and adds Lewin’s conception of learning through feedback, which was inspired by electrical engineering (to which this paper returns). Kolb’s Learning Cycle takes concrete experience as its starting point; reflective observation on this experience in turn builds abstract conceptualisation, which feeds through in to active experimentation, the source of further concrete experience. There are two levels at which the Learning Analytics Cycle develops Kolb’s cycle. Firstly, taking the system as a whole, there is a direct correspondence: actions by or about learners (concrete experience) generate data (observation) from which metrics (abstract conceptualisation) are derived, which are used to guide an intervention (active experimentation). Secondly, at an individual level, learning analytics can greatly facilitate the learning process of individuals, by making reflective observation and abstract conceptualisation easier and more readily available. These stages correspond to the ‘interventions’ component of the Learning Analytics Cycle: the learning analytics system makes metrics available to an individual, who observes, conceptualises, and then experiments by making (or attempting to encourage) some change to learner behaviour. Kolb’s cycle and related ideas have been critiqued extensively (see e.g. [19]). One main line of critique is that they are reductive of a holistic, emotional process to a rational, cognitive phenomenon, which would apply equally to learning analytics. The other fundamental charge against Kolb’s model – that it lacks strong empirical evidence – is one that learning analytics is in an excellent position to refute, or should be. 3.2 Schön. Another prevalent theorisation of learning arises from the work of Donald Schön [1, 17, 18] on reflective practice: how professionals learn and adapt their behaviour. Schön emphasised the importance of reflection-in-action and reflection-on-action. In this view, reflection is a form of feedback process or loop, an iteration between espoused theories and theories-in-use. The Learning Analytics Cycle instantiates and enables reflective learning, at both an individual and organisational level. As with Kolb, the ‘intervention’ stage of the Learning Analytics Cycle is where reflective practitioners compare their espoused theories with theories-in-use. One significant conceptualisation developed and popularised by Schön and Argyris [1] is a distinction between single-loop learning and double-loop learning. Single-loop learning is aimed at achieving a set outcome by adjusting practice; double-loop learning includes the possibility of changing the set outcome. They use the example of a domestic thermostat: it turns the heat on or off to achieve its set temperature (single-loop learning); but a human can adjust the set temperature (double-loop learning). A learning analytics system may be used simply to attempt to achieve set goals (single-loop learning); greater value and insight will come if those goals themselves can be interrogated, challenged, and developed (double-loop learning). Learning analytics can thus be a powerful force for informing and validating learning theories. 3.3 Laurillard. In the UK, another widely-cited theory is Diana Laurillard’s Conversational Framework [12], which draws on Kolb’s cycle and Pask’s Conversation Theory [14]. In this theory, learning takes place through a series of ‘conversations’ between a teacher and a student (and with other students), underpinned by reflection and adaptation. These conversations happen on two levels: at the level of action, and at the level of conception or description. At an individual level, a Learning Analytics Cycle facilitates the conversation between the teacher and student: providing information on the students’ actions and conceptions, enabling richer adaptations and feedback in turn from the teacher’s constructed environment. The parallels at a whole-system level are less transparent but perhaps even richer. The Learning Analytics Cycle can be conceptualised as enabling conversations at multiple levels, between multiple actors, with iterative, adaptive feedback. 3.4 Other educational literature. The approaches to learning literature (e.g. [16, 21]) identifies qualitatively different approaches to study – a deep, surface or strategic approach. This literature has uncovered associations at the population level between approaches (of the learner and teacher) and the final outcome, including to widely-used evaluation questionnaires. Learning analytics offers the possibility of tracking and researching these associations in real time, and – most importantly – using them to enhance the learner’s experience before they come to the end of their study. 4. ENGINEERING THEORY. The educational theories mentioned above take inspiration from the cybernetic conception of control theory, and in particular, the closed-loop control system used widely in engineering of all sorts. In a closed-loop control system, the output(s) of the system is measured and then processed by a controller, which in turn makes an appropriate adjustment(s) to the input(s), creating a feedback loop. In an open-loop control system, the controller adjusts the input purely based on its own settings, without taking any account of the output. Open-loop control systems are typically quicker, simpler and easier to implement. However, closed-loop control systems are more robust at achieving the desired output, particularly when something within the system changes or the system is complex. The parallels for learning analytics are readily apparent. Organising learning without feedback from the outputs is akin to an open-loop control system: it may be quicker, but the final output may not be the desired one. The Learning Analytics Cycle works analogously to a closed-loop control system: the data generated by or about learners is the output, which is compared to some reference (e.g. previous learner data, or a desired outcome), which is then used to drive an intervention which alters the learning process (input). 5. IMPROVING EFFECTIVENESS: SPEED AND SCALE. A key consideration for the effectiveness of the feedback cycle is the speed and scale of the intervention. These are properties of the entire system: that is, they include the people, policies and practices connected to the learning. The people involved can be classified in to the following four stakeholder groups: - learner – anyone engaged in learning - teacher – anyone engaged directly in facilitating learning: includes teaching assistants, associate lecturers, adjunct faculty, faculty, academic staff, and peers in some contexts such as MOOCs - manager – anyone responsible for the organisation or administration of teachers: includes departmental-level and institutional-level management (e.g. managers, administrators, heads of department, Deans, executive officers (CxOs), presidents, provosts, vice-chancellors, rectors and their deputies) - policymaker – anyone responsible for the setting of policy, whether at a local, regional, state, national, or transnational/ intergovernmental level, and including funders. As shown in figure 2, the learner is the closest to the learning activity. They can make very quick changes to their own learning, but limited changes to others’. The teacher is one step away from the learning activity, but is able to make interventions that may span several learners. At one further remove is the manager, who is slowed down by the need to receive second-hand reports from the teachers, but may well be able to make interventions that affect more learners than an individual teacher. Furthest from the learning activity is the policymaker, likely to be slowest of the four in speed of response, but with the widest responsibility. Figure 2: The scale (horizontal axis) and speed (vertical axis) of intervention readily achieved by different stakeholder groups in a learning analytics system, with proximity to the learning activity (depth axis). In a given learning analytics project, there are three strategies by which the effectiveness of the cycle can be improved. Firstly, the speed of response can be enhanced, e.g. by real-time feedback to stakeholders who can act more quickly, such as the teacher and the learner themselves. Secondly, the scale of response can be enhanced, e.g. by providing feedback to a larger number of stakeholders. Thirdly, the quality of the intervention itself can be improved, e.g. by testing the intervention to see whether it is effective, through feedback from the outputs of the learning (Schön’s double-loop learning discussed above), or by enabling more stakeholders to participate. (This has parallels in the Open Source Software dictum that “Given enough eyeballs, all bugs are shallow.” [15]) There may well also be an increase in quality if feedback is directed to those who have the best information about the learning, such as learners and teachers. The idea of improving learning through feedback via teachers and learners themselves is far from new (see e.g. [7, 8]), and notably, the Signals project at Purdue University [2], perhaps the bestknown successful example of learning analytics, has feedback to learners and teachers at its heart. 6. ASSESSMENT AND INAPPROPRIATE USE OF METRICS. Assessment can be considered to be a special case of the Learning Analytics Cycle. In traditional marking, a learner takes a test, which the teacher marks and returns to the learner (learner generates data which is processed in to a metric). All too often, the cycle is not completed at this point. Many learning analytics models treat assessment as the final outcome measure to be optimised, rather than an interim one. This is extremely valuable. But assessment data has far more potential than this: treating it as an input or intermediate variable can yield extremely valuable insights, and learning analytics systems can provide assessment-like feedback even in informal settings (e.g. [6]). It has been established for at least 40 years [20] that learners identify the ‘hidden curriculum’ revealed in the assessment. Teachers may say they want their students to pursue intellectual problems, apply their creativity and make mistakes from which they then learn, but if the assessment tasks predominantly reward rote learning, learners are likely to study that way. This is a specific instance of a general risk in learning analytics: of optimising to a metric that does not reflect what is more fundamentally desired as an outcome. All metrics carry a danger that the system will optimise for the metric, rather than what is actually valued. This danger is not new – Kolb argued emphatically that ‘learning is best conceived as a process, not in terms of outcomes’ ([11] p. 26) – but learning analytics makes it more pressing. Thus learning analytics should generate metrics that relate to what is valued in the learning process. If the final assessment rewards undesired behaviour, improving the control system to more effectively optimise the results will make the learning worse. 7. OPENNESS. Being open and transparent benefits learning analytics in (at least) three different ways. Firstly, it makes learning analytics more effective. As discussed above, if more people can see the metrics, there are more people to understand. Opening up metrics reduces potential barriers to effective working (e.g. teacher’s password expired, wrong permissions set, system complexity and performance). Secondly, transparency leads to greater social acceptability. Egregious misapplications of analytics are more likely to be identified and challenged by stakeholders – and correct, if the learning system of the organisation does not prevent it. Thirdly, data protection legislation may make it a legal requirement. This is not, of course, simple or straightforward. One cannot simply make all learners’ data and metrics available to the entire world on the web. However, significant potential is lost when restrictions are added needlessly. 8. CONCLUSION. The Learning Analytics Cycle, with its theoretical grounding, suggests ways in which learning analytics projects can be made more effective. Fundamentally, this requires closing the feedback loop through effective interventions that reach learners. These loops can be made more effective if they are faster, or larger in scale. Strategies to achieve this include considering learners and teachers as audiences for learning analytics as well as managers and policymakers."