"1 Introduction. Lecture video capture solutions (e.g., opencast [1], echo360 [2], epresence [3], virtproducer [4]) are rapidly being adopted by traditional higher education institutions to increase the levels of blended learning available to students. This adoption is driven in part by the dramatic reduction in the costs of technology to institutions and students, the high availability of broadband internet access, the proliferation of media rich devices such as smart phones and tablets, and an interest in repurposing the traditional and widespread, “sage on the stage” model of teaching for anytime anywhere learning. Despite this adoption, very few studies have been done on the ways students use lecture capture technology to assist in their learning. This paper furthers this area by considering explicitly how students use the underlying technology, not necessarily its effects on student marks or enthusiasm. Using a bottom-up approach, we examine student interactions in the environment and outline a model for user tracking. The most important result from this work is the demonstration that low-level tracking data collected from lecture capture systems can be used to describe students around pedagogical goals. We augment this investigation with a more traditional top-down student survey on perceptions and experiences. Using grounded theory, we distil a variety of high-level student opinions about why they used the lecture capture system the way that they did into a set of categories that describe their system use. 2 System & Study. 2.1 Recollect Lecture Capture System. The Recollect system was developed in house at the University of Saskatchewan as an automated lecture capture, processing, and delivery system. It runs on commodity hardware inside of the classroom and can record the projector signal, one or more video cameras, and a single audio input. Students receive emails when lectures have been processed and published online, and they view lectures using an Adobe Flashbased web interface. Recordings from various video devices are merged into single streams based on declarative templates that are configured on a class by class basis. For instance, some classes may have a template that shows a single VGA feed with audio, while others may include two smaller NTSC-based camera feeds to the left of the VGA feed. Students cannot switch between feeds while viewing the lecture; they must watch the video using the template that the instructor has chosen for the class. Students cannot download videos, and must watch them through the web interface. Recollect allows instructors to create a template for how they would like the video to appear. All of the templates include a thumbnail menu on the left-hand side of the screen. The thumbnails in this menu are auto-detected segments of the lecture (see [5] for more information) that can be used for navigation. The traditional media “scrubber” widget is shown underneath all videos, and allows for navigation and adjustment of video parameters (e.g., sound levels). Instructors are given several options when creating their template. A typical classroom deployment can be seen in Fig. 1, where the template chosen by the instructor shows the lecturer next to his slides. Fig. 1 The Recollect system showing a typical classroom deployment, likes, VGA of the presenter desktop on the right-hand, and a camera view of the instructor roughly in the centre. Novel to this system is that it has been created with user modeling in mind. In short, the field of user modeling posits that by tracking individual accesses to software features and content, profiles of students and student cohorts can be created. These profiles can be used for automatic reasoning to provide content recommendation, environment adaptation, and just-in-time tutoring. While a full treatment of this area is well beyond this paper, we draw experience and motivation from two works in particular: the ecological approach, and attention metadata. The ecological approach is described succinctly by its creator, Gordon McCalla, who indicates that it “...involves attaching models of users to the information they interact with, and then mining these models for patterns that are useful for various purposes” [6]. Once patterns are found in the usage data, it informs the creation and operation of intelligent tools. These tools can then work in concert with the original system, creating new user modeling data for other tools in a sort of ecosystem of to support earning. 2.2 Situated Study. The Recollect lecture capture system has been deployed for several years, and a study investigating its use was conducted over one 15 week academic term in 2010. During this term, students from professional colleges and a number of different disciplines, including the sciences, social sciences, and humanities, were invited to use the tool to augment their in-class learning. The tool was made available to every student enrolled in the courses being recorded, as well as, depending on the instructor, to students in similar courses taught at different times. A total of 1,125 students participated in using the lecture capture system, out of roughly 2,000 eligible students1. Students were shown a brief five-minute introduction to the tool on the first day of class, and were not given external motivation to use the tool through payment or academic reward (e.g., marks for participation). At the end of the term, students were invited to fill out a 20 minute survey, on paper or online, describing their experiences with the lecture-capture system for a chance to win a gift certificate. This study had two goals: to create a low-level semantic logging framework that collected student interactions within the learning environment, and to analyse student interaction and perception data to form groups based on learning preferences. The stateless, distributed, and often disconnected nature of web systems makes user tracking difficult, and most content management systems which include video content do so only at the coarsest grain size. For instance, kaltura [8] and blackboard [9], only capture analytics at a high level, such as the number of times a student viewed a video or page. Unlike other forms of web content, video usage data at such a high level is difficult to use because the content changes rapidly and a single page impression does not characterize how long or the way in which a user interacted with the video. To address this, we built Recollect with an event model where every interaction the user has with the player (e.g., clicking on a button, seeing a popup thumbnail, seeking to a new time position) sends a tracking event to the server streaming the video. The player was also built with a “heartbeat” mechanism where a new event was created for every 30 seconds of continuous playback. Each event, regardless of its originating action, identified the user and aspects of the high-level video-player state, such as the current position in the video and whether the video was playing or paused. Many events also included specific details relevant to the action; for instance, a seek in the video would include both where (the time offset) in the video the user was when they started seeking and the position in the video they chose to seek to. Over the four month period of the study 3.4 million events were logged with the Recollect system, 1.8 million of which were “heartbeat” events that we analyse in the following section. (1) Exact numbers of eligible students are unable to be calculated as instructors or other students could share lectures with whomever they wished. 3 Quantitative Results. We hypothesised that students could be categorized into different groups based on their access patterns. In particular, we formed several sub-hypotheses for each kind of group that we predicted would occur: - H1: There will be a group of minimal activity learners. These students may have preferred methods of achieving their learning goals and will investigate the tool but not adopt it in any regimented fashion. Note that we only consider students who try the tool in our analysis, so this group will contain students who have viewed at least single lecture through the recording software. - H2: There will be a group of high activity learners. These students may not watch all of each lecture, but will watch some content each week. The key element of this group is that they are embedding video lectures in their learning routine. - H3: There will be a group of disillusioned learners. These students will be keen enough to use the tool near the beginning of the course but will stop using it because they found it did not aid in their learning. - H4: There will be a group of deferred learners. These students will not use the tool at the beginning of the course but began to use the tool closer to the end of the course will exist. This could be because students are leaving learning to the end of the course, or find latter course content builds on early content thus requiring more/deeper review. To test these hypotheses, we inspected heartbeat data for each student who used the tool for each week in the course. We discarded the sixth week from each student’s data as an outlier because the university was closed for holidays and accesses to the lecture playback tool were minimal. Using k-means clustering [10] with the Weka toolkit [11], we aggregated data for a large class participating in our study2. We limited our investigations to a single class to control for class-specific effects such as the timing of assignments and exams, or the cancelation of class due to holiday or instructor illness. We changed student access data into nominal values of “y” indicating that the student watched at least 10 minutes of lecture video that week or “n” to represent that they didn’t watch 10 minutes or more video. Only students who had accessed the system were included. The question of the number of clusters to choose when using k-means is always an issue, with fewer clusters generally seen as better since the introduction of each new cluster can lead to over-fitting the model to the data. We chose a number of clusters equal to our hypotheses plus one as an initial metric. The addition of the fifth cluster was to account for a group that we believed would exist but would be hard to classify; those students who used the tool intermittently to “catch up” on classes they may have missed. The results of k-means clustering with five categories are shown in Table 1. The Weka-based k-means clustering showed strong support for three of our four hypotheses. In particular, c0 corresponds well with H1, c4 corresponds with H2, and c3 corresponds with H4. Notably, H3, the hypothesis that students would start using the tool at the beginning of the term but drop off as the term progressed, was not verified. (2) The class we picked to analyse, a second year Chemistry class, was chosen because it had a high number of participants. Data from other classes was omitted due to space constraints. Table 1. Results of k-means clustering with five categories versus the 15 weeks of the academic term. The vast majority of students fall into the first cluster representing minimal or no accesses to the video playback tools. The most surprising result was the formation of two clusters around watching the video only in week eight or the combination of week seven and week eight (clusters c1 and c2 respectively), the former being extremely large. Referencing the course syllabus, the end of week eight corresponded to the placement of the midterm exam. Thus we present a new hypothesis backed by this data: - H5: There are a group of just-in-time learners. These students use the tool only for midterm exam review, though midterm review may stretch over several weeks of academic lecture time. Despite a good fit to our initial hypotheses, we experimented with both more and fewer clusters. The most interesting result was running k-means with a cluster size of 6 (shown in Table 2). This data shows students can be clustered well into all of the hypothesis previously given, including moderate support for H3 through cluster c5’. Drop off in engagement could be related to the tool, the content, or other factors. As the cluster includes an increase of viewership before the midterm examination it’s unlikely that students’ dropping the class is the main characteristic of this cohort. Further investigations into why students quit using the tool are being planned, including an active monitoring of course registrations. It should be noted that the increase in number of clusters results in a greater potential for data over fitting, especially as some clusters become quite small. Students who miss a few lectures and review them strategically are not well represented by these clusters. Evidence from student surveys and data analysis suggests that, while this group is small, it does exist. This may be because the clustering algorithm considers all attributes weighted evenly; as a result, it over specifies when more generic higher-level attributes might be a better indicator of the presence of this group. Further, the effects that other assessment mechanisms in the course have on viewing is not clear; the course had several assignments which are not represented in this data, but the sheer number of students identified as just-in-time learners suggests that this warrants deeper investigation. It was surprising to see that during the time between the last week of classes (week 13) and the final exam (end of week 15) no clusters included viewership of lecture material. This indicates that student learning strategies with respect to lecture capture vary between when class is in session and when formal instruction has ended. This begs for further investigation; our initial expectation was that students would have more time and thus be more likely to use recorded content right before the final examinations. Table 2. Results of k-means clustering with six categories. Very similar to the results with five categories shown in Table 2, this clustering now includes moderate support for H3 where we expected students to use the tool initially but gradually reduce their usage over time. 4. Qualitative results. At the end of the study period we conducted a survey to collect both qualitative and quantitative data. The qualitative data collection was intertwined with the quantitative by inserting open-ended questions into the appropriate sections of the survey. We then used grounded theory [12] to extract themes by coding each participant's responses to open-ended questions about system features and use. As a result of the open-ended nature of the questions, it was possible for students to express multiple ideas, which could result in a response being coded as belonging to multiple themes. Even though we collected student opinions about specific system features, we focus this paper on participant responses to system use questions since their responses regarding features were limited to technical details of those features rather than the motivation behind their overall system use or lack thereof. There were several students who did not use Recollect. When students were asked about why they chose to forego using Recollect they were able to select a reason from a predefined list or explain their reasons. Of the students who responded to this question, 81 selected a response from the list. They selected ""I thought the recoded lectures were not valuable"" most frequently (52), but other reasons were also chosen (19 were unaware that Recollect was available and 10 had limited computer and internet access). The explanations given by 176 students who expanded on their behaviour helped us to understand their approach to how a video capture system, such as Recollect, should fit into a class at the university level. We identified several themes from their responses that we combined into the following categories: - Logistical & Technical Problems: Scheduling or technical limitations that prevented students from using the system. This includes system avoidance because of previous bad experiences and the negative reports of others. - Unknown Resource: Students were unaware of Recollect's features or existence. - Don't Want/Need it: Students who thought that their current resources and learning efforts were sufficient or better than those provided by Recollect. - Anti e-Learning & e-Support: Students who were uncomfortable with or against the idea of online learning materials as well as those who did not want to help students who had been absent. - Will Only Use When Needed: Students who only used or would only use Recollect when they missed class or needed to review a concept. - Haven't Studied Yet: Students who claimed that they would use Recollect to study for their final exam. In Fig. 2 (a), we can see that the primary reason for students abstaining from Recollect use was that they did not feel that they needed the provided support. Many students ""...only used it when [they] missed class..."" or ""...as a back-up reference..."". The second most common response involved their lack of use of Recollect because of scheduling or technical constraints, such as the inability to read what was written on the chalkboard or hear student questions. While many students did not use Recollect, many others did; we asked these students what they liked best about Recollect. We analyzed their 207 open-ended responses and identified themes that were grouped into the following categories: - Logistics & Scheduling: Recollect allowed for anytime anywhere access to lecture material, which meant that students could maintain the schedules that fit their lifestyles and did not need to interrupt class or bother others to obtain course material. This also includes positive changes to the classroom environment that resulted from Recollect's use, e.g., only having interested students in class. - Class Attendance Not Required: Students were happy that they could miss class when ill or because of unforeseeable events. - Review & Notetaking: Students could review materials that they had missed or failed to understand, including verifying and completing personal lecture notes. - Having It: Students expressed a desire or appreciation for having the video, which includes positive statements about specific system features and feature requests. - Helps Understanding: Recollect in some way facilitated student understanding of course material. This includes accommodating learning styles and allowing students to focus on the message being delivered in class rather than on taking notes. As Fig. 2 (b) shows, students appreciated being able to miss class when they needed to. Recollect enabled them to stay at home when they were sick without sacrificing their ability to receive a similar educational experience as they would have had in class. Students also felt that Recollect ""...helped when [they] missed class or couldn't hear..."" what their instructor was saying. One nursing student said that she ""...was able to get a missed lecture and take better notes..."" by using Recollect's pause and play functionality. The second most appreciated functionality provided by Recollect relates to this latter student comment since the system allows students to review materials. The flexibility that a lecture capture system provides by enabling anywhere, anytime learning is phenomenal: ""...[Recollect] reduced stress if [I was] forced to miss a class for whatever reason...”. Students appreciated that they ""...didn't have to worry about contacting [their] classmates to know what [they] missed..."". Fig. 2 a) An analysis of why students, across all courses, did not use the video lecture capture system. While most felt it was unnecessary, the majority were not against its use. b) What students, across all courses, liked most about having video lectures available to them. One student who felt particularly challenged by his workload said that “...[he] enjoyed using it for the particularly difficult concepts in the course, and it was also really handy when [he] was being destroyed by a multitude of assignments and midterms within the span of a week...[He] was able to use the time [he] would be in lecture to attempt to pass all of those things, and [he] got caught up in class using recollect without being one of those mass-email jerks that ask for notes in every class...”. Students also liked having recorded lectures because it allowed them to listen to their instructor's explanation, which allowed them to focus on understanding so that they could ask questions in class and fill in the details in their notes later. Some students even went as far as never taking notes during class: ""I'd listen to lecture and not take notes, then watch video later and then take notes"". The argument that attendance is reduced when using lecture capture technologies is one of the principal issues facing adoption. While we did not collect metrics to measure attendance, one computer science student’s comment indicated that a lack of attendance is not always a negative when supporting classroom learning: “... [Recollect] also played a part in lowering the attendance of students who were largely disruptive in class, improves the classroom environment.” Unlike other lecture capture solutions, Recollect allowed users to take notes, within the system, while watching the video. Notes are indexed by the relative slide position using automated indexing. Students can convert these notes into printable PDF’s which include copies of the slides, and notes are automatically shared with other students in the class through the note-taking interface. Despite the student demand for using lecture capture for note-taking, very little activity was observed in the provided note-taking tools included in Recollect. Most students used the videos to take their own notes on paper. Student responses to lecture capture were far from uniformly positive; some students were ambivalent to the learning benefits it might bring or even directly hostile to the opportunities it might present to others. One chemistry student stated that “I go to the lectures, so I don't bother watching videos to relearn what I already did that day”. One of the more hostile responses came from a biosciences student who resented people who did not attend class. He said: “I showed up for class. Student's who don't show up for class should not be rewarded with lecture videos. Lecture podcasts should only be used for off campus education. Showing up for work is a reality of life, and students should get used to showing up for commitments they have made.” 5 Conclusions & Future Research. The adoption of lecture capture is increasing, with many institutions making selected lectures available on the web for free viewing through initiatives like the OpenCourseWare (OCW) consortium [13], portals like iTunes U [14], or YouTube Edu [15]. Lecture capture is particularly well suited for traditional higher education institutions that want to leverage their faculty and classroom experiences in increasingly connected online learning environments. This paper has made contributions to the understanding of what kinds of students use lecture capture systems, when those students engage in reviewing content online, and why they are motivated to use this technology. Analytics in learning systems can be used to provide both auditing and interventions in student learning. While we intend neither of these explicitly with this work, we aim to scaffold support for them by demonstrating how a low-level video logging tool can use automated clustering techniques to group students into pedagogically motivated cohorts. This form of analytics has been largely unexplored when it comes to lecture capture, and fits well with the heritage of intelligent and adaptive learning systems described in the user modelling and artificial intelligence in education communities. Building intervention tools to take advantage of these clusters, such as intelligent content recommendation or help seeking tools is a natural next step. What motivates students to use lecture capture is a broad question. We explored this by asking students a mixture of closed and open-ended questions. Students provided candid feedback and, while the diversity of opinion on how and even whether lecture video should be used varies, the opportunity to review a lecture and make notes seem central to the learning process. Our analysis limits itself to a single cohort for quantitative measures and to a single semester of data for qualitative measures. Tying these two data sets together into a single model is difficult; collecting qualitative data is expensive, and the diversity of teaching approaches in different courses makes collapsing usage data into one coherent set non-trivial. Nonetheless, even a surface analysis as we have done results in interesting and pedagogically useful results. Armed with the knowledge of how students use lecture recordings, we can begin to build intervention tools and strategies to increase student learning and satisfaction in rich media education environments."