"Introduction. According to a recent survey conducted by Campus Computing (campuscomputing.net) and WCET (wcet.info), almost 88% of the surveyed institutions reported having used an LMS (Learning Management System) as a medium for course delivery for both on-campus and online offerings. In addition to various student information management systems (SISs), LMSs are providing the educational community with a goldmine of unexploited data about students’ learning characteristics, behaviours, and patterns. The turning of such raw data into useful information and knowledge will enable institutes of higher education (HEIs) to rethink and improve students’ learning experiences by using the data to streamline their teaching and learning processes, to extract and analyse students’ learning and navigation patterns and behaviours, to analyse threaded discussion and interaction logs, and to provide feedback to students and to faculty about the unfolding of their students’ learning experiences (Hung & Crooks, 2009; Garcia, Romero, Ventura, & de Castro, 2011). To this end, data mining has emerged as a powerful analytical and exploratory tool supported by faster multi-core 64 CPUs with larger memories, and by powerful database reporting tools. Originating in corporate business practices, data mining is multidisciplinary by nature and springs from several different disciplines including computer science, artificial intelligence, statistics, and biometrics. Using various approaches (such as classification, clustering, association rules, and visualization), data mining has been gaining momentum in higher education, which is now using a variety of applications, most notably in enrolment, learning patterns, personalization, and threaded discussion analysis. By discovering hidden relationships, patterns, and interdependencies, and by correlating raw/unstructured institutional data, data mining is beginning to facilitate the decision-making process in higher educational institutions. This interest in data mining is timely and critical, particularly as universities are diversifying their delivery modes to include more online and mobile learning environments. EDM has the potential to help HEIs understand the dynamics and patterns of a variety of learning environments and to provide insightful data for rethinking and improving students’ learning experiences. This paper is focused on understanding live video streaming (LVS) students’ learning behaviours, their interactions, and their learning outcomes. More specifically, this study explores how the interaction of students with each other and with their instructors predicts their learning outcomes (as measured by their final grades). By investigating these interrelated dimensions, this study aims to enrich the existing body of literature, while augmenting the understanding of effective learning strategies across a variety of new delivery modes. This paper is divided into four sections. It begins by reviewing the literature dealing with the use of data mining in administrative and academic environments, followed by a short discussion of the way in which data mining is used to understand various dimensions of learning. The second section explains the purpose and the research questions explored in this paper. The third section describes the background of the study and details its methodological approach (sampling, data collection, and analysis). The paper concludes by highlighting key findings, by discussing the study's limitations, and by proposing several recommendations for distance education administrators and practitioners. Data mining applications in administrative and academic environments At the intersection of several disciplines including computer science, statistics, psychometrics (Garcia et al., 2011), data mining has thrived in business practices as a knowledge discovery tool intended to transform raw data into highlevel knowledge for decision support (Hen & Lee, 2008). To this end, a wide range of tools that can be used for collecting, storing, analysing, and visualizing data, such as the SPSS Modeler (formerly Clementine) and the SAS Enterprise Miner, have been developed in the business world. These tools use sophisticated computing paradigms including decision tree construction, rule induction, clustering, logic programming, and statistical algorithms. Although data mining has been widely used in business environments to predict future trends and consumer behaviours (Harding, Shahbaz, Srinivas, & Kusiak, 2006; Ngai, Xiu, & Chau, 2009), the data mining method has been dramatically under-used in education research in general (Faulkner, Davidson, & McPherson, 2010). Only recently have higher education institutions started to exploit the potential of this powerful analytical tool (Black, Dawson, & Priem, 2008). However, according to Romero and Ventura (2010), educational data mining (EDM) has emerged as a new field of research capable of exploiting the abundant data generated by various systems for use in decision making. The enthusiastic adoption of data mining tools by higher education has the potential to improve some aspects of the quality of education, while it lays the foundation for a more effective understanding of the learning process (Baker & Yacef, 2009). EDM, when integrated into an iterative cycle (Romero, Ventura, & Garcia, 2008) in which mined knowledge is integrated into the loop of the system not only to facilitate and enhance learning as a whole, but also to filter mined knowledge for decision making (Romero et al., 2008) or even to create intelligence upon which students, instructors, or administrators can build, can notably change academic behaviour (Baepler & Murdoch, 2010). From an administrative perspective, Chang (2006) argues that the predictive capacity of data mining can further enhance enrolment management strategies by increasing the HEIs’ understanding about their admitted applicants. Similarly, Delavaria, Phon-Amnuaisuka, and Reza Beikzadehb (2008) contend that data mining knowledge techniques are capable of enabling higher learning institutions to make better decisions, to put more advanced planning into place to direct students, and to predict individual behaviours with higher accuracy, and, in so doing, to enable the institutions to allocate resources and staff more effectively. Without inflating the merits of data mining in rethinking administrative and academic processes, it is clear that data-mining is gaining ground and is providing powerful analytical tools capable of converting untapped LMS and EPR data into critical decision-making tools with the potential of enhancing students’ learning experiences (Garcia et al., 2011). From a learning perspective, according to Castro, Vellido, Nebot, and Mugica (2007), data mining is being used in higher education - to assess students’ learning performance - to provide feedback and adapt learning recommendations based on students’ learning behaviours - to evaluate learning materials and web-based courses, and - to detect atypical students’ learning behaviours. Following this line of thinking, Perera, Kay, Koprinska, Yacef, and Zaiane (2009) used clustered data mining techniques to support the learning of group skills by building automated mirroring tools capable of facilitating group work. In a similar study, Sun, Cheng, Lin, and Wang (2008) used rules based on data mining results to form high interaction-learning groups. For their part, Hung and Zhang (2008) applied data mining techniques to server logs, both to reveal online learning behaviour patterns and to support online learning management, facilitation, and design. Their study’s results revealed students’ behavioural patterns and preferences, which helped them to identify active and passive learners and which extracted important parameters for the prediction of the students’ performance (Hung & Zhang). Using a similar approach, Ba-Omar, Petrounias, and Anwar (2007) analysed web access logs to identify learning patterns and offline learning styles. In a recent study, Abdous and He (2011) used text mining as a detection tool for the common technical problems faced by students taking video streaming courses. Elsewhere, Zaiane and Luo (2001) analysed server logs to understand online learners’ behaviours in an effort to improve their web-based learning environments. Later, Zaiane (2002) used association rule mining to construct a recommender-system based on data from online learners’ profiles, access histories, and collective navigation patterns. This system can “intelligently” recommend learning activities or shortcuts to learners, based on the actions of previous learners. Similarly, Burr and Spennemann (2004) have pointed out that analysis of the patterns of user behaviour is important from both the technical and the pedagogical perspectives in order to predict network and traffic load, to align pedagogy with users’ behaviours, and to plan and deliver services in a timely manner. For their part, Dringus and Ellis (2005) proposed a data mining approach for “discovering and building alternative representations for the data underlying asynchronous discussion forums.” This approach is intended to improve the instructor’s ability to evaluate the progress of a threaded discussion. More recently, Lin, Hsieh, and Chuang (2009) conducted a study to investigate the potential of an automatic genre classification system (GCS) that can be used to facilitate the coding process of the content analysis of a threaded discussion forum, Of particular relevance to our study, we discovered several studies which have used various EDM techniques to predict students’ performance as measured by final grades. Minaei-Bidgoli and Punch (2003) used web-use features such correct answers, number of attempts for doing homework, total time spent on problems, participation in communication, and reading of material as predictors of students’ final grades. Their prediction accuracy varied between 51% and 86.8%, depending on the type of classifier used. Similarly, Falakmasir & Jafar (2010) used data mining to rank students’ activities which affected their performance, as measured by their final grade. Their findings suggest that students’ participation in virtual classrooms had the greatest impact on their final grades. For their part, Zafra and Ventura (2009) used a grammar-guided genetic programming algorithm to predict students’ success or failure. These predictions were used to provide alternative learning activities that would enhance the students’ chances of success. Using Learning Management Systems-generated student tracking data (Macfadyen & Dawson, 2010), we propose the development of a customizable dashboard-like reporting tool. This tool is intended to provide instructors with real-time data on both students’ engagement and the likelihood of their success. Unsurprisingly, their findings confirm that students’ contribution to the course discussion board is the strongest predictor of their success. In reviewing the literature, Romero, Espejo, Zafra, Romero, and Ventura (2010) identified several avenues for using classification in educational settings: discovering student groups with similar characteristics, identifying learners with low motivations, proposing remedial actions, and predicting and classifying students using intelligent tutoring systems. For their parts, Anand Kumar & Uma (2009) used the classification process to examine various attributes affecting student performance. Castellano and Martínez (2008) used collaborative filtering techniques to exploit students’ grades in order to generate group profiles which could facilitate academic orientation. Along the same lines, Vialardi et al. (2011) used data mining techniques which employed the students’ academic performance records to design a recommender system in support of the enrolment process. In sum, this quick overview of the literature suggests that using various data mining techniques to predict students’ performance as measured by final grades has been examined by several different studies of traditional learning management systems. However, none of the studies has explored the dynamics of online interaction in a live video streaming environment. With these considerations in mind, we aim to apply both regression analysis and clustering analysis in order to explore students’ learning behaviours (students’ participation, login frequency, number of chat messages, and the type of questions submitted to instructor) along with their final grades. More specifically, we attempt to answer the following two questions: - What are the major themes emerging from LVS students’ online questions? - How do these emerging themes predict the students’ course grades? Figure. 1. LVS interface. Methodology. Context of the study. This study was conducted in a public research university in the mid-Atlantic region which serves 17,000 undergraduate and 6,000 graduate students and offers more than 70 bachelor’s degree programs, 60 master’s degree programs, and 35 doctoral degree programs in a variety of fields. Located in a major maritime, military, and commerce hub, this institution offers strong emphases in science, engineering, and technology, especially in the maritime and aerospace sciences. The university is also known as a national leader in technology-mediated distance learning, having served students at over 50 sites in Virginia, Arizona, and Washington state for more than twentyfive years. This extended distance learning capability provides the university with a variety of delivery mode options (i.e., ways in which a course can be delivered). Courses can be offered simultaneously via three different delivery formats: face-to-face, via satellite broadcasting, and via live video-streaming. Using the live video-streaming (LVS) delivery mode, students participate in the class, in real time, via personal computer, over which they view a live feed of the class lecture and during which they can interact with their instructor by sending text messages through the LVS course interface. Using the same interface, LVS students are able to chat with their LVS classmates during class. At the receiving end (i.e., in the physical classroom), questions submitted by LVS students are displayed instantaneously on a monitor next to the instructor. Instructors have the option to read/answer the messages, or to save, archive, and email them for later review. This tool is intended to enable instructors to seamlessly integrate LVS students into their classroom dynamic, without distraction and without overburdening instructors during their class time (Figure 1). Figure 2. Interaction in VS courses. Participants. In total, 1,144 students completed 138 courses in a variety of subjects (e.g., accounting, computer engineering, information technology, human services, etc.) via the video streaming (VS) delivery mode during the Fall semester of 2009. All of the student-to-instructor questions, the two-way student-to-student chat messages, and the total login times were collected. Those VS students who never asked questions or chatted with their peers online were excluded from the actual analysis. The reasons why those students failed to get involved in the VS course discussion are suggested to be included in future investigation. One possible explanation is that some instructors never took the effort to invite their VS students to ask questions or to engage in online discussion. As a result of the pre-processing of the data, 298 students (those with complete information about their number of questions, number of chat messages, total login times, and final grade) were included in the data analysis. Due to factors such as privacy and university policy, the university’s registrar’s office could not provide us with the age or gender of these students, nor could we obtain the grading scales of each course. (The grading scale for each course at our university is determined by that course’s instructor.) Table 1. Distribution of students by college. Table 2. Distribution of students by academic level. The questions and chat messages posted by those 298 students, along with their course ID, Student ID, the date, and a time stamp were saved in the database. Clustering analysis: Identification of emerging themes. Our analytical approach included three phases: pre-processing, in which raw data is transformed into a usable format, mainly by cleaning, assigning attributes, and integrating data; mining the data by applying various mining strategies and tools such as classification, clustering, and visualization; and post-processing, which allows for interpretation and use of the gained knowledge in rethinking processes or in making decisions (Garcia et al., 2011). All of the questions posted by the students were recorded in the Microsoft SQL server Database. To prepare the data processing for clustering analysis, we wrote a program using the PHP programming language to aggregate questions from the same students within the same course in order to form a case which included the sequence of questions posted by the students. Subsequently, we used NVivo 9 software to apply an automatic coding technique to each of the student question cases. Nvivo is a leading qualitative analysis tool on the market and has been used and tested by many researchers for content analysis (Zha, Kelly, Park & Fitzgerald, 2006). Automated coding one of NVivo 9’s features; it allows for automatic coding of a text document by text strings. After nodes were generated from each student question case, a clustering analysis was conducted in order to classify these nodes into different clusters with NVivo 9. According to Nvivo, nodes are containers for specific themes, people, places, organizations, or other areas of interest. Researchers can organize nodes into hierarchies – moving from more general topics (the parent node) to more specific topics (child nodes) – in order to support their particular research needs. Clustering analysis is a well-studied technique in data mining (Lin, et al., 2009) that uses an exploratory technique to visualize patterns by grouping sources which share similar words or attribute values, or which are coded similarly. From a data mining perspective, clustering is the unsupervised discovery of a hidden data concept. This approach is used in those situations in which a training set of pre-classified records is unavailable. In other words, this technique has the advantage of uncovering unanticipated trends, correlations, or patterns; no assumptions are made about the structure of the data (Chen & Liu, 2004) The purpose of clustering analysis in this study is to classify students based on the student-shared characteristics in their questions. The cluster analysis tool in the NVivo 9 software confers upon researchers a different perspective on the unstructured textual data. Using the calculated similarity in each word that appears in the text of the nodes, NVivo 9 groups the nodes into a number of clusters. In our study, a statistical method named the Pearson correlation coefficient (-1 = least similar, 1 = most similar) was used as the similarity metric for the clustering analysis. The Pearson correlation coefficient is the preferred similarity metric used with Nvivo. More information about the clustering analysis of Nvivo can be found in Nvivo’s online documentation website, http://www.qsrinternational.com/support.aspx. To gain further insight from the textual questions or chat messages, we also applied the SPSS Clementine tool, which allowed us to analyse the unstructured textual data. The SPSS Clementine tool provides linguistic methods (extracting, grouping, indexing, etc.) for researchers to use in order to explore and extract key concepts from the text. As the result of the text mining, key concepts in our study were extracted and identified for analysis. Measurement of final grade. The students’ final grades, submitted to the University Registrar by each course instructor, were supplied to us by the University Registrar. In the actual data analysis, the final grades were categorized into three groups: A- to A, B- to B+, and Others. Quantitative data analysis: Predictive relationship between online question theme and final grade In the current study, all of the quantitative data analysis was implemented using SPSS 17.0. Furthermore, the alpha levels were set at the .05 level for all significance tests. Due to the ordinal nature of the final grade, ordinal logistic regression analysis (Norusis, 2008; O’Connell, 2006) was implemented in order to examine the predictive relationship between the online question theme as the predictor and the final grade as the criterion variable. Specifically, a cumulative odds model was fitted to the data. The use of ordinal logistic regression, which was closely related to logistic regression, helped to avoid the statistical consequences that could occur from the violation of assumptions in linear regression, such as normality of errors and linearity in the parameters (King, 2008). The log transformation in logistic regression also ensured that the predicted probabilities for the event of interest would range from 0 to 1 without imposing the numerical constraint on the predicted log odds from the logistic equation (Cohen, Cohen, West, & Aiken, 2003). Given the ordinal nature of the final grade, ordinal logistic regression was used to take into account the information regarding the rank ordering of the outcomes (Hosmer & Lemeshow, 2000). The overall predictive utility of the ordinal logistic model with the online question theme as the predictor was assessed by testing the improvement of the model fit relative to the null model with no predictor, with the χ2 likelihood ratio test of the differences in deviances (O’Connell, 2006). The individual parameter estimate (i.e., the location coefficient) for the predictor variable was tested with the Wald test (Norusis, 2008). In ordinal logistic regression, two cutoffs (A- and B-) were used sequentially to form the cumulative odds equal to or higher than those two cutoffs, respectively. As a result, the probabilities of falling into three possible categories of final grade (A- to A; B- to B+; and Others) could be derived. Two different pseudo R2 (Cox and Snell R2 and Nagelkerke R2) were also computed in order to quantify the overall model fit (O’Connell). The larger the pseudo R2, the better the model fit. The parallel lines assumption in ordinal logistic regression was checked with the χ2 likelihood ratio test (Norusis, 2008) to see if the relationship between those two research variables remained the same across two cutoffs (A- and B-). Results. Identification of online question themes. In the current study, questions from each student during a semester were combined into one student entry so that students could be classified into different clusters based on the characteristics of their questions. The cluster analysis tool calculated each different word that appeared in the text of the entries by using the similarity metric. Then the entries were grouped into a number of clusters by NVivo 9, based on the calculated similarity index between each pair of entries. As a result, four major clusters of students were formed, based on the similarity of their questions. A multi-level, multiple cluster hierarchical structure was generated by clustering analysis (see Figure 3). These clusters were reviewed and interpreted collectively by two researchers and a graduate assistant. The two researchers had recently received specialized training about Nvivo 9 from the software producer. Differences in the review were compared, discussed, and resolved to reach an agreement. The coding results were further reviewed and discussed with an educational researcher to validate their accuracy. Figure 3. A cluster hierarchical structure. After a close review of the student questions in each cluster, four major themes were found (see Table 3). Table 3. Four major themes in students’ online questions. Descriptive statistics of final grade. The descriptive statistics of students’ final grades by their online questions are listed in Table 4. Overall, about half (144, 48.32%) of the participants obtained a grade of A- or higher. Among the 298 participants, 90 posted mainly check-in questions and 87 posted questions related to deadline and schedule. The number of participants who posted questions mostly related to learning and comprehension was the lowest, relative to the number of their counterparts posting questions on other themes. Table 4. Descriptive statistics of final grade by online question theme (N = 298). Predictive relationship between online question theme and final grade In the ordinal logistic regression model (see Table 5), the results of the chi-square likelihood ratio test supported a nonzero predictive relationship between the online question theme and the final grade, χ2 (3, N = 298) = 10.017, p < .05. Furthermore, the results did not indicate the violation of the parallel lines assumption, χ2 (3, N = 298) = 2.051, p > .05. Therefore, the predictive relationship between the online question theme and the final grade remained constant across two cutoffs of final grade (Norusis, 2008). The Cox and Snell R2 and the Nagelkerke R2 were .033 and .038 respectively, and indicated a modest predictive relationship. Overall, the online question theme would prove to be a useful predictor for the final grade. The logistic regression coefficients (i.e., the location coefficients) for question themes 1, 2, and 3 were all positive and were statistically significant at the .05 level. Due to the way in which the ordinal logistic regression model was set up in SPSS (Norusis, 2008), the above statistically nonzero, positive regression coefficients suggested that the odds of getting a higher final grade, relative to all lower final grades at various cutoff values, were higher for the participants whose questions concerned learning/comprehension (Theme 4) in comparison with participants with the other three question themes (i.e., 1: Check-in; 2: Deadline/Schedule; 3: Evaluation/Technical). Specifically, for participants with the question theme of Learning/Comprehension, the odds of obtaining a grade equal to or higher than those two cutoffs (A- and B-), relative to all other lower grades, were 2.214 times higher than for the students whose questions had the theme of check-in, 3.020 times higher than those whose questions had the theme of deadline/schedule, and 2.361 times higher than the students whose questions had the theme of evaluation/technical. While using Question Theme 1, or Question Theme 2, or Question Theme 3 as the reference category respectively, no differences in the odds of obtaining better grades were found among the three theme groups. The computed predicted probabilities of obtaining a final grade of A- to A+, B- to B+, and Others, respectively, for participants in those four question theme groups (1: Check-in; 2: Deadline/Schedule; 3: Evaluation/Technical; 4: Learning/Comprehension), were 47.07%, 32.89%, 20.04% in the Theme 1 group, 40.75%, 34.77%, 24.48% in the Theme 2 group, 45.48%, 33.43%, 21.09% in the Theme 3 group, and 66.31%, 23.51%, 10.17% in the Theme 4 group. Table 5. Ordinal logistic model with online question theme as the predictor for final grade (N =298). Two cutoffs were set for the ordinal criterion variable, final grade, to examine how the increase in the faculty engagement score was related to the change in the odds, and in turn, to the probability of obtaining a higher final grade (O’Connell, 2006). The odds of obtaining a higher final grade at two cutoffs were the ratios of the probabilities of: A to all lower grades, and A through B- to all lower grades. The faculty engagement scores as the sample mean (i.e., 20.697 in raw score) and the one standard deviation (i.e., 5.560 in raw score) above the sample mean were examined to demonstrate the way in which the probability of obtaining a higher course final grade changed with the increase in faculty engagement (Norusis, 2008). Given an increase of one standard deviation in the faculty engagement score from the sample mean (i.e., from 20.697 to 26.257), the predicted probability of obtaining a final grade of A increased from 46.71% to 59.78% at the first cut-off. At the second cut-off, the predicted probability of obtaining a final grade of B- or higher increased from 78.79% to 86.30%. Moreover, with the faculty engagement score as the sample mean (i.e., 20.697 in raw score), the predicted probabilities of obtaining one of those three categories of course final grade (A, A- to B-, or Other) were 46.71%, 32.08%, and 21.21% respectively. While the raw faculty engagement score increased by one standard deviation to 26.257, the predicted probabilities of obtaining one of those final grades became 59.78%, 26.52%, and 13.70% respectively. Therefore, the increase in the faculty engagement score was accompanied by the increased probability of obtaining a better course final grade. Discussion. A student’s final grade depends on many factors, including the student’s motivation, learning style, and previous background, the instructor’s teaching and grading scales, the exam’s and assignment’s difficulty levels, etc. A holistic view of student demographic and institutional variables, as opposed to the single variable, must be examined in determining the overall online learning experience (Herbert, 2008). In this study, our data shows that online VS student participation cannot be safely used to predict final grades. Perhaps the uniqueness of our VS interface (text-based chat in a live-video-streaming environment) explains our findings. Otherwise, previous studies including Macfadyen and Dawson’s study (2010) found that students’ participation and contribution to discussion boards in traditional learning management systems remain some of the strongest predictors of students’ success. However, our analysis found that there is a correlation between questions posed to instructors and chat messages posted among students. Those who chat often also interact more often with their instructor. We also analysed the chat messages (student-to-student communications) using the SPSS Clementine text mining tool. We noticed two outstanding concepts in the students’ chat messages (among themselves) and their frequency: they discussed technical problems (videos, sound, etc.) at 5% and test/exam issues at 2%. However, they addressed the same concepts in their messages to the instructor with this frequency: technical problems at 2% and test/exam issues at 2%. Thus, it seems that students are more likely to discuss technology problems with their peers and try to help each other than to discuss those issues with their instructors. The messages also revealed interaction patterns including topics related to project and assignment collaboration, discussion of grades, socialization, and greetings. In addition, the data reveals that students with a higher number of logins asked more questions and exchanged more chat messages with their classmates. In contrast, students with fewer logins rarely participated in the class; in fact, some of them rarely even logged into the system. Conclusions and future research. This study was conducted in order to exploit the untapped data generated by LVS students. Our results revealed several student learning behaviours, ranging from active participation and interaction with the instructor to a lack of participation or even of attendance. Overall, our findings corroborate those of a previous study (Abdous & He, 2011). In spite of the limitations related to self-selection bias and to the use of final grades as a measurement of student learning outcomes (Abdous & Yen, 2010), we believe that we can provide some ways in which the learning experiences of LVS students can be improved and made more successful, based on our years of experience of working with faculty who teach VS courses. To this end, the following recommendations are made: - Ensure faculty readiness and training prior to teaching LVS courses. - Develop facilitation techniques to assist faculty in integrating LVS students into the dynamics of the classroom. - Implement a tracking system for LVS students’ attendance. - Encourage active participation and interaction during LVS sessions. - Provide students with tips on effective participation and interaction during LVS sessions (writing messages"