"1. Comparison of tutor dialogue moves within off topic modes and base rate. As was expected, student and tutor off topic dialogue moves along with other socially- focused dialogue moves occurred more frequently during the non-pedagogical modes [8]. Consistent with this expectation, problem solving and other pedagogically-focused dialogue moves occur outside of non-pedagogical modes. In particular, the absence of tutors asking questions, students answering questions, and tutors giving feedback demonstrates that non-pedagogical modes are truly a separate, distinct time of the tutoring session. Table 2. Comparison of student dialogue moves within off topic modes and base rate. Those occurrences which break from these general patterns are of particular interest. The only pedagogical move to occur more frequently in non-pedagogical modes was preview. This could indicate that non-pedagogical modes are being used as a transition between new topics or problems. Preview could potentially be occurring within Introduction, which can be thought of as a preview to the entire tutoring session. For student dialogue moves, the higher occurrence of metacomment shows that non-pedagogical modes contain discussions of the student’s knowledge. Many metacomments are delivered in response to tutor comprehension-gauging questions (i.e., “Do you understand?” “Okay?”). The fact that comprehension-gauging questions are not significantly occurring during non-pedagogical modes suggests that student knowledge and comprehension is being discussed in a different context. However, the strongest occurrences by far were still tutor (d = 2.45) and student (d = 1.79) off topic dialogue moves. Given that these dialogue moves serve as a catchall for any topic outside of the tutoring topic (e.g., algebra), it is difficult to truly determine what occurs during non-pedagogical modes. While it was casually observed that these dialogue moves ranged from after school activities to study strategies, the exact proportion of each is currently unknown. There are two options for expanding our analysis of off topic dialogue moves. One is to create a new coding scheme that makes finer distinctions in off-topic. The other option, which we discuss next, is to use a text analysis tool to look for text-level features that might show what’s going on inside off topic. 3.2 LIWC analysis of off topic modes. The comparisons between the Off Topic and Scaffolding modes along LIWC dimensions were done using a series of paired t-tests. The means (values in % of words), standard deviations, t-values and effect sizes (Cohen’s d) of each comparison can be found in Table 3. The analysis was conducted on both tutor (T) and student (S) contributions during the tutoring sessions. These same comparisons were also made using the Wilcoxon’s signed-rank test, as a normal distribution of scores cannot be assumed. However, those results very closely mirror the results of the paired t-test, and so only the paired t-test comparisons are presented here. A Bonferroni correction was not used in this analysis; as this is exploratory research that will be used to orient future research, the authors felt that a conservative correction would result in a loss of critical, if minor, information. In sum, the results here seem to indicate that every significant category difference favored the Off Topic mode, with the exception of when students use ACHIEVEMENT and FUTURE words. Table 3. Occurrence of LIWC category words. In general, it may be said that there is more to off topic conversation than simple socializing and “time wasting.” Instead, there is a complex dynamic within the Off Topic mode, where the tutor and student are achieving a balance of work-related discussion and subtle socializing. Though off topic conversation seems to be an unlikely place for WORK words to arise, the Off Topic mode contains significantly more WORK words than does Scaffolding. At first, this seems counterintuitive; however, this difference may be due to the way in which work is talked about in these two different modes. In Scaffolding, work may not be discussed on a superficial level, as this is where work is actually performed. Off Topic may be a place to discuss work on a superficial level, without content. The authors of LIWC list examples of WORK words being things like “class” and “graduate,” so perhaps Off Topic conversation is a place where the student and tutor talk generally about schoolwork and homework. This could be supported by the significantly elevated use of HOME words in Off Topic as opposed to Scaffolding. Perhaps HOME is being discussed in the context of homework. Word categories that would hint at a more social use of HOME words, like FAMILY and FRIEND words, do not significantly differ from Off Topic to Scaffolding. In fact, those two categories make up less than 1% of the words in both modes. This suggests that HOME words are not being used in a social context, but rather, in work-related discussion. This is similar to the results of [8]; in that work, tutors engaged in “social talk” with their students, which involved discussing learning strategies. Our tutors may be doing something similar during their “social talk” (Off Topic). It may be the case that expert tutors use off topic conversation to discuss more general studying strategies. This may also explain why TENTATIVE words and NONFLUENCIES occur significantly more in Off Topic than they do in Scaffolding on the part of the tutor. Rather than overtly stating problem solving and study suggestions, the tutor may use TENTATIVE words and NONFLUENCIES to lessen the face threatening nature of these suggestions. In one tutoring session, the tutor tells the student that, although the teacher assigned the odd problems for homework, she should work additional problems to get more practice. While this can be portrayed as a mere suggestion, it is alluding to the student’s deficient abilities and need for further practice. Suggesting additional work like this may induce the tutor to use more words like “maybe,” “perhaps,” and “umm”, given that barking orders is unlikely to lead to the completion of this additional work. Although the LIWC results do suggest the presence of work-related discussion, there is an undeniable socio-emotional factor involved in off topic conversation. Generally, the Off Topic mode contains more POSITIVE EMOTION words than does Scaffolding; this aligns with work by [16], which found that the emotion “happiness” was much more likely to occur with tutor and student off topic conversation than other portions of the tutoring session. These off topic conversations, then, may be used as a sort of short “break” from the tutoring material that restores positive emotion and builds rapport between the tutor and student. This positive emotion and rapport building may act as a buffer against some of the direct, negative feedback that expert tutors give [17]. However, other affective LIWC categories like AFFECTIVE PROCESSES, NEGATIVE EMOTION words, and ANXIETY are not used in significantly different amounts between Off Topic and Scaffolding. This may be indicative of students’ greater comfort in discussing positive emotions, as their negative emotions are likely tied to past and current academic struggles and failures. However, it may instead reflect that the purpose of Off Topic is not to discuss the emotional state of the student during learning. Which may also mean that off topic conversation does not necessarily include “pep talks”, as [8] suggest. In addition, tutors do not use a larger amount of ACHIEVEMENT words Off Topic, suggesting that they are not trying to overtly bolster students’ feelings of confidence. Instead, rapport seems to be built in more subtle ways, such as by using more SOCIAL PROCESSES words like “we” and “us”, and perhaps by using higher-level strategies of rapport building like humor and solidarity statements. 4 Conclusion. These two methodologies seem to converge and support our initial casual observations that off topic conversation is more than simply social talk or irrelevant ramblings. Off topic does not seem to be simply an “other” category. We feel that the evidence supports claims that off topic dialogue may serve motivational uses, to discuss more global pedagogy or study skills [8], build rapport [11], and in certain cases serve as a much needed mental break from tutoring. Exploratory studies often are limited by the use of a single methodology. This study benefits from the use of two distinct approaches to investigating off topic conversation in tutoring. One approach utilized pre-existing coding schemes to determine whether the activities generally assigned to pedagogical conversation are occurring to any degree during off topic conversation. The second approach brings in a new and different analysis of the dialogue. This approach allows for a more objective look at the data, whereas results from the pre-existing coding scheme could be critiqued as simply an artifact of our coding methodology. Given that both approaches showed off topic conversation to be complex and multidimensional, this convergence gives support to further exploration. These results have given an improved depiction of what occurs during off topic conversation; however, we are still only able to speculate on its anatomy. Future research will reveal the true role and importance of off topic conversation in tutoring. These findings have afforded a framework to begin future, more directed investigations. First, they have allowed us to determine whether further exploration is even worthwhile. These exploratory findings suggest that further investigation is, in fact, warranted. Second, the findings here can be used as the basis for future coding schemes. Whether a manual coding scheme or an automated methodology, such as probabilistic topic models [19], is employed, either can be used to determine the proportion of off topic conversation that is dedicated to global pedagogy, building rapport, social topics, and possibly even irrelevant ramblings. Through the accurate depiction of off topic conversation, its most advantageous features can be applied to the building of ITSs. While implementing off topic conversation into ITSs under the current conception of discussing social topics seems peculiar, incorporating those pedagogical and rapport- building dimensions of off topic conversation would be a useful addition. So while an ITS may never form a deep, meaningful social bond with a student, it could help to increase learning in a broader scope than simply the present topic. ITSs such as MetaTutor already incorporate strategies similar to our proposed global pedagogy during learning [20-22]. By incorporating the global pedagogy of expert tutors as well as more local pedagogical strategies, ITSs can give greater aid to struggling students in over many disciplines. Further analysis will be needed to know the exact nature of off topic conversation and its potential usefulness in building ITS systems. Acknowledgements. This research was supported by a grant awarded by the U. S. Office of Naval Research (N00014-05-1-0241) and the Institute of Education Sciences (R305A080594). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Office of Naval Research or the Institute of Education Sciences."