"1.1 Cognitive and Social Conflict in CSCL. Within this paper, we focus on cognitive and social conflict within computer supported collaborative learning, categories found to have relevant impacts on student learning within non-computer-mediated collaborative learning. Studies from the 1970’s have shown that cognitive conflict promotes cognitive development [11,13,21,23,25], in line with Piaget’s [16] writings on the equilibration process. Piaget claimed that one source of progress in the development of knowledge is found in the imbalance that forces a subject to seek new equilibrium through assimilation and accommodation. Many researchers have found results indicating that cognitive conflict and learning emerges from the process of collaboration, when students mutually engage to co- construct shared knowledge [5,14,19]. In fact, Moshman and Geil [12] and Kruger [10] have argued that the conceptualization of cognitive change as either a process of conflict or a process of cooperation is a false dichotomy, claiming that productive cognitive conflicts take place solely within a cooperative context, and not via competition or interpersonal conflict. In Moshman and Geil’s view, productive cognitive conflict does not emerge from students arguing in favor of their own views, but from co-constructing a consensus solution. Similarly, Howe [7] suggested a separation between types of conflict that involves transactive [3] dialogues (e.g. cognitive conflict) and interpersonal conflict that involves aggression. These studies argued that these two types of interaction occur in distinct groups, depending on students’ gender and temperament. However, other studies have provided evidence suggesting that cognitive conflict does not solely occur in purely collaborative and consensus-based process. For example, Arsenio and Lover [2] and Shantz [22], give evidence that the conflict of ideas often leads to interpersonal conflict. Hence, it appears to still be an open question whether productive cognitive conflict, and the learning that emerges from it, only occurs when students show collaborative behaviors, or whether it still occurs in conjunction with interpersonal conflict. This question is especially relevant within the context of online collaborative learning. Online learning has different affordances than the face-to-face learning settings where much prior collaborative learning research has taken place – in particular, online collaboration, due to the anonymity potentially afforded, is prone to a very high rate of insults, often called “flaming” in the online collaboration literature (e.g. O’Sullivan & Flanagin [15]). To investigate these questions, we consider the relationship between learning and the different behaviors students display in anonymous online collaborative learning, focusing on the insults indicative of interpersonal conflict. Our hypothesis is that the cognitive conflict which occurs in online learning is highly likely to produce interpersonal conflicts, demonstrated by insults; however the online medium may also reduces the social consequences associated with insults, meaning that insults may not impede learning gains. 2 Analysis of Learning Associated With Speech Acts. 2.1 Methods. Twenty four sixth-grade students from a suburban elementary school near Pittsburgh, PA, participated in this study. The study was conducted in a genuine setting of learning, involving authentic learning materials. Because one of the students did not use the chat interface during the two lab days, that student was excluded from analysis (that student’s partner was still included in the sample because this student used the chat interface to discuss the problem solving and complain about his partner’s lack of interactivity), and the sample was reduced to 23 students. Each student made, on average, 84.3 utterances, for a total of 1940 utterances. Each student used a mathematics tutoring program covering problems on fraction addition, subtraction, multiplication, and division [9], in collaborative pairs mediated through TuTalk. TuTalk [8] is a collaborative problem solving interface that include two online panels: a chat, and a collaborative interface for the problem solving built in the CTAT authoring tool [1]. Figure 1. Problem-solving interface [9]. The students worked in their school lab computer, in pairs, using TuTalk, with their chat dialogues and problem solving contributions (within the interface) logged for later analysis. The arrangement of the lab was designed so that the students could not easily talk with their pairs outside of the chat interface, with the identity and the seat location of the collaborating pairs hidden from their partners. Each student individually took nearly isomorphic pre-tests and post-tests, covering knowledge of the material covered in the tutor, during a 30 minute period during on separate days from tutor usage. The students collaborated in learning fractions within TuTalk within two lab sessions, each lasting 45 minutes. This design enabled us to investigate the student’s knowledge gains based on the pre- and post-tests, and to analyze students’ collaborative and individual learning behaviors. 2.2 Analytical Method. In analyzing these dialogues, we divided student behavior into a selected number of categories relevant to our analyses. Cognitive communicative categories were split in accordance with Youniss and Damon’s [26] interpretation of Piaget’s views on social relations in the individual construction of knowledge, where cognitive conflict and knowledge construction can occur either through disagreement, where one student perceives a misconception or other error in his partner’s thinking and disagrees, attempting to express why it is wrong (called disagree with concept in our coding scheme). Within this type of disagreement, a student is arguing in favor of his or her own views, an ultimately competitive act. In one example, one student said, “i dont think thats the common denominator”. By contrast, a student may also refine their partner’s ideas by expressing their perspective on an idea, and informing the partner as to their beliefs (called inform belief in our coding scheme), attempting to co-construct a solution – a more collaborative manner of expression. One example of this within our corpus is, “the common denominator is 54”. These two categories are shown in Table 1. The two categories do not form an exhaustive list of possible cognitive communicative acts, but are particularly relevant to the analysis presented here. A fuller taxonomy of speech acts is given in the first author’s doctoral dissertation [17]. Table 1. Description of the cognitive communicative categories. Table 2 Description of social communicative categories. The offer collaborative act and insult categories are social communicative categories (Table 2). A student who offers collaborative act offers to do something to forward the problem-solving goals, generally without having specifically been asked to do so. As such, offer collaborative act is a reflection of the peers’ social collaboration. By contrast, an insult reflects interpersonal conflict within the dialogue, and is in contrast to social collaborative behavior. This set of four categories was coded by the first and sixth authors. Both authors coded a subset of 225 utterances made by students during collaborative learning. Cohen’s [4] Kappa was 0.80, indicating good inter-rater reliability. Afterwards, the protocol analyses were based on the first author’s codes for the entire corpus. We also developed a machine-learned model that was able to accurately code these categories, discussed later in the paper – however, for this analysis human coding was used, as a tractable gold- standard. 2.3 Results. We analyzed the correlations between pre- and post-test learning gains and the frequency of each category of our coding scheme in each pair’s dialogue. The overall pattern of results is shown in Table 3. As can be seen, the number of inform belief speech acts a student made or received was not significantly correlated with learning gains, respectively t(22)=-0.31, p=0.75, t(22)=-0.10, p=0.92 (all tests reported are two-tailed). The number of offer collaborative act speech acts a student made or received also was not significantly correlated with learning gains, t(22)=0.00, p=0.99, t(22)=0.64, p=0.52, for a two-tailed t-test.) Hence, neither of the two cooperative behaviors coded were associated with significantly higher learning gains. By contrast, the two non-cooperative behaviors were associated with positive learning – but only in the student being non-cooperative. The amount a student disagreed with concept was associated with statistically significantly higher learning gains for the disagreeing student, r=0.53, t(22)=2.93, p<0.01, but not for their partner, t(22)=-0.38, p=0.70. The disagree with concept act has the intention to alter the peer’s reasoning with conflicting ideas, but appears to have been more of a marker of the disagreeing student’s learning than a learning opportunity for their partner. Interestingly, the amount a student made insults was also associated with significantly higher learning gains, r=0.70, t(22)=4.53, p<0.001, but receiving insults from another student was not associated with higher learning gains, r=0.26, t(22)=1.26, p=0.21. Hence, students who acted in ways that create or indicate interpersonal conflict appeared to have higher learning gains in this study. Students who behaved in a more cooperative fashion did not appear to have higher gains. However, the mechanism explaining this result is not clear. Did students learn more because they allowed cognitive conflict to move into interpersonal conflict, or did students engage in interpersonal conflict because they had learned more than their partner, and were impatient with them? It is possible, in particular, that the anonymity of the online learning system facilitated students who had just learned the material in choosing to insult their partner rather than help them. Table 3. The relationship between learning gains and different dialogue acts (p values shown). Statistically significant results (p<.05) in boldface. 3 Development of Collaboration Behavior Detectors. Having coded a significant number of utterances, the next step was to determine whether it would be possible to develop a machine learned model that could automatically detect these four categories. Such a detector could be used to drive automated interventions by the CSCL environment. (Possible interventions will be discussed in the discussion section). These four categories were combined with additional data coded with twenty-eight other categories, representing a wide span of possible dialogue acts within collaborative learning. The full coding scheme is discussed in detail in the first author’s doctoral dissertation [17]. In total, there were 170 utterances coded with the 4 speech acts discussed above, and a total of 1940 utterances coded with the full set of 32 speech acts. Rosé et al’s [20] TagHelper tool kit was used to develop a machine learned model that could identify the set of speech acts in students’ utterances. TagHelper provides text classification services, designed for use with several languages, and access to a variety of metrics for validating model goodness. It also automatically distills features previously found to be useful for linguistic analyses, such as bigrams and the presence of “stop” words. We used TagHelper to develop models, and to quantify our success in terms of agreement with the hand-coded gold standard corpus. The Naïve Bayes classification algorithm was selected, and applied to the 32 speech acts on the dialogue data. The Kappa [4] statistic was used, in combination with 10-fold cross validation, to assess reliability of the model’s coding. Non-stratified cross-validation was used, under the assumption that multiple utterances by a single student on a single topic are unlikely to be highly correlated to each other (as opposed to other types of behavior, where a student’s responses may be more characteristic), especially when all students are discussing the same mathematical topics. For instance, terms used to discuss fractions, or to disagree about fractions, are likely to be similar between students. The model was successful at classifying student utterances. Within the whole set of 32 speech act categories, kappa was a respectable 0.65. Within the set of four utterances that were previously thought to be particularly relevant for modeling and understanding learning (two were indeed found to be statistically significantly associated with learning), kappa was an excellent 0.91. This was better than our human judges’ degree of agreement, suggesting that the model was highly successful. 4 Discussion and Conclusions. In the previous section, an automated detector of a variety of speech acts was presented and shown to be reasonably effective at distinguishing between a variety of speech acts, including the four categories discussed in detail in this paper: inform belief, disagree with concept, offer collaborative act, and insult. With Kappa values between 0.65 (all categories) and 0.91 (categories discussed in this paper), it seems quite feasible to use the model for detecting and responding to different types of speech acts. However, using the model to drive appropriate interventions depends on understanding the implications of each type of speech act, which leads to a need for analyses such as the one presented here. In the analysis in section 2 of this paper, learning gains were correlated with speech acts. Understanding this gives us an important first piece in the puzzle of deciding how a CSCL system should respond to those acts. Developing a full understanding of what prompts different speech acts will help us even further. Within this study, learning gains were positively related to interpersonal conflict. Arsenio and Lover [2] and Shantz [22] previously found, in face to face collaboration, that the conflict of ideas can lead to aggressive behavior, including the types of interpersonal conflict observed here. In those studies, the aggressive behavior harmed students’ interpersonal relationships. However, the anonymity of communication in our study may have enabled students to insult each other with lower interpersonal cost, eliminating one of the negative factors associated with aggressive behavior in collaborative learning. In broader usage of such a CSCL system, this anonymity could persist within internet usage while not persisting in classroom usage (because over time, students can determine who they are collaborating with in a classroom, eliminating anonymity). This does not mean, however, that insults are to be encouraged. Insults are clearly perceived as problematic in online communication (e.g. O’Sullivan and Flanagin [15]), and may be associated with the abandonment of usage of online learning environments (cf. Reinig et al. [18]). In general, this work supports the hypothesis that positive cognitive conflict can coincide with interpersonal conflict. It is not at all clear from our results that the interpersonal conflict had a positive impact on cognitive conflict or learning – for instance, it may have been a side-effect of one student’s greater learning, with no positive impact on the other student. Studying this issue in richer depth will require a combination of methods, including time series analysis on a significantly larger corpus of data, and perhaps experimentally manipulating interpersonal conflict via not transmitting students’ insults, in order to determine insults’ causes and impacts on learning. One clear implication of our results is that insults and interpersonal conflict play a prominent role in collaborative learning, which cannot be safely ignored. An overly harsh response to student insults may also interfere with the positive learning that insults appear to be associated with. One approach may be to attempt to develop designs which guide students in moderating their comments to others, without disrupting the cognitive conflict which insults appear to be associated with. However, if insulting another student produces pleasure for the insulting student and increases the insulting student’s desire to persist in the use of a learning environment [cf. 24], it may be feasible for a CSCL environment to automatically strip out insults from the text the insulted student actually receives. Further research on how software that supports CSCL can optimally handle insults and other interpersonal conflict behaviors, given the ability to detect those acts, appears to be warranted. 5 Acknowledgements. This work was funded in part by NSF grant REC-043779 to ""IERI: Learning-Oriented Dialogs in Cognitive Tutors: Toward a Scalable Solution to Performance Orientation""."