As part of an ongoing project where SWoRD, a Web-based reciprocal peer review system, is used to support disciplinary writing, this study reports machine learning classifications of student comments on peer writing collected in the SWoRD system. The student comments on technical lab reports were first manually decomposed and coded as praise, criticism, problem detection, solution suggestion, summary, or off-task. Then TagHelper 2.0 was used to classify the codes, using three frequently used algorithms: Naïve Bayes, Support Vector Machine, and a Decision Tree. It was found that Support Vector machine performed best in terms of Cohen’s Kappa.
1. Performance of Classification Approaches. Table 1 shows the experimental results. With the training set, the highest performance measured as Cohen’s Kappa was achieved with SVM. Although the performances of the models were a little decreased with the test dataset, the highest Cohen’s Kappa was still found with SVM. This result is consistent with other text classification studies. To identify the source of errors or performance reduction, we analyzed the confusion matrices provided by TagHelper. All the three approaches revealed consistent problems: Praise comments were correctly categorized (80% correct vs. 20% incorrect for example in SVM) while problem detection comments tended to be confused with solution suggestion. Interestingly, problem detection was categorized as solution suggestion more than solution suggestion was categorized as problem detection. 5 Conclusion. This study has presented machine learning technologies applied for classifying peer comments in writing. As demonstrated with TagHelper, the machine learning technologies were found to be useful to categorize student comments on peer writing. Especially SVM achieved a noteworthy performance. Another important result was that the machine learning technologies, especially SVM, was good at categorizing tonal information into praise vs. non-praise. Finally, it should be noted that a hidden benefit of the text classification technologies seems to help researchers develop coding schemes precisely. Obviously, one of the benefits in using text classification technologies is automatically categorizing a large corpus of peer comments. This may be important in reciprocal peer reviewing of writing. Along with the class size, students tend to exchange an exponential amount of comments. Thus, automatic corpus coding technologies may be greatly helpful to help instructors to monitor reciprocal peer reviewing of writing in their classes. In addition, the current findings have been used to develop an intervention for student reviewers. Thus, based on the helpful comment model developed with TagHelper, student comments can be classified online before they are passed to their authors in order to help students generate constructive comments.
About this resource...
Visits 379
Categories:
6 comments
Do you want to comment? Sign up or Sign in
2023/07/15 10:17
2020/06/11 14:11
2020/04/19 14:14
nice post
2020/01/31 18:59
nice post
2019/12/17 16:38
After 8 years as a sleep expert in a couple of mattress stores and seeing thousands of people experiences with many brands, he decided to write and put all of these years of experience into help guides so others can make the right decision when buying sleep products without having to suffer sleepless nights as a result of choosing the wrong mattress. regards
2017/05/24 12:44
Thank you so much for sharing this information. You have wonderfully created this article and it has amazing substance and is carefully formed and even I am revived by your composed style as well. Go for this cheap essay writing service to having a wonderful assistant in writing and even you can get the guidelines too.
Nice post
coffsharbourcarpetcleaning