Assessment on collaborative student behavior is a longstanding issue in user modeling. Nowadays thanks to the proliferation of online learning and the vast amount of data on students’ interactions this modeling issue features some alternatives. The purpose is not to depend on teachers or students assessments, which either requires management effort difficult to assume (due to some students-per-teacher ratios) or depends on individual motivations (i.e. student willingness on providing explicit feedback related to collaboration). In our research we have shown that based on frequent and regular analysis of those interactions it is feasible to obtain collaborative assessments that concurs with expert valorizations. This approach relies on data mining and machine learning techniques, which are applied to infer collaborative significant student’s characteristics such as regularity, in terms of activity and initiative, and student acknowledgment of fellow-students. The advantages of the approach are to obtain domain-independent assessments, applicable in different learning management systems and exploitable over different courses and learning settings without the teacher involvement in the analysis process. The method has been developed from a collaborative experience involving hundreds of students over 3 consecutive year-courses. Here we focus on discussing the improvements on measurements provided during a new collaboration learning experience of this academic year.
"INTRODUCTION. Student collaboration assessments can improve learning and motivate students (Swan et al., 2006), albeit they must come from a frequent and regular student collaboration analysis (Johnson & Johnson, 2004). Thus, students’ interactions, mainly communication interactions, should be frequently analyzed to provide a timely collaboration assessment, which can be used by students and teacher to improve the collaboration learning process. To offer frequent and timely assessment the expert-based analysis approach is almost unaffordable (Bratitsis et al., 2008), only researchers who used quantitative student interaction and automatic methods were able to cope with the requirements (Gaudioso et al., 2009). In current LMS-based e-learning scenarios communication is mostly done through forums, and that is why they have been extensively used to reveal relevant students’ collaborative characteristics (Dringus & Ellis, 2005). Once the collaboration assessments were inferred, most related research has advocated for displaying assessment to students and teacher (Bratitsis et al., 2008, Gaudioso et al., 2009). Our approach is based on frequent and regular analysis of learners’ interactions to obtain collaborative assessments that concurs with expert valorizations. To this end data mining and machine learning techniques have been applied. The advantages of the approach are to obtain domain-independent assessments, applicable in different learning management systems and exploitable over different courses and learning settings without the teacher involvement in the analysis process (Anaya & Boticario, 2011). To support the assessment we have previously compared clustering and quantitative metric approaches and finally proposed a quantitative Metric Approach, which draws on a range of decision tree algorithms to inferring quantitative indicators such as regularity, in terms of activity and initiative, and student acknowledgment of fellow- students (Anaya & Boticario, 2010; Anaya & Boticario, 2011). From the lessons learnt after three years of experimentation with hundreds of students in a real collaborative learning experience we have organized this year- course experience in which there are several improvements on the Metric Approach and display strategies. Following the collaborative learning experience is introduced. Next, the Metric Approach steps are summed up. Then, collaborative assessments results and displaying strategy are commented, and to conclude the future analysis. COLLABORATIVE LEARNING EXPERIENCE. We have offered to students of Artificial Intelligence and Knowledge based Engineering (AI-KE) at UNED (2010-11) a collaborative e-learning experience using dotLRN platform, which provides documentation support and forums to collaborate. The experience is divided into two phases. In the 1st phase the students perform an individual task, which allows them to participate in the 2nd phase, where they are grouped into three-member teams and every team has to carry out five collaborative tasks. Team members’ communications are managed through group forums. METRIC APPROACH. From mining techniques applied on collected data from forum interactions the Metric Approach, which is based on machine learning techniques, proposes a mathematical formula that uses quantitative indicators to measure students’ collaboration (Anaya & Boticario, 2011). That formula is refreshed on a regular basis to cope with the course pace. Here we sum up the main issues involved: • Twelve quantitative statistical indicators are proposed (see Table I). These indicators are related to relevant student’s characteristics: initiative, activity, regularity and acknowledgement. • A set of decision tree algorithms (BFtree, DecisionStump, Functional trees, J48, Logistic trees, NBtree, Random tree, REPTree, Simple Cart) are applied to research the relationship between those indicators and students’ collaboration labels supported by expert-based analysis (required for the configuration phase not any more on different courses). The research shows that the most collaborative-related indicators are (Anaya & Boticario, 2010): the regularity of the student initiative (L_thrd) and activity (L_msg), and the students’ acknowledgment (N_r_msg). • A metric (mathematical formula) is built from the above quantitative statistical indicators (Metric = L_msg + N_reply_msg + L_thrd). This metric is selected because it outperforms (i.e. less error and better discrimination of collaborative levels) other metrics, which consider alternative indicators, data set filters and normalized additions. Table I: Quantitative statistical indicators of the student interactions in forums. COLLABORATION STUDENTS ASSESSMENTS. The collaborative learning experience started on February the 21st 2011. 100 students signed up for the collaborative learning experience. 43 of them finished the 1st phase and 15 teams were created. The 2nd phase started on March the 10th 2011 and finished on April the 17th 2011. All along the 2nd phase the quantitative statistical indicators were measured and the students collaboration metric were calculated on a daily basis. Collaboration assessments were displayed to 11 teams and statistical indicators were displayed to 6 teams out of them. 4 teams made up the control group. From the lessons learnt in previous experiences we opted for simplifying displayed results (see Fig. 1). Fig. 1. Assessments displayed to team-members. FUTURE ANALYSIS. Students are currently facing the final exam and they are answering an evaluation questionnaire. From this data we will be able to compare the usefulness of the metric and displaying strategies, and the expected improvements with respect to previous collaborative learning experiences, as it was reported in (Anaya & Boticario, 2011)."
About this resource...
Visits 155
Categories:
0 comments
Do you want to comment? Sign up or Sign in