1. In terms of probability, this means that the occurrence of X and the occurrence of Y in the same transaction are independent events, hence X and Y are not correlated. It is easy to show that the lift is 1 exactly when added value is 0, the lift is greater than 1 exactly when added value is positive and the lift is below 1 exactly when added value is negative. Further AV(X→Y) tends towards 1 when lift(X→Y) tends towards infinity, and AV(X→Y) tends towards -1 when lift(X→Y) tends towards 0. Note that lift  X→Y =∣X ,Y∣. n∣X∣.∣Y∣ so the result is proportional to n, the total number of transactions. As opposed to cosine, lift does not hold the null-invariant property. 2.4 Typical values for cosine and lift. To fix ideas let us look at typical values for these measures Suppose that among n transactions, m contain either X or Y or both, with m ≤ n, and that n - m transactions contain neither X nor Y. First consider the case where all m transactions contain both X and Y. Then: cosine(X→Y) = 1. Conversely, it is easy to show that cosine(X→Y) = 1 implies that all m transactions contain both X and Y. As for the lift, lift  X→Y =m . n/m. m=n /m . So if m = n, lift(X→Y)=1. If m = ½ . n, lift(X→Y)=2 and so on. Consider now the case where 90% of the m transactions contain both X and Y, and 10% of the rest contain X but not Y. Then: FORMULA_2. Now consider again the case where 90% of the m transactions contain both X and Y, but 5% of the rest contain X and not Y, and the other 5% contain Y and not X. In other words X and Y are evenly spread among the transactions containing either X or Y but not both Then: FORMULA_3. Table 1 summarizes further results. Lines should be read as follows: (a,b,c) means that a % of the m transactions contain both X and Y, b% contain X and c% contain Y. Therefore (75, 100, 75) means that 75% of the m transactions contain both X and Y and that the remaining 25% contain X but not Y (X is present in 100% of the transactions and Y in 75% of them), while (75, 87.5, 87.5) means that X or Y are evenly spread among the 25% of the remaining transactions. Discussion: In the case of strong symmetric association rules, which means that ∣X∣ , ∣Y∣ and ∣X , Y∣ are all big numbers close to n, cosine and lift do not rate rules the same way, as pointed out in [7]. In this case, cosine performs better than lift. Added value and lift rely on probabilities, which make more sense when the number of observations is large. Further we see also that lift and added value, unlike cosine, depend on the number of transactions that contain neither X nor Y. In the educational field it is not clear that these null-transactions should play a role. We come to the same conclusion as [3]: double check the interestingness of association rules with cosine first, then with lift if cosine is not conclusive. Table 1 suggests that a value around or below 0.65 is rejected by cosine : as we can see 0.66 corresponds to the lowest threshold with 50% of common values (50, 75, 75). In case of contradictory results then decide using the information that these two measures represent. Table 1. Typical values for cosine and lift, where the 3 figures of the first column show the percentage of transactions containing X and Y, X and Y. 3 Improving Teacher Support: Case Study. The present case study describes a standard use of a Learning Management System (LMS) for providing additional resources to students in a face-to-face teaching context. Teachers want to figure out whether students use these resources and possibly whether their use has any (positive) impact on marks. The LMS Moodle [10] was used in the context of the course Formal Basics of Computer Science for first semester students enrolled in the degree “Computer Science and Media” at the University of Applied Sciences TFH Berlin during Winter Semester 2007/08. The cohort of 84 students enrolled in that course is divided into two groups. Students had a 3- hour weekly lecture. It includes formal teaching where concepts are explained, paper/pencil exercises to apply these concepts, and exercises discussed on the spot. To pass this course students take two exams. The first one takes place about 8 weeks after the beginning of the semester and the second one at the end of the semester. The present case study uses data gathered till the first exam. Moodle is used for posting lecture slides and accessing the following extra resources: - Book: a link to the homepage of the text book “Introduction to Automata Theory, Languages and Computation” used for this course [4]. From this homepage students could access a set of exercises with solutions. - DP: extra reading “Design Patterns for finite automata” [6]. - Jflap [5], a software to practice automata construction. - Ex1, Ex2 ... Ex7 : a set of seven extra self-evaluation exercises. One exercise is published in Moodle each week right after the lecture. The last exercise Ex7 was put 2 weeks before the exam. - TrEx01 and TrEx02 : two sample exams, published 3 weeks before the exam. - TrEx01S and TrEx02S, the solutions to the sample exams, published 10 days before the exam. The use of Moodle, its additional resources and its self-evaluation exercises were not compulsory though strongly encouraged. Therefore for the teacher it is quite important to know: what do students do with those extra resources? What do they view? Is there any relationship between their use of these resources and their result in the exam? To answer these questions we have used solely the log data available in Moodle. Log data gives, for each resource and each student login, when the resource was accessed. It also gives, for each exercise and each student login, whether the exercise has been attempted, and whether the first trial was a success or not. 3.1 Exploring Data. From the 84 students enrolled in the course, 81 were enrolled in Moodle. The case study considers only those 81 students. From them, 52 passed the exam, 8 failed and 21 did not come. From the 60 who took the exam, statistics on their marks is given in the first line “General” of Table 4. Did students do the exercises? Table 2 summarizes the figures. Lines should be read as follows. For example line 2 means that 46 students did not attempt exercise 1, 21 students gave a correct answer on their first trial and 14 gave a wrong answer on their first trial. One notices that as time goes there is always less students attempting exercises. Table 2. Exploring exercises among all students. Table 3. Viewing resources. Did they access other resources? Table 3 summarizes the figures. The first column says that 59 students have viewed the first sample exam, the second column says that 52 students have viewed the solution of the first sample exam, and so on. One extra column has been added. AtLeast1Ex says that 38 students have attempted at least 1 exercise. What are the results in the exam for each group of Table 3? Table 4 summarizes the results. Two extra lines have been added. NoEx shows the results for students who have never attempted any exercise. AtLeast1Ex shows the results for the students who have attempted at least 1 exercise. Table 3 and Table 4 suggest that the standard preparation for the exam is to look at sample exams and/or their solutions. Students who invest some more time with extra material tend to have better marks. The biggest positive impact on the marks is given by DP. Table 4. Viewing resources and marks in the exam. Table 3 and 4 confirm the expected outcome. Table 4 also shows something that was not known before: students tend to access a sample exam more that its solution. This first exploration gives also directions for more investigation: If students attempt exercise 2, do they also attempt exercise 1? If they look at the solution of a sample exam, do they also look at the sample exam itself? This kind of questions can be investigated with association rules. 3.2 Association Rules. We begin with association rules tackling sample exams. The following rules again confirm the expected finding. If students look at the solution of a sample exam, they look also at the sample exam itself. Further, if they view the second exam, then they also view the first one. The other way round does also hold, but with a slightly lower confidence. Table 5. Association rules for sample exams. Results are similar when rules are mined restricting the population to the students who came to the exam as shown for the first sample exam in the lines in italic of Table 5. Notice however that the lift diminishes as the rules become stronger [7]. Table 2 gives a direction for further rules to investigate: Is there any association between attempting exercise i and exercise j? One expects that many students enthusiastically have begun with exercise 1 at the beginning of the semester and then slowly have stopped doing them, till exercise 4 where a bunch of students just keep doing them. The rules we have obtained confirm this interpretation. We have mined these rules restricting the data to students who have attempted at least 1 exercise, which means 38 transactions. Rules with a high confidence relate attempting exercise 2 and exercise 3, exercises 4 to 7, as well as exercise 1, and not attempting exercises 2 to 3, or not attempting exercises 4 to 7. Table 6 presents a sample of the extracted associations. Note that !Ex2 means that exercise 2 has not been attempted. So the first line says if students don't attempt exercise 2, then they don't attempt exercise 3. Table 6. Association rules for attempted exercises. For all these rules, except the last one, cosine and lift rate associations the same way. The drop between attempting exercises 1 to 3 and attempting the others has led us to investigate the marks of this population. Surprisingly, their average mark is smaller than for all students who have attempted at least 1 exercise. Table 7. Attempting exercises 4 to 7 and marks in the exam. As for the other resources, were they consulted by the same students? We have looked at associations between DP, Jflap, Book and AtLeast1Ex considering the full population and show two rules found in Table 8. Here lift does not confirm the non-interesting rating given by cosine. As before, !DP means that the resource DP has not been viewed. Table 8. Association rules for the other resources. Keeping in mind the meaning of measures can help deciding what to do with an association. Let us consider the last rule of Table 6. Cosine indicates that, among the students who have not done Ex5 nor Ex6, over 60% had done Ex1 (consult the typical figures in Table 1), while lift indicates that the proportion of students who have not done Ex5 and Ex6 is not larger in students who did Ex1 (which represented 43% of the students, according to table 2) than in all students. However, from a pedagogical point of view, the case of students who did not attempt Ex1 is not relevant for this analysis. Therefore the teacher would probably find it useful to keep this rule, hence following cosine, though lift gives here an interesting complementary information. Let us now consider the second rule of Table 8. Cosine gives us the following information: among the students who consulted the Book web site, the extra material on Design Patterns (DP) and done at least one exercise, less than 40% used Jflap (refer to the typical values in Table 1). The lift gives us the following information: the proportion of students who looked at Jflap is higher among the students who looked at the Book web site, the DP material and done at least one exercise than in the whole student population. Given the very small number of students (as we can see from the support, there are 10 students who satisfied these three criteria on the left hand side of the rule), it is prudent to follow cosine and reject the rule. It is interesting to note though that if the cosine and lift had given similar values, but with a higher number of students satisfying the three criteria in the left side of the rule, it would then have been advisable to follow the lift and retain the rule. Conclusion. Association rules are useful in Educational Data Mining for analysing learning data. This technique requires not only that adequate thresholds be chosen for the two standard parameters of support and confidence, but also that appropriate measures of interestingness be considered to retain meaningful rules and filter uninteresting ones out. In this paper we revisited and gave an interpretation for two interestingness measures: cosine and added value (which we saw is closely related to the lift). We presented typical values for these measures. An association rule is rated uninteresting by cosine if its value is around or smaller than 0.65, whereas it is rated uninteresting by the lift if its value is around or under 1. We came to a similar conclusion as in [3]: the interestingness of a rule should be first measured by the cosine, then with lift if cosine rated it as uninteresting. In case of conflict between the two measures, the user needs to take into account the intuitive information provided by each measure and decide upon it. The case study presented in the paper depicts a standard situation: a LMS provides additional resources for students in complement to the face-to-face teaching context. Teachers want to figure out whether students use these resources and whether these have any (positive) impact on marks. Few association rules (without being strong symmetric ones) came out with a contradictory result for cosine and lift. Keeping in mind the intuition behind cosine and lift helped to decide whether to discard these rules. Another conclusion of this work is that common LMS are far from being data mining friendly. Log data concerning access to resources and test data are not stored the same way for example. Complex data manipulation is needed to get all data consolidated in a useful form. LMS present statistics, however these are very limited. LMS should be enhanced with a special module with good facilities for exploring data. Data mining tools for LMS should have an association rules module with good facilities to choose the attributes to derive association rules for and with the two interestingness measures cosine and lift.