"1. Table 1. Description of the datasets for investigating pace rank consistency. For each dataset, we sorted the students twice, according to their pace in the relevant sub- groups (the student with the highest pace was ranked as ""1"", the student with the second- highest pace was ranked as ""2"", and so on). These two ranks were correlated using Spearman's rho (ρ) and Kendall's tau (τ), two common alternatives for non-parametric correlation coefficients ([-1,1]) which are often being compared, however without a sharp recommendation towards neither of them [9, 12, 17]; it is known that the Kendall's coefficient is usually lower than the Spearman's. 4 Results. Day/Night Consistency. Results for Dataset1M and Dataset1P, in which day/night situation was examined in the two learning modes, are given in Table 2. It might be concluded from the results that there is a significant relatively high correlation between pace ranks between day and night in both modes. It was also found that there is a significant difference when comparing means of pace values between day and night groups: Mean pace over night sessions was higher than the mean pace over day sessions; t values were 2.11* (df=330) for Dataset1M, and 2.33* (df=284) for Dataset1P. Table 2. Day/night consistency of pace rank. Results for Dataset2M and Dataset2P, examining consistency of pace ranks over time, are given in Table 3. As might be seen, correlation coefficients are pretty low. On average, beginning and last sessions are differed by pace of action within them: Students tend to work faster at the end, as shown by t values of 3.33** (df=2,649) for Dataset2M, and 3.64** (df=1,357) for Dataset2P. Table 3. Over time consistency of pace rank. Another way of looking at these results is to scatter plot a two-dimension representation of the students according to their ranks in both groups, and to look at the four quadrants formed by the median lines. If pace rank is consistent, it is anticipated that the faster students will be faster in both dimensions, and same for the slower students, hence quadrants I (top-right) and III (bottom-left) should be occupied with most of the dots (students). For example, let's take a look at such a scatter plot for Dataset2P, which relates to the beginning/end situation for the Practicing learning mode. The examination of pace rank consistency for this dataset showed a low yet significant correlation (ρ=0.20**). The scatter plot for this example is presented in Figure 1. According to our calculations, the first and the third quadrants each holds 30% of the dots, which means that the second and fourth quadrants hold together 40% of the students. Figure 1. Scatter plot of pace ranks at the beginning (x) and the end (y) for Dataset2P (Practicing learning mode), N=1,358. Across Modes Consistency. Results for Dataset3 are given in Table 4, representing the examination of pace rank consistency across learning modes. Correlation coefficients are relatively low for this situation. Furthermore, there is a significant difference between the means of the two groups: On average, Memorizing sessions were faster than Practicing sessions with t(767)=7.99**. It is a good point to recall the similarities and differences between the two learning modes being discussed here. While Memorizing and Practicing modes share a very similar GUI, and work according to the same principle (browsing over pages each consisting of a 10-row table of words/phrases), the main difference is that the Memorizing tables show the meaning of the term, while the Practicing tables hide it. As suggested by the results, students spend more time on Memorizing pages than on Practicing pages, and pace ranks across modes have a low correlation. This might imply that pace of action is affected by a set of skills needed for progressing in either of the modes. Table 4. Across modes consistency of pace rank. Random Division Consistency. Results for Dataset4A, Dataset4M and Dataset4P are given in Table 5. These three datasets relate to a more technical situation than the previous ones: random division of each student's sessions to two groups, and examination of pace rank consistency between these two groups. While Dataset4A takes into consideration all the sessions from the log file, Dataset4M and Dataset4P relate only to Memorizing and Practicing sessions, accordingly. Table 5. Random division consistency of pace rank. It might be seen that for the general case – correlation is relatively low, however when examining pace ranks within the same learning mode, correlation is resulted with relatively high values of coefficients. Also, no significant difference was observed in the means between the two groups within each of the datasets. To conclude the results of this study, there were only two situations in which pace rank was found to be consistent with relatively high values of correlation coefficients: a) Day/night division within the same learning mode; and b) Random division of each student's sessions within the same learning mode. In all the other situations - namely: over time, across modes, and all-inclusive random division - pace rank consistency was found to be relatively low, with correlation coefficients (ρ) between 0.20** and 0.36**. 5 Discussion. Many EDM studies often handle fine-grained data in the action/session level, like pace measures. However, when examining the student level, mainly since vector variables are not easy to cope with while applying data mining algorithms, scalar measures of these variables are often being used (e.g., average or median pace over different sessions). Time-related variables (usually describing the time taken for answering a question or for completing a task) are quite common in EDM research [1, 8, 11], but others are also often being averaged, for example: attempts for answering a question [1, 11], hint/help usage (usually per question) [1], and intense of activity (usually in terms of number of actions per session or frequency of certain activities) [6, 15]. While doing this, a hidden assumption – regarding the variable in question being a trait – is lying behind the calculations. It is our obligation to deeply investigate the consistency of each variable before projecting it on a 1-dimensional measuring scale and assuming it is of a trait type, as was clearly presented by Baker [2]. This is why we choose a rather primitive variable, namely pace of actions, in order to study its consistency. As the results obtained with our data suggest, correlation between pace ranks in different situations was sometimes very low. The minimal correlation coefficient (for Dataset2P) was 0.20**, which is almost a zero correlation. The maximal correlation coefficient (for Dataset4M) was 0.62**, which is relatively high but still quite far from a perfect correlation. To be honest, these results was, at first, very surprising, as we expected to see much higher correlation values. The fact that for one situation (beginning/end consistency, Practicing mode) 40% of the students were located at the second and fourth quadrants of the pace ranks scatter plot (Figure 1) – indicating they were above the median rank in the beginning and below it in the end, or vice versa – is thought-provoking, and explicitly shedding light on the questionability of the assumption of pace rank consistency. Furthermore, the surprisingly low correlations might imply that our choice of pace was not at all of a simple variable as we first thought, as pace of actions depicts different kinds of processes in which the online student is involved while learning, e.g., reading, memorizing, recalling previous knowledge, thinking, processing, typing, and navigating. Besides the clear effect of different learning components on learning time/pace, individual components also heavily affect it, such as ability to understand instruction or quality of instruction events, as was seminally proposed by Carroll [5]. Considering that pace measurement embodies different task-related and/or student-related components (and potentially others), it is clear that replicating this study with different learning systems and/or with different pace metrics is necessary before generalizing any conclusion regarding the consistency phenomenon. In general, many educational studies investigate all kinds of students' attributes; however, EDM researches often analyze data drawn from relatively long periods of time, therefore our hand on the reduction trigger is likely to be more itchy. Further research and a deeper investigation is needed in order to better understand which behavioral attributes in online learning are indeed students' traits and which are heavily situation dependent."