"1. Introduction. Initially the term web portal was used to refer to well-known Internet search and navigation sites that provided a starting point for users to explore and access information on the World Wide Web (Winkler, 2001). The term “Internet portal” or “Web portal” began to be used to describe mega-sites (such as Yahoo!, Excite, AOL, MSN, Netscape Netcenter, and others) that many Web visitors used as a ‘starting point’ for their web surfing. Since that time, Web portal have significantly expanded and matured, and a diverse range of portal types have been developed and used in different contexts (Portals Community, 2001). Nowadays, Web portals are generally defined as gateways to information and services from multiple sources (Tatnall, 2005). One important component is the organization, navigation, labeling and indexing of their content in order to facilitate searching of information and services, so that users can search, identify and access the most appropriate resources for their needs. Learning portals, Web portals that offer learners or educators with a large selection of learning resources, are essential to the further integration of information technologies and learning (Holden, 2003). The purpose of a learning portal is not simply indexing and delivery but to facilitate actual reuse and sharing (Duncan, 2002). Thus, the expected usage of services and resources found in learning portals could be considered as different compared to the usage of other types of portals (such as entertainment, information or commercial ones). In order to explore more about the way that users of learning portals interact with the portal services and the indexed content, it is often useful to engage analytics - usually by studying the log files of the portal. Such a log analysis can take place in a systematic, repeatable but also practical way, allowing portal owners to explore the actual usage of their learning portal and to identify potentially interesting patterns of use (Piearrakos et al., 2003; Machado et al., 2003). In this paper, we examine the case of a Web portal that supports users in finding digital learning resources to support and enrich their teaching activities. It is the case of the Organic.Edunet portal (www.organic-edunet.eu) that aggregates resources on organic agriculture and agro-ecology, allowing educators to find and retrieve this content from a single point of access. We particularly examine how the portal has been used before, during and after the organization of training events for potential users (i.e. educators) in several European countries. This analysis focuses both on the general usage statistics of the portal, but it also looks at the user level, trying to identify changes on the typical user profile of the portal due to the training events that were organized. 2. Background. The Organic.Edunet Web portal aims to facilitate access, usage and exploitation of digital educational content related to Organic Agriculture (OA) and Agroecology (AE). Organic.Edunet aims to support stakeholders producing content about OA & AE in order to publish it in an online federation of learning repositories and describe it according to multilingual, standard-complying metadata. The portal front-end features a multilingual user interface translated into fourteen (14) languages, providing access to more than 10,500 learning resources from a federation of eleven (11) institutional repositories. The targeted audiences of Organic.Edunet portal are mainly educators: school teachers looking for resources to help them prepare their relevant teaching activities (e.g. how to set up a school garden to support hands-on environmental education); and academics (professors, teaching staff or researchers) in agriculture and life science topics, looking for resources to help them with their teaching, learning and research activities (e.g. writing a case study report on a particular crop or farm type). Potential users also include the stakeholders producing content, such as agricultural libraries and academic publishers. The portal has attracted, until the end of November 2010, 22.100 visits and 120.000 page views from 14.300 unique visitors from approximately 130 countries, out of which more than 1.000 have registered to the portal. These numbers assure that the portal has been used from a wide audience, thus allowing an analysis of the users’ behavior to reach safe conclusions as far as the portal usage is concerned. In order to introduce the potential users to the portal, a series of Open Days have been organized in various countries, featuring validation events where educators had the chance to work with the Organic.Edunet portal. These events were organized based on specific guidelines and also provided clear instructions to all the participants (http://virtuelleschule.bmukk.gv.at/projekte-international/euprojekte/organicedunet/open-days/). From its launch, the Organic.Edunet Portal has been linked to a Google Analytics (http://www.google.com/analytics) account in order to track the visits and be able to document its usage. Another envisaged use for this account was to try and evaluate the portal through analyzing focused statistics having to do with a range of parameters, like the day or time of visit, the geographic areas of the visitors, but also the origins of their visits. The visitors’ log files have been collected and analyzed using the Google Analytics tool in order for observations to be made in relation to the way the behavior of users was affected by the organization of such training events. 3. Methodology. A brief outlook on existing literature on evaluating learning portals has showed that there are different ways in which a portal can be evaluated. One option would be to provide users with questionnaires or web-based evaluation tools and analyze the feedback provided (Silius & Tervakari, 2003; van der Heijden, 2002). This evaluation has already been carried out in Organic.Edunet with an online questionnaire (http://www.ieru.org/organicsurvey/). A second option would be to apply an analytics approach, by analyzing the log files of the users in order to study their visiting behavior (Buckley et al., 2006; Carr et al., 2008; Hasan et al., 2009). Hybrid methods using both approaches can also been found in the literature (Stacey & Rice, 2002). 3.1. Context of study. The analysis carried out in the context of this paper, particularly focuses on studying how the usage of the Organic.Edunet portal was affected by the organization of the focused training events (i.e. Open Days). Overall, thirteen (13) of the Open Days that have been organized in five (5) different countries are examined. These Open Days had 160 participants comprising mainly from school teachers and academic staff. Brief information on these Open Days is presented in Table 1: Table 1. Overview of the Open Days organized in Schools & Universities 3.2. Steps Taken and Data Used. In order to study the portal statistics before, during and after the Open Days, some variables have been defined and some assumptions were made. More specifically, when looking at the data on Table 1, it’s apparent that the Open Days took place on two distinct periods and not throughout the whole period of reference. The first group of Open Days generally took place during March & May 2010 (the fact that one Open Day took place during early February it is not considered to affect results so much) whereas the second group of Open Days took place on August-September 2010. Our analysis focuses on the period before the two groups of Open Days (Pre-OD), the actual Open Days (OD) and finally on the period after the Open Days (Post-OD), more specifically: a) The time before the first group of Open Days is symbolized as “Pre-OD”, whereas the first period of Open Days is symbolized in all graphs as “OD1”. This time following the first group of Open Days is symbolized as “Post-OD1”. Respectively, the second Open Days’ period is called “OD2” and the period following the second group of events took place is symbolized as “Post-OD2”, b) The portal-related variables selected to be examined were: Visits for the portal, Page Views, Unique Visitors, Traffic Sources and Most Popular Pages. These statistics are calculated for all the periods defined in (a), c) The user profile that is also examined in this study, is comprised by a set of statistics that try to focus on the user-level behavior that can possibly lead to new insights on the use of a thematic portal with a focused community around it. More specifically, the variables measured here include: Average Time on Portal, Average Time on Page, Pages Accessed per Visit, Visitor Loyalty and Depth of Visit Definitions. Deep visitor: Deep visitor would be the visitor that in one visit would open more pages than the average pages per visit of all the visitors put together for the same period of time. Statistics for this metric were calculated using the “Depth of Visit” available in Google Analytics, showing the percentage of users that visited one page per visit, two pages per visit, three pages, etc. up to the class of 51-100 pages. Bounces: Any visit during which the user views only one page and then “exits” the portal, is considered as a bounce Visitor Loyalty: Loyalty of each visitor shows the times per time period (i.e. day, week, and month) that a user visits a website. Data post-processing and analysis was carried out using Microsoft Excel where exported CSV files from Google Analytics where processed. The results are presented in the form of tables and figures that focus on the variables that are examined. Column charts were chosen to present focused quantitative data whereas tables were mainly used to provide overviews of the data per region. The main research question explored in this initial analysis is: How do the usage statistics both on the portal level but also on the individual user level, change, after a series of training events on the portal is organized? E.g. do educators start using more the Organic.Edunet Web portal? How is that reflected on the user-level statistics? 4. Initial Results. 4.1. Portal-level Statistics. In this section, we present the statistics related to the portal in general, trying to identify the way in which they change, if so, after the organization of an Open Day. Table 2. Overview of portal statistics before, during & after the Open Day periods defined Looking at Table 2, it would have been expected to get higher values for the periods of the Open Days in all metrics, when compared to the respective “Pre” and “Post” Open Day periods. So, for example, from 32.6 visits per day before the second Open Day period, we moved to a 54.2 visits per day during the Open Day period, which dropped to a 44.9 after the Open Days ended. It’s interesting to note that even though the numbers drop after the Open Days, they do not decrease to the same extent, showing that the Open Day actually brought upon some kind of change to the portal usage. One exception to this rule is the case of Page Views per day for the second period of Open Days, which from a 159.9 went up to 250.6 but after the Open Days, it dropped to 153.7 page views per day. Overall, comparing January 2010, when the portal started operating with November 2010, long after the end of the last Open Day, visits per day were tripled, page views per day rose by almost 60 views, whereas visitors per day were five to six times more than in the beginning. Examining the sources of visit, one could argue that time given, the users of the portal are using it more and more, and therefore it could be safe to assume that they would also visit the portal directly and not through search engines or other referencing websites. This assumption is contradicted by findings, as direct visits drop from a 65% before all Open Days, gradually to a 23.9% after their end. Visits generated by search engines do not seem to follow and clear pattern (ranging from 18.5% to 32%) whereas visits from referencing sites gradually increased from a mere 14% to almost 60%! The explanation to this comes from a factor outside the portal itself. The Organic.Edunet portal was mostly referenced in websites of consortium members in the beginning of the period examined but was also promoted through a network of affiliated partners which was developed during the course of the project, and mainly during the last quarter of 2010. In total, 83 affiliated partners put links to Organic.Edunet on their websites, which can explain the rise of visits coming from referencing websites. The interpretation of bounces is also interesting. Overall, bounces seem not to be affected by the Open Days organized, in the sense that they do not fluctuate at all before and after the Open Days, but they show a steady increase throughout this period. Looking at the bounces in relation to the visits to the portal per day, it can be seen that the majority of visits per day, actually include bounces also, since bounces are visits with only one page visited at a time. This shows that the few visits that did not bounce off actually went into great depth. For example, for “Post-OD2” period, only 18.2 visits per day did not bounce off the portal, but these visits generated 127 page views per day, which are almost seven pages per visit on an average. Table 3, shows the most famous pages of the portal measuring the visits per day that each page attracted. As it was expected, most of the visits to the portal originate from the homepage, so this is the highest ranking page of all. Stats for the homepage do not show anything out of the ordinary, as visits per day rise for the periods of Open Days and they fall during periods in between. Simple search (textbased search) is also a very popular destination within the portal, but in this case, the visitation rates fluctuate less than the ones for the homepage. Despite that, comparing all the search functionalities offered, text-based search ranks on the top. Help on how someone can use these functionalities is also visited pretty often, with high numbers during the Open Day periods, also. Browsing through the resources based on specific criteria (educational level, language, difficulty, etc.) is also a widely used search method in the portal which follows a steady but decreasing usage. Browsing is followed by the Semantic Search which involves searching through the terms of a domain specific ontology deployed as a tree of terms and concepts. Semantic searching seemed to gain some ground before and during the second Open Day period, but again it fell to the levels of visitation before the Open Days. Table 3. Most popular pages (visits per day) before, during & after the Open Day periods defined “Educational Scenarios” is a portal page that offers content specifically designed for use in the classroom of either schools or universities, through elaborated scenarios on various topics related to Organic Agriculture and Agroecology. Overall, this page shows that despite a promising start (before the Open Days it ranked 3rd), in the end, it represents the least popular page, close to the tag-based search. Finally, tag-based search remained largely unused throughout the period examined. All these statistics, related to ways in which users search for content can greatly influence the interface design of the portal. 4.2. User-level Statistics. In this section, we present the statistics related to the users in specific, trying to identify the way in which their behavior changes, if so, after the organization of an Open Day. Looking at Table 4, it seems that the average time spent on the Organic.Edunet portal by each user is dropping throughout the period, starting from approximately six minutes, down to almost three minutes. So, overall, more visitors came to the portal as it was depicted in the portal level statistics, spending all and all, less time on the portal as it is shown here. Someone may have expected that this would also be reflected in the average time spent on each page, this is not the case. Although this time is also decreased, the decrease is both not that big and it also fluctuates from 59 to 73 seconds. Pages viewed per visit are dropping gradually, which was also reflected on the most popular pages. Looking at the depth of visit, which is closely related to the pages per visit figure, it seems that the depth in which the visitors use the portal, is more or less steady. This shows that each time, almost a quarter of the total visitors actually spend time with the portal, opening more pages, looking for content and using the search functionalities. Table 4. Overview of portal statistics before, during & after the Open Day periods defined User loyalty was measured as the percentage of visitors that came back to the portal one or more times during a week. This limit was defined in the context of this study to measure the loyalty of the users of educational portals, as no such information could be retrieved in other studies. Starting from the preOpen Day period, only about 5% of the users would return to the portal more than once a week. As expected, during the first period of Open Days, this number of loyal visitors tripled to 15% which more or less was sustained during the period that followed the first set of Open Days. As expected, the second set of Open Days enhanced visitor loyalty, with one fifth of the visitors coming back to the portal at least once a week. As with most of the stats that were analyzed so far, user loyalty seems to have been “helped” by the Open Days, increasing during them and returning to a lower level after them, which is marginally above the initial loyalty, that was measured before any Open Day took place. Figure 1: Visualization of the changes in the user profile through the course of different periods As it can be seen in Figure 1 the typical user profile changes gradually through the Open Days. For example, it’s apparent that Depth of Visit remains the same whereas Visitor Loyalty greatly fluctuates. Similar visualizations of the user behavior may help in depicting different types of usage profiles for similar learning portals. 5. Conclusions. This paper presented an initial analysis of the log files of the Organic.Edunet portal, a Web portal that supports educators in finding digital learning resources to support and enrich their teaching activities. It particularly examined how the portal has been used before, during and after the organization of focused training events for potential users in several European countries. It tried to study any notable changes in both the portal statistics overall and also study any changes in the behavior of the users, before, during and after the training events. The present paper reached the following conclusions: - The portal statistics retained some of the dynamics that were created during the Open Days, even two months after their completion. This was confirmed for visits per day, visitors per day and pages accessed per visit, - Bounces continued to rise throughout the period showing no direct relation to the Open Days organized. This rise could be partially explained by the fact that the visits originated from referencing sites in a big percentage which could mean that visitors accidentally stumbled upon it and exited after viewing only one page, - It seems that traditional search functionalities worked better for the users that largely did not use tag-based search. Semantic search and browsing per topic were used, but still not as much as expected, - The typical user before the Open Days would not return to the portal more than once a week and would spend more than six minutes in the portal in total in each visit. This user would occasionally visit more pages than the portal’s average, taking a “deep visit”. After the end of the Open Days the same user rarely returns to the portal more than once a week and spends significantly less time on the portal. The frequency in which “deep visits” occur stays the same - Many project-related, outside of the portal scope, parameters affected the statistics on the portal (i.e. affiliation strategy of the project, press releases circulated) by drawing visits from a wide scope of people that will influence the usage of the portal As far as the analysis of the Open Days in groups is concerned, one can argue that the results must be compared before and after each Open Day, but this was not chosen for two reasons. First of all, a preliminary analysis showed that the results were more or less as expected showing no research interest when analyzing Open Days per country; i.e. traffic is heavier during Open Days, with more visits or that time spent on the portal increases due to the structured exercises given during the Open Days. The second reason had to do with selecting a bigger sample of users by grouping the Open Days together, from which safer conclusions could be drawn, whereas the limited sample of 15 participants per Open Day could not provide that. Another issue has to do with the fact that there is no standard threshold that can characterize a user as loyal or otherwise, because this number is more or less affected by the nature of the website examined, the services it offers, etc. In a recent survey by Chitika Inc. (http://insights.chitika.com/2009/digg-facebook-loyal-readers/), related to Facebook and other popular websites, a loyal visitor was the one that came back to the website four or more times per week. In the case of an Educational Portal, this number might be too high of an expectation, so for the purpose of this study, it was decided to set forth our own threshold in characterizing someone as “loyal”. Future work should focus on delving deeper, trying to elaborate the generic research question into a set of research hypotheses that will be statistically explored. Through such an analysis, it can be decided if a measurable and confident change in the users’ behavior is observed. A more detailed analysis can also take place focusing on data such as the sections of the portals that are used more during and after the training events (i.e. do the users spend more time on new features/services that fit their needs?). Finally, we are interested to view results in the light of the interaction of different audiences with the portal (e.g. academics/researchers, teachers, public, other). This could also indicate if separate portal interfaces are needed to better serve each community. 6. Acknowledgements. The work presented in this paper has been funded with support by the European Commission, and more specifically the project 250525 “Virtual Open Access Agriculture & Aquaculture Repository: Sharing Scientific and Scholarly Research related to Agriculture, Food, and Environment” of the ICT Policy Support Programme (ICT PSP), Theme 4 - Open access to scientific information."