"1. INTRODUCTION. Nationally, more than half of students who arrive in college intending to complete degrees in STEM disciplines fail to do so [3]. The most disastrous drop-off is associated with gateway introductory courses in math, physics, chemistry, and biology [8]. These courses are usually large and always challenging, with average grades well below those in typical college courses. Many students emerge from these gateway courses having done worse than expected; their confidence is undermined and their desire to continue in a STEM discipline strongly diminished. This happens to students across the spectrum of performance: from solid A students receiving their first B+, to struggling C+ students slipping into the D range. If we wish to increase the number of students completing degrees in STEM disciplines, we must address the loss of potential STEM majors due to large, impersonal gateway courses.’ Students in gateway STEM courses are diverse by many measures, yet we ask them to learn using a single generic approach. They all read the same texts, hear the same lectures, do the same homework and class assignments, get the same advice, and are assessed using the same exams. We have worked hard at the University of Michigan to design physics classes that optimize learning for the typical student; these courses are excellent in the mean. But we have done little to adjust our teaching methods to meet the unique needs of individuals. We can do better. Technology exists which can give each student individualized coaching that will dynamically recognize their strengths, weaknesses, and performance trends, understand their motivations and goals, and guide them through the course, all the while encouraging them to continue toward a STEM degree. Intelligent tutoring systems focusing on domain specific knowledge have a long heritage [1][5] and are known to be effective. More recently, educational systems that focus on learner’s motivation and affect have received increased attention [2]. Tailored communication techniques are well established in the world of public health, where their efficacy has been extensively tested. Our goal is to gather relevant data about the students, collect the expert advice of both faculty and students, encode this in tailoring logic, and deliver personalized expert electronic coaching to the more than 1900 students who take introductory physics at Michigan each term. 2. CUSTOMIZING THE APPROACH. To better understand this approach, consider some of the details of the introductory physics course at Michigan. In such a course, we offer students a dozen tools for achieving their learning goals: commercial textbooks, custom coursepacks, pre-lecture reading quizzes, online homework systems with real-time feedback, interactive lectures with Peer Instruction, modeling of expert problem solving with additional problems to work in groups, multiple practice exams with solutions, notecards to use during exams, student led study groups organized by the UM Science Learning Center, all day tutoring facilities, an up-to-date online gradebook for feedback, and office hours with faculty. This wide array of learning tools is provided in the hope that it will meet all the needs of a diverse group of students. Unfortunately, most students receive no individualized advice about how to utilize these tools. They lack the flexibility to skip what will not help them or to access more of what they really need, and never receive feedback or encouragement that is aware of their personal goals, identity, or interests. A few percent do get personal advice and feedback; those who visit office hours regularly. Faculty experience with the lucky few suggests that specialized advice and encouragement can make a substantial difference in outcomes. Indeed personalized mentoring strategies of this kind are perhaps the only proven tools for STEM retention [10]. To provide customized, personal advice to each student, we must accomplish three separate tasks. First, we must generate some actionable intelligence – to gather data representing the state of each student adequate to decide what help and support they need. Second, we must know what action to take in each case – to gather expert advice. Finally, we need to have a mechanism for delivering the appropriate feedback to each and every student in the class – something which is impossible with the current system of face-to-face office hours. We need a technological way to provide each student with customized advice. 3. GENERATING ACTIONABLE INTELLIGENCE: THE BETTER-THANEXPECTED. The data which inspired this project have been collected and studied as part of the UM Physics Department’s “Better Than Expected” (BTE) project [6]. For this project we have gathered detailed information describing the progress of 48,579 students through our introductory physics courses over the last 14 years. From the University’s data warehouse, we combine a portrait of each student at the time they enter the class with internal gradebook information and final grades. These data allow us to quantify the impact of student preparation and background on course outcomes, and to construct predictive models of student performance. While a number of parameters correlate with final grade, prediction with a half letter grade dispersion can be accomplished using just one parameter: each student’s University of Michigan GPA at the start of the term. With hindsight this is not too surprising: A students are largely A students and C students are largely C students. Several examples of the correlation between incoming GPA and physics grade are shown in Figure 1. Our ability to track outcomes for many groups is especially important, as we have clear evidence that subsets of students underperform relative to others in these courses, even when controlling for a variety of parameters related to technical preparation. For example, first generation college students and those from low income families (<$50K/year) receive final grades about a quarter of a letter grade lower than their classmates when compared at the same entering ACT Math score. Female students are similarly disadvantaged, falling 0.2-0.3 letter grades below male students at all measures of incoming preparation: SAT or ACT math score, High School GPA, and even prior University of Michigan GPA (see Figure 1). Reducing these disparities with appropriate interventions is one of our primary goals. Research suggestsError! Bookmark not defined. that eliminating this underperformance will also have a substantial impact on the STEM retention of these students. Figure 1: Example results from the UM “Better than Expected” project for Physics 140, the first semester course for engineering and physical science students. Left: Mean grades and dispersion as a function of UM GPA at the time the course begins. Right: Mean grades as a function of SAT math score. Results are shown for both male (diamonds and dotted lines) and female (triangles and dashed lines) students. Dotted and dashed lines show the 1 dispersion for male and female students repectively A strong gender disparity in physics grade is seen, with female students faring worse than male students at all levels of GPA and SAT math score. 3.1 Deciding what to do: gathering expert advice for success. From the BTE project, we know what performance to expect for each student, but significant dispersion in outcomes remains. It is here that we aim to act: we want to find out what interventions would help every student do better-than-expected. To this end, we gathered expertise from three sources: individual students, physics faculty members, and student study group leaders employed by the UM Science Learning Center. To begin, we have identified subsets of students who did better than expected (BTE) or worse than expected (WTE) in previous terms and have conducted structured interviews of them to help us understand what leads to these disparate outcomes. The initial round of interviews has revealed several important predictors of success previously invisible from our data. Response to the first exam is often the most important factor in ultimate student performance. Students who change their approach to the class are likely to improve their outcomes. We need to be able to encourage this behavior change, and to provide students with detailed information about how they should change. Prompt attention to setbacks is also essential. We have also conducted interviews of two groups of instructors; physics faculty members with many years of experience, and more advanced student study group leaders who have successfully completed these courses and often provide advice to current students. Advice from all three groups tells us what we should do with each student – what we should do with our actionable intelligence. Now we need only a method for delivering our intervention. 4. THE MICHIGAN TAILORING SYSTEM AND INDIVIDUALIZED EDUCATIONAL INTERVENTIONS We are able to provide individual advice and coaching to every student by leveraging a powerful, proven tool: the Michigan Tailoring System. MTS is an open-source tailoring toolkit created by the UM Center for Health Communications Research (CHCR) [4]. The CHCR team has worked for decades to deliver expertly tailored health behavior interventions over a wide variety of topics, populations, settings, and communications channels. Computer tailored communications harness the power of personalized coaching from an expert, based on specific knowledge of the subject, while delivering services inexpensively to large, distributed populations. Systems of this kind outstrip the abilities of individual expert coaches in important ways; they access a wider range of information, intrinsically quantify the efficacy of their advice in the outcomes of a wide variety of subjects, and are not limited in time or space. MTS applications can be constantly refined; their efficacy always being improved, never forgetting a lesson learned, and never running out of time, patience, or enthusiasm. A great strength of computer tailoring is tireless scalability. Once the systems are constructed, they can coach ever larger groups of subjects with minimal additional investment. Tailoring approaches have been extensively tested within the public health community, where their efficacy is clearly established in peer reviewed journals. To give one example, a two group randomized trial of web-based tailoring in a smoking cessation project showed a 25% increase in continuous abstinence compared for those who received tailored as opposed to untailored communications [9]. Tailoring approaches in public health have also been commercialized widely. One example, HealthMedia Inc., was founded in 1998 by UM Professor, CHCR Founding Director Vic Strecher. HealthMedia now provides services to millions of participants each year across a broad range of interventions. During Fall 2010 the UM Physics Department joined forces with CHCR. Using support from the EDUCAUSE Next Generation Learning Challenge (NGLC), we began working to create an educational adaptation of the MTS system for use in our large introductory physics courses. This project has now been implemented, with a first intervention beginning in January 2012. 4.1 E2Coach. We call this educational adaptation of tailoring “E2Coach”, where we intend the E2 to evoke both electronic and expert. At the start of each class, E2Coach uses the results of a survey to absorb a complex array of information about each student. This voluntary initial survey will provide a rich portrait of each student who opts-in as they enter the course, including details about their background in physics and mathematics, their motivations for taking the course, desired and expected grades, attitudes toward physics, and confidence in being able to accomplish their goals. This initial portrait is augmented as the term goes on, with new information coming both from the gradebook of the course. The combination will provide a real-time portrait of each student’s progress. Combined with historical expectations for their final performance based on the BTE project, we have what we need to intervene. E2Coach provides the interface between students and the extensive and powerful resources available in each course, customizing recommendations for study habits, assignments for practice, feedback on progress, and encouragement they receive. At important points in the course, each student receives detailed feedback on their current status, along with normative information about how their work compares to their peers and predictions for what final grade they are most likely to receive if they continue to approach the course in the same way. A significant strength of this system is its ability to realistically predict how much better each student might hope to do if they improve their approach to the course. These predictions are based on the extensive historical information from the BTE project. E2Coach advice is delivered to each student as a personal web page filled with information and advice tailored to both their state and identity. Research in public health tailoring has clearly shown the power of personalized communication, with the efficacy of advice given in this way rising substantially with increased degrees of personalization. For example, testimonials provide a very effective way of delivering advice, and are much more effective when the identity of the testifier is closely matched to that of the recipient. For our purposes, testimonials are derived from UM students (the study group leaders). Each student receives advice from a former student who shares their background, goals, and concerns. So premedical students will not receive testimonials from particle physics faculty or engineering students who love physics, but from other premedical students less familiar with physics when they started and who, like the current student, felt their future as a physician was put at risk by this class. 4.2 E2Coach progress and evaluation. NGLC funding for the E2Coach system for Physics began in May 2011 and will continue for 15 months. The system has undergone a rapid development cycle, and was launched across all introductory physics courses in January 2012. Refinements will take place during Summer 2012, and a second semester of E2Coach courses will be offered in Fall 2012. This NGLC project will deliver tailored coaching messages to 3800 introductory physics students before the project ends in December 2012. By uniting student activity data with continual measures of performance, we also establish a powerful system for quantifying efficacy one that is intrinsically sensitive to the diverse nature of our student population. Because of the grade prediction schemes detailed above, we can separately assess the impact of preparation on students across the spectrum: those likely to struggle, certain to succeed, and headed for an average outcome. Since the system addresses each student individually, we have the opportunity to improve the performance of our students at all levels. Indeed this ability to have a positive impact on both at-risk and exceptionally successful students using the same system is one of the most attractive features of tailored coaching. To test the overall impact of E2Coach we will compare the performance of students using it to that of the 48,579 students in our BTE historical data set. We will look in particular for changes in some of the measures which motivated this work. At the most basic level, we will compare the performance of students in tailored classes to what we have found in untailored classes offered in the past. We have found that performance in these classes is affected not only by preparation but also by gender, socioeconomic status, and degree of family experience with higher education. These performance disparities may well be caused by the kind of psychosocial influences which tailoring is particularly well suited to address. The recent success of a values affirmation intervention in reducing the gender disparity at the University of Colorado supports this notion [10]. Eliminating these disparities is a central goal of this project: no group taking these classes should be disadvantaged by psychosocial factors unrelated to content knowledge. To test our success, we will compare the magnitude of each disparity in our new tailored courses to that seen in our historical data. The BTE project has also shown a disappointingly strong correlation between first exam performance and subsequent work for all students. Ideally struggle on a first exam would trigger a revised commitment to the course, rather than sealing a student’s fate. This provides a principle focus for our tailoring design: to encourage struggling students to change their study habits, use course resources more effectively, and address weaknesses in their preparation directly. To test the impact of E2Coach on the likelihood that students will recover from a rocky start, we will compare the correlation of first and later exams in tailored and untailored courses. It is possible that the three goals outlined above might be achieved by merely increasing the level of communication with students. It is important that we should separate the effect of individual tailoring of messages from that of merely increased communication. We will conduct this test during the Fall 2012 semester, randomly dividing students into two groups. The first group will receive the fully tailored E2Coach intervention. The second group will all receive identical communications tailored for the statistically average student. Neither instructors nor students will know who occupies each group until after the term. Once the term is complete, we will compare results for the individually tailored and uniform communication students. This will allow us to confidently separate the effects of tailoring from the effects of merely increased communication. A significant goal of this paper is to make the Michigan Tailoring System better known to the learning analytics community, and we will provide an overview of the work required to adopt this mature, open-source tool for educational applications."