formularioHidden
formularioRDF
Login

Sign up

 

Computational Infrastructures for School Improvement: How to Move Forward

InProceedings

The instructional practices common in today's schools reveal a disconnect between instruction and evidence of the effects of that instruction on student learning. In this paper, we propose the creation of computational infrastructures that will help teachers make more informed decisions in their practice. These infrastructures formalize student and teacher routines to facilitate data collection and mining, in order to create actionable information. We then show an instance of such a computational infrastructure and describe its potential for improving instruction.

"1. In SPACE, project assignments are comprised of ordered sets of task assignments. Each task is subject to a set of assessment criteria, each of which is related to one or more standards of goals (the standards may be State or Federal Standards or local constructs). As students (working alone or in groups) do their work in SPACE, the electronic artifacts they create (the student tasks) are related to the appropriate task template as well as to the authors themselves. Teachers' and students' assessments of work, along with information about (co-)authorship, form a kind of incidental social network, the implications of which we discuss below. The SPACE user interface makes it easy for students to find out what work they need to do and what criteria will be used to assess it. Teachers and students may easily browse the database to see the work of others and to assess that work. Aggregate representations make it simple for teachers to discover which skills need the most instructional focus. At present, these aggregations are of the most simplistic kind, showing, for each standard or assessment criterion, on a by-student, by-assignment, or by-cohort basis, the mean, standard deviation, and trend over time. Presently SPACE employs relatively simple EDM techniques, but as we discuss next, combining the structured data of a system like SPACE with advancements in EDM could dramatically increase the quality and quantity of actionable information available to teachers. Figure 1: SPACE Information Architecture. 5 SPACE in Practice. SPACE was intended to help teachers enact cycles of formative instruction throughout the course of project-based Inquiry. The following example arose during a recent classroom implementation of SPACE supporting a science fair project: Students are working on different projects and they are at different stages of progress. Students submit work to be assessed by their peers and the teacher. The work can then be revised and resubmitted as needed to complete the task. Managing the student schedules is critical because the science fair represents a hard deadline that must be met. This is a lot of data for a teacher to keep track of, and students (and the teacher) can easily fall behind. Figure 2 shows a visualization of an eighth grade class midway through a project. The stages represent progress towards the science fair presentation. As an example of what these stages represent, stage one is to read some current events and find something technological that excites the student and take some notes on it (e.g. Nike’s new tennis shoe). Stage two is finding a related area of interest (e.g. materials science), and stage three is a project proposal (e.g. comparing springiness of different materials). From there stage ten and eleven are building a data table and collecting data for analysis. Finally, stage 14 and 15 is the write up and presentation of the work. At the time that this figure was generated several students were starting on stage 10. A tremendous amount of data is being generated: the student work (the pieces of work individually, but also chains of revisions), the critiques, rubric-based assessments, and logs of who’s looking at what. This data must be analyzed and presented in a way that is useful to the teacher to become actionable knowledge. Analysis and visualization techniques appraise teachers of students that are waiting for teacher feedback in order to go forward. Other visualizations indicate whether the student is ahead or behind schedule for the science fair. Looking at this visualization the teacher can know immediately which students are falling behind or need attention. The ‘w’ indicates tasks that are waiting for teacher feedback. The large ratio of ‘w’s to non-‘w’s indicates that this specific teacher is significantly behind in offering feedback, and may be overwhelmed. Moreover, the number of pink and red boxes illustrates that students are struggling to turn work in on time; the number of lagging revision required symbols illustrates that many students have been asked to revise but have not, perhaps because they are confused about what the teacher wants them to do. The representation in Figure 2 is a high-level view of the underlying data; it foregrounds punctuality and task status, while backgrounding the actual content of the work and assessments thereof. Teachers can click the column headings, author names, or project titles to see all instances of work on a task (across projects), to see information about the authors (including links to other work), or all work on the project, respectively. This representation is only one of many possible ways of aggregating project work. We have already implemented a skill-based aggregator that shows summary statistics about how students’ work (at the individual student, project, or cohort level) has been assessed, on a skill-by-skill basis, showing teachers what skills students have mastery of and which they need additional support in. Because all interfaces are massively hyperlinked, it is easy for teachers to find concrete examples of relevant work. We imagine the application of a number of information retrieval and data-mining techniques to provide additional depictions of the data. The following section describes several possibilities. 6 A Way Forward. Singley and Lam describe a number of interesting heuristics that could be helpful to practitioners trying to decide where to direct their attention [14]. Their Classroom Sentinel alerts teachers to conditions, identified through data mining, such as a student’s grade dropping significantly from past assessments, a struggling student performing above average on assessments, the student being within range to increase their letter grade if they do well on the next assignment, or ESL students performing below average on a mathematics assessment. [8] describes a framework for data driven decision-making in schools that “highlights three key components: the process by which raw data becomes useable information, the role of prior knowledge of the decision-maker, and the effect of the data-reporting tool in shaping that process”. We try here to describe, in a more concrete fashion, what infrastructure for such decision-making might be. Figure 2: SPACE Project Progress View (for Cohort). The data structure behind SPACE affords a number of powerful analyses each of which promises to deliver actionable information to teachers and leaders. These analyses range from discovering students' mastery of necessary prior knowledge or related activities, to the leveraging students' social networks, to recommendations of pedagogical options, each of which we now describe. The SPACE database makes it simple for teachers to connect summary statistics to concrete examples of student work. For example, the Illinois state standards for literacy state that middle school students should be able to ""identify appropriate resources to solve problems or answer questions through research""; this standard is relevant to any activity where students must collect sources ""from the wild"" (e.g., the Internet) in order to do their work. Consequently, in a full implementation of SPACE across a school or district, there would be multiple assignments, in a variety of subject areas, that are related to the standard. Through SPACE, a teacher can see not only summary statistics about students' performances related to the standard, but also concrete examples of work. Teachers could search for all examples (in his/her own class or in others') of poor performance in selecting appropriate articles, or find out which students tend to be poor selectors. Having this rich data offers interesting opportunities for increasing the (measurement) validity of teachers' assessment practices. Ideally, there would be a high correlation between teachers' assessment of students' work with respect to some set of standards and other measures of those same students' skill with respect to those same standards. For example, if a recent assessment says that a student has trouble summarizing texts, then we'd expect that teacher assessments of that student's summaries of texts should be similarly low. Divergence between the two measures may indicate a disconnect between teacher understanding of standards and how those standards are measured. Tools like SPACE would allow teachers to take problem areas identified by standardized tests, and then comb through a student’s work to verify similar pre-existing issues. This verification process helps teachers identify and recognize problem areas for present and future students. Consequently, tools like SPACE offer the potential to allow teachers to reflect on their practice as well as make standardized tests formative rather than summative assessments. Education research has found patterns in peer groups and achievement using social network analysis [10,13]. The SPACE database also affords a number of social network analyses. For example, students' critiques of each others' work might represent an edges between students. We can use the amount of subsequent improvement in students' work as the strength of the edges. We can then analyze a network constructed in such a manner over time to understand which student’s feedback influences another student (i.e., students' whose critiques are useful to a broad spectrum of peers). This information could be highly actionable for teachers, supporting decision-making about instruction (e.g., knowing who needs help giving good critiques) and classroom management (e.g., which students to pair up so that they'll be maximally mutually supportive). CISIs have potential beyond what we have described here. Not only could they better support the daily cycles of instruction, but they can also be used to aggregate information at the school level to make decisions. For example they could assist school leaders making decisions about how to allocate discretionary support resources (e.g., teachers’ aids, additional training, supplemental money for extra teacher hours). Similar to how teachers have an increased awareness of students’ learning, leaders could spot school level problems early on (such as the teacher falling behind in the above example) and intervene to be proactive instead of reactive. 7 Conclusion. Personalizing instruction for students is demanding of teachers. Presently teachers do not have tools that support their daily instructional practice. Computational infrastructures that merge data storage, mining, and presentation can help teachers manage classroom data to make more informed and responsive decisions. Through the use of these sorts of tools, levels of instruction that previously required vast pedagogical content knowledge and heroic effort could now be much more reasonably achieved, with benefits for every student."

About this resource...

Visits 100

0 comments

Do you want to comment? Sign up or Sign in