From Analytics to Evidence-based Design

TLC Ed Tech KeynoteThe 2012 TLC Educational Technology Conference drew over 100 faculty and staff to workshops and speakers on November 14 and 15. Two keynote speakers engaged the audience in thought-provoking discussions about the use of data to predict learner progress and to drive the design of learning activities.

Janet Corral set the stage on our first morning with an overview of the relatively new field of learning analytics. The basic premise is: given that our systems are collecting a lot of data about how our students interact with course content, what can that data tell us? Can it provide us with useful information about how each student is progressing and what activities are yielding the best results? Can we predict whether students who interact with the system in one way will do better than students who interact in another way?

We are all familiar with how businesses push advertisements to us based on prior purchases or ‘likes’ on Facebook. If a student’s posts to the course discussion forums generate a network of replies and new threads amongst the other students, what does that tell us about that student? Here at UCSF, does accessing particular learning objects correlate with performance in some way? As you can imagine, designing a fruitful analysis can be quite complex, but Dr. Corral presented some of the ingredients for thinking through that process.

The second keynote speaker, Candace Thille, generated a lot of excitement as she described the groundbreaking work at Carnegie Mellon’s Open Learning Initiative (OLI), where they have been developing open online learning environments based on the science of learning since 2002. Course designers use student activity data to inform an iterative refinement of their course materials. Dashboards for faculty provide insights into how students are progressing with the content so that class time can be optimized to address areas where students are struggling and spend less time on areas where students are doing fine.

Many of us were quite envious of the team of experts they draw on to create a course at OLI. In addition to the domain experts, the core development teams consists of instructional designer, instructional technologist, project manager, user experience designer, software developer, and a learning scientist. Without these extensive resources, we wondered, what can we do here at UCSF? One possibility is to tap into the open courses offered, such as OLI’s Probability and Statistics course. Another is to use the evidence they have gathered and apply it to our own teaching.

Thille closed her presentation with a quote we can take to heart, from Herbert Simon, Nobel laureate and CMU Professor: “Improvement in post secondary education will require converting teaching from a ‘solo sport’ to a community based research activity.”

Watch the recorded sessions below:

Click here for more resources on learning analytics.

Photo Credit: Erin Hayes

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>