As part of my work on the NGLC-funded Kaleidoscope Project I’ve been thinking about practical learning analytics. Why “practical”? My goal with practical learning analytics is to provide access to data in ways that an average teacher, with no special training, can leverage in order to help her students succeed. This is, of course, an extremely tall order.
As I began to mull over some common conventions that teachers could interpret without training (e.g., time flows left to right, scores move higher and lower) I realized that there’s already a tool available that provides visualizations like this – the Google Motion Chart Gadget.
This interactive visualization (play with it here) shows “visits” on the x axis and “current score” on the y for about 65 students over the first seven weeks of a course. (You can visualize up to 4 variables simultaneously with the GMCG, but I wanted this demo to be easy to understand.) Watching the data move through time makes it extremely simple to find students whose grades are dropping (as their circle drops), students whose course activity is waning (as their circle slows or stops in its left to right motion), etc. When you see something “interesting,” you just pause the viz and mouse over the circle in question to get the student’s ID so you can then reach out to motivate, offer help, etc.
Now, you might think that it’s quite simple to generate a viz like this since the GMCG provides the framework. However, try asking your LMS how many times a student logged in during week 3. Or what a student’s grade was at the end of week 5. Your LMS doesn’t actually keep this data at all (or anywhere that you can get to it). So if you want day by day or week by week granularity you’re forced to hack your way around by generating reports daily and calculating deltas to get the day’s activity. Even if your LMS provides a “1 day” report, you can only get 1 day – not all of the “1 day”s in the semester. So you’re still left downloading reports every day and combining them by hand (or script) into the form you need. And since you want to provide daily updates to this viz, that’s actually Ok, except that… this is extremely tedious to do since most LMSs provide no interface for pulling out the necessary data programatically (i.e., you’ll be pulling these reports by hand). And if you want to provide this view to teachers in very nearly real-time (which should absolutely be our goal), you’re consigned to pushing this button by hand at the same time every day – like the fellow on Lost who had to push the button every 108 minutes in order to save the world.
Anyway, lots more to unpack here, but I wanted to post this (1) because I think it’s a great approach that demonstrates how a simple visualization can be made to serve learning analytics purposes, (2) to send a shout out to my Kaleidoscope Project friends, and (3) to complain yet again about how difficult LMSs are to work with. Hopefully someone will find this work inspiring and build something even better and more intuitive.
I love this post and you make some great points. I have been studying learning analytics for a while now and you bring up some of the biggest pains in any data analytics field. Data scientists, generally speaking, follow a simple model: Obtain – Scrub – Explore – Model – Interpret. Sometimes the biggest challenge is that the data you want is not available or even worse you cannot get to it – as you have indicated. I think the problem tends to be on what information is important to track, and who defines important. Systems (like LMSs) may not truly understand what types of data should be tracked and more than likely the users of the data want data that is not tracked by those systems.
In most cases, like yours, we tend to find workarounds. Data analytics, to me, is a dirty job. Scraping information, running reports, generating scripts – all in a day’s work to obtains, scrub, explore, model and interpret information.
I wish we did not have to go through this craziness, but until data becomes more open and accessible, not sure what can be done. I don’t think the answer is more standards, just more openess and flexibility. I would love to share other ideas and stories with you. Great post.
This is such and elegant, creative and yes, practical application of dataviz! Awesome.
Tweeting/FB-sharing now 🙂
Oh my…this is BEAUTIFUL! How long DID it take you to extract the data for these 62 students? I am curious what it would look like if you did add more variables…like what students were DOING when they visited the site.
Thank you for sharing this!!