Anna asks about the “Key Challenges for the OER Movement.” While the field earlier faced different challenges, here in late 2011 I believe there is one primary challenge the movement faces in the coming decade, and it is almost never discussed.
1. The Complete and Utter Lack of Assessment in the OER Space. Humans are famously terrible at judging whether they’re “getting it” or not during learning. One of the primary reasons the CMU OLI courses are (and have been shown to be) so incredibly effective in supporting learning is because they include frequent formative assessments that help learners check their own understanding. These assessments provide immediate feedback, allowing informal learners to determine with greater confidence whether or not they’re “getting it.”
The vast majority of OER in the world do not include any assessments. This means that you can use the OER in a variety of ways, but you can never really be sure (beyond your own biased estimates) if you’re grokking it or not. This is kind of like having access to a wide range of weaponry (guns, bows, crossbows, canons, catapults, etc.) and an unlimited amount of ammunition, but never being able to see whether (or not) you’ve hit the target.
A tiny minority of OER provide non-interactive assessments – these are things like practice problems delivered as text in a pdf. You can work the problems, but you don’t know whether or not your work is accurate because there is no feedback.
The CMU OLI courses, an even tinier minority of the world’s OER, provide interactive assessment, but do so using a technology which is neither reusable, revisable, remixable, or redistributable. Awesome for them, #fail for the rest of us.
Now that the OER snowball is rolling down the hill and growing in size every day, at least some people in the field need to turn their attention to the creation of Open Assessment Resources (OAR). And unless we believe that a single practice opportunity per OER is sufficient, we need multiple OAR per OER. If there are 500M OER in the world, we need something on the order of 1.5B OAR. And somebody needs to align these things (a fabulous task for a tool like delicious, btw) so that, after I’ve studied some OER, I can find an associated OAR.
The work on the Open Badge Infrastructure is only slightly encouraging with regard to this challenge. Much like the LOM work of the last decade it seems to be driven primarily by technologists and not educators. Given that there is a very rigorous academic field dedicated to assessment (psychometrics), one might expect that a few big name psychometricians would be part of the core development team on such a high profile project. That doesn’t appear to be the case. Please don’t get me wrong – I think it’s awesome that someone is dealing with issues relating to the technological requirements of alternative credentialing. However, we can’t credential (responsibly) without assessments, and in the midst of all the excitement about badges I’ve yet to hear of any novel (or even uninteresting) work on the underlying assessments themselves. (And I would love to learn that I’m wrong if someone could just point me to work out there I’ve just missed.)
Who is going to create new and appropriate assessment models? Once we have appropriate models, who will create the assessments themselves? This is the key challenge for OER in the coming decade, and a challenge standing in the way of alternative credentialing as well.
Given the OAR acronym, users of OER are literally up a creek without a paddle. And will be until we start making some progress on the OAR issue.
