A student of mine, now DOCTOR Sean Duncan (congrats again!) has posted his excellent dissertation studying reuse of OERs online under a CC-BY license. This was one of the most enjoyable dissertations I have ever chaired. I’ll cover highlights below, but I encourage you to check out the full text of Patterns of Learning Object Reuse in the Connexions Repository for yourself.
The study examined patterns and amount of reuse within the Connexions OER repository at Rice. CNX seemed like a great choice for examining reuse because the system is built specifically to support both adapting individual modules and remixing individual modules into courses / collections. Importantly, through system metadata that CNX also makes openly available, all these relationships can be explored programatically in a straightforward way. So CNX is in many ways a best-case scenario for studying reuse, adaptation, and remixing.
Terminology and definitions are very important if we’re going to be doing quantitative measures of different aspects of reuse. Here are Sean’s terms and definitions.
- Use – Count of each inclusion of an original module as-is in a collection.
- Reuse – Count of all but the initial use of an original module as-is in a collection (i.e., Use – 1).
- Translation – Count of each derivative of an original module where the language differs between the modules.
- Modification – Count of each derivative of an originating module where the language does not differ between the modules.
- Recycled – Count of each reuse, translation, and modification of an originating module.
- All Use – Count of each use, translation, and modification of an originating module.
Sean opens the results chapter by saying:
A critical, unstated assumption made by the researcher turned out to be false, but its identification is, in and of itself, an important finding. Specifically, the researcher assumed that there was significant use of Connexions modules within Connexions collections. Because the study goal was descriptive quantitative data, this was not a fatal flaw and its early identification resulted in important methodological revisions.
And here is a sample of some of the findings:
The total count of Unique Modules Published was 5,221, but the total count of unique published Connexions modules that were used in any collection or as the originating module for another derivative module (i.e., unique modules used), was only 3,519. In other words, 32.6% of modules published in the Connexions repository are not used at all.
The next calculations were the count of how many times modules were included in any collection (i.e., the total use count), which was 4,713, the count of how many times an individual module exceeded an initial use (i.e., the total reuse count), which was 967 times, and the count of modules that were used in two or more collections (i.e., the total unique modules reused), which was 724. These calculations indicated a reuse of discrete modules as 20.57% of the unique modules used, with a reuse rate of 1.34 times for each reused module.
There were only 1,013 module uses where there was no common author [between the module and the collection].
Of the 3,519 unique modules used, 105 were translated into 174 derivatives, while 101 were modified into another 120 modules.
Ultimately, of the 3,519 modules used in Connexions, 861 of them were recycled in some way for a total of 1,262 uses. This means that almost a quarter of all modules that were used at all were recycled. Between basic reuse, translated derivatives, and modified derivatives, recycled modules were used almost 1.5 times beyond their originating object’s initial use.
Only 15 modules were used, translated, or modified more than five times (see Table 5).
As I said, read the full Patterns of Learning Object Reuse in the Connexions Repository for yourself.
To me, this study begins to confirm the “dirty secret” of OER – that the reuse emperor has no (or only very scanty) clothes. I’m planning similar studies of other collections now, and looking for graduate students to run these studies. Looking for a thesis or dissertation topic? Let me know!
If these results hold true (and those of us who run OER sites and repositories have large reason to believe they do based on our own experiences) we would do well to take a step back and remind ourselves that the goal was never just “reuse” in and for itself. Rather, I think there were myriad goals which reusable resources, whether they were named “learning objects” or “OER”, were aiming at; improving quality through iterative development, improving access, reducing costs and time… it’s a long list. But if we recall these goals and put the focus back on them, we may find that the *how* of OER is equally, if not ultimately MORE, important than the *what*. That examples like Groom’s WPmU sites and the UBC mediawiki/wiki inc syndication module are important not only because they lead to simpler reuse, but because they are based on free, simple, collaborative and open tools to begin with, and so achieve some of these goals before the question of reuse even gets asked. Dirty secret indeed.
Nice one! This will be up there with the Reusability Paradox Dave! And very similar causes too. To my mind, it depends what we call OER… I’m personally suspicious of anything calling itself “educational”, more often than not because it simply results in a too narrow consideration of the issue. What if we thought of the Internet as the OER? The only real barrier to reuse is of course some people’s take on copyright, so how about we look to the open Internet – being all the information, media, and communications that is implicitly available for access and reuse. For example, Youtube. While OER purists reject Youtube content, it is actually openly accessible, and encourages reuse, and is relatively easy to make copies. Probably a better example would be Blip.tv and Flickr who do go to the effort to allow CC licensing. There are many others of course. If we were to include such things in our definition of OER, then the reuse is HUGE. I’d say the problem with OER is the E word.. resulting in resources that are too complex and too contextualised to be of any use to reuse. Your paradox again. If OER repositories and authoring platforms like Connexions and WIkieducator were to simply aggregate content from all across the open Internet, and if we measured the reuse of individual items of media instead of whole compositions, then the reuse would be impossible to count – not least because the line between educational and non educational is harder to see.
Hey David — I echo my congratulations to DOCTOR Sean! This is a timely contribution — particularly as we’re beginning to think about the technological grease that could smooth and scale remix and reuse scenarios in the OER world. I’m off to read the dissertation!
Over at WikiEducator we’re exploring the reusability paradox and possible solutions. Assuming success with our funding proposal we’re hoping to build a bridge between Connexions and Mediawiki. This could create interesting opportunities for “remixing”: (1) Producer-consumer models of OER production with (2) Commons-based peer-production models.
Given WMFs recent announcement in migrating to Creative Commons Licensing — we could see some very interesting remix scenarios for the future 🙂
In OER, I think the differences between OpenCourseWare content and Learning Objects affect reuse.
I’d be interested to see how often the learning objects were accessed by content designers. In computer programming, most code is not reused, but nowadays programmers rely on google code and other code search engines to find out how others solved similar problems. I’d guess that if content designers can observe the learning objects of others, the result would be better learning objects.
And an obvious next step would be to interview the users of the repository, to find out what makes them reuse the learning objects, or why they don’t. I’d have to guess that the teachers are afraid to reduce the novelty effect; when a student comes to a new class and the learning objects are old, the student might express disapproval. Another contributing factor in the use or disuse of the learning objects in the repository would be training and the support of a peer-network. If the higher-ups don’t encourage learning object reuse, then I’d suppose many will be inclined to use their old methods of designing lessons.
This was an interesting study, especially because it raised many more questions than it answered.
Hi David,
Interesting, but so not surprising! We’ve been looking at the same thing over here in Europe. Here’s a complementary (not published!) for your collection, where I looked at the use and reuse on 2 platforms: LeMill and Calibrate from European Schoolnet.
My twist was to study cross-boundary use and reuse, i.e. teachers reuse of learning resources that are in a language other than their mother tongue and originate from different countries than they do.
We also found that the general reuse stays around 20% (this was what Ochoa found in his , too), and that the cross-boundary reuse was notably less (37% to 55% of it). Moreover, in some of the repositories only around 10% of resources were ever added to a collections!
Hi David,
Interesting, but so not surprising! We’ve been looking at the same thing over here in Europe. Here’s a complementary (not published!) study for your collection, where I looked at the use and reuse on 2 platforms: LeMill and Calibrate from European Schoolnet.
My twist was to study cross-boundary use and reuse, i.e. teachers reuse of learning resources that are in a language other than their mother tongue and originate from different countries than they do.
We also found that the general reuse stays around 20% (this was what Ochoa found in his ,PhD too), and that the cross-boundary reuse was notably less (37% to 55% of it). Moreover, in some of the repositories only around 10% of resources were ever added to a collections!