Dark Matter, Dark Reuse, and the Irrational Zeal of a Believer

I recently reported the results of Sean Duncan’s dissertation, which calls into question the actual rates of reuse of open educational resources. A number of people have expressed concern or disbelief with his results. As a highlight, if you missed the earlier post, the study looked at rates of use, reuse, and adaptation within the Connexions collection, and found that of 5,221 modules published on the site only 3,519 of these were ever reused in a collection or adapted in any manner elsewhere on the site, and that only 15 modules were used, reused, or adapted more than five times.

Perhaps the most frequent criticism I hear is that “reuse and adaptation are happening other places (outside the Connexions repository), you just can’t see them.” This line of thinking turns my mind to the construct we call “dark matter.” Dark matter, which we cannot observe directly, is an astronomical construct created to explain behavior that cannot be explained by appealing to the objects in the sky we can see directly (using the verb “see” in it’s broadest sense).

It seems to me that open educational resource apologists have created a related construct that might best be called “dark reuse.” The difference between dark matter and dark reuse is significant, however. While the dark matter construct was created to explain unanticipated-but-observed behavior, the dark reuse construct is created to explain anticipated-but-unobserved behavior. Rather than accepting the message of data which indicate that reuse is occurring only very infrequently, the apologists imagine an unobservable space offline in which reuse must surely be occurring. With the irrational zeal of the too often caricatured believer, members of the Church of Reuse seem rather resilient in the face of data. (I mean no disrespect to people of faith, writing myself as a Christian with a firm faith in Jesus Christ and a steadfast hope in a better life to come.) The OOP literature has been telling us for decades that very little reuse happens in the world of object-oriented programming. If the intellectual heritage of open education runs – at least partly – through OOP to learning objects and on to open educational resources, should we really be surprised to find similar results in our sphere? I don’t think so.

The dearth of empirically verifiable reuse of OERs begs the question – where is the “work we are doing in developing “field” of open educational resources really going? What is our real goal? If our goal is catalyzing and facilitating significant amounts of reuse and adaptation of materials, we seem to be failing. And history indicates we may experience additional failure in the future.

If our goal is to create fantastically popular websites loaded with free content visited by millions of people each month, who find great value in the content but never adapt or remix it, then we’re doing fairly well. But so are CNN, the New York Times, the BBC, and other sites chock-full of freely available copyrighted content. We all understand that clearing copyright is the most expensive part of what we do…

So… what are we doing?

This post isn’t meant as a crisis of faith, just an honest question. I’m not planning to leave the world of open education anytime soon. =)

On the Lack of Reuse of OER

A student of mine, now DOCTOR Sean Duncan (congrats again!) has posted his excellent dissertation studying reuse of OERs online under a CC-BY license. This was one of the most enjoyable dissertations I have ever chaired. I’ll cover highlights below, but I encourage you to check out the full text of Patterns of Learning Object Reuse in the Connexions Repository for yourself.

The study examined patterns and amount of reuse within the Connexions OER repository at Rice. CNX seemed like a great choice for examining reuse because the system is built specifically to support both adapting individual modules and remixing individual modules into courses / collections. Importantly, through system metadata that CNX also makes openly available, all these relationships can be explored programatically in a straightforward way. So CNX is in many ways a best-case scenario for studying reuse, adaptation, and remixing.

Terminology and definitions are very important if we’re going to be doing quantitative measures of different aspects of reuse. Here are Sean’s terms and definitions.

  • Use – Count of each inclusion of an original module as-is in a collection.
  • Reuse – Count of all but the initial use of an original module as-is in a collection (i.e., Use – 1).
  • Translation – Count of each derivative of an original module where the language differs between the modules.
  • Modification – Count of each derivative of an originating module where the language does not differ between the modules.
  • Recycled – Count of each reuse, translation, and modification of an originating module.
  • All Use – Count of each use, translation, and modification of an originating module.

Sean opens the results chapter by saying:

A critical, unstated assumption made by the researcher turned out to be false, but its identification is, in and of itself, an important finding. Specifically, the researcher assumed that there was significant use of Connexions modules within Connexions collections. Because the study goal was descriptive quantitative data, this was not a fatal flaw and its early identification resulted in important methodological revisions.

And here is a sample of some of the findings:

The total count of Unique Modules Published was 5,221, but the total count of unique published Connexions modules that were used in any collection or as the originating module for another derivative module (i.e., unique modules used), was only 3,519. In other words, 32.6% of modules published in the Connexions repository are not used at all.

The next calculations were the count of how many times modules were included in any collection (i.e., the total use count), which was 4,713, the count of how many times an individual module exceeded an initial use (i.e., the total reuse count), which was 967 times, and the count of modules that were used in two or more collections (i.e., the total unique modules reused), which was 724. These calculations indicated a reuse of discrete modules as 20.57% of the unique modules used, with a reuse rate of 1.34 times for each reused module.

There were only 1,013 module uses where there was no common author [between the module and the collection].

Of the 3,519 unique modules used, 105 were translated into 174 derivatives, while 101 were modified into another 120 modules.

Ultimately, of the 3,519 modules used in Connexions, 861 of them were recycled in some way for a total of 1,262 uses. This means that almost a quarter of all modules that were used at all were recycled. Between basic reuse, translated derivatives, and modified derivatives, recycled modules were used almost 1.5 times beyond their originating object’s initial use.

Only 15 modules were used, translated, or modified more than five times (see Table 5).

As I said, read the full Patterns of Learning Object Reuse in the Connexions Repository for yourself.

To me, this study begins to confirm the “dirty secret” of OER – that the reuse emperor has no (or only very scanty) clothes. I’m planning similar studies of other collections now, and looking for graduate students to run these studies. Looking for a thesis or dissertation topic? Let me know!