OER: Some Questions and Answers

Earlier this week I read an op-ed – sponsored by Pearson – titled “If OER is the answer, what is the question?” The article poses three questions and answers them. Below I share some thoughts prompted by the article. (The questions from the article are presented in bold; unattributed blockquotes are from the original article.)

How do we deliver better learning experiences to more students?

There are fantastic learning resources out there of all breeds bringing different types of value to the learning process. OER often shine in their variety and ability to deepen resources for niche topics. Where proprietary courseware (textbooks, etextbooks, or online courseware) stand apart is in pedagogical organization and the unique value of authorship. While it’s possible to build a complete course from OER, the finished product often lacks the scaffolding found in courseware authored by single author/editorial/product teams. That scaffolding connects concepts and practice together, guiding students through the content in a way that maximizes learning.

I’m glad that the author goes straight to the issue of student learning. When all is said and done, the degree to which resources like commercial textbooks and OER support student learning is the only thing that matters. (I will use the language of effectiveness rather than efficacy below, for very important reasons I discussed previously.)

Absent any effectiveness data, for decades faculty who were evaluating educational resources had no choice but to settle for characteristics of resources that reasonable people might believe correlate positively with effectiveness. These proxies for effectiveness included famous authors, name brand publishers, large product teams with diverse skill sets, Stephen Spielberg-like production quality in graphic design, layout, and imagery, and highly formalized editorial and review processes. Unfortunately, sometime during these passing decades faculty began to believe that only resources with these proxy characteristics could be effective in supporting learning. They began to doubt that other development models must necessarily result in materials that are less effective. I don’t think it would be controversial to say that this “content worldview” was encouraged by publishers.

A growing number of peer-reviewed studies and other research reports are demonstrating that when faculty who previously used commercial products as their core instructional materials replace them with OER, student learning either stays the same or increases. Hilton’s review of this research suggests that this “same or better” outcome for OER users holds about 93% of the time.

This result – that freely available resources can support student learning as well as very expensive resources – runs counter to people’s intuition that “you get what you pay for.” As we see in other areas (e.g., climate change), when the truth differs significantly from people’s beliefs, there can be a steep communications hill to climb. This has certainly been the case for OER, and is the primary reason why it is so critically important that more empirical research on the relative effectiveness of OER be conducted and published in peer-reviewed academic journals.

While we need a larger, more robust literature addressing the question of the relative effectiveness of commercial resources and OER, there is another sense in which this research is utterly meaningless. If I purchase the rights to an out-of-print textbook from Pearson and relicense it CC BY, is it now more or less effective than it was the day before? This is a ridiculous question.

The question of whether the language on the copyright page will significantly influence student learning is completely irrational, yet such are the questions we must answer over and over again. As people continue to raise questions in the face of mounting evidence, I’m reminded of Nobel Prize-winning economist Daniel Kahneman, who wrote:

The mystery is how a conception that is vulnerable to such obvious counterexamples survived for so long. I can explain it only by a weakness of the scholarly mind that I have often observed in myself. I call it theory-induced blindness: Once you have accepted a theory, it is extraordinarily difficult to notice its flaws. As the psychologist Daniel Gilbert has observed, disbelieving is hard work.

There are no results from the instructional design, learning science, or cognitive science literature demonstrating that the language on the copyright page is a critical factor in promoting student learning. There is a growing body of research demonstrating that OER can be just as effective – or more effective – than commercial materials. It should be obvious to anyone that the features of instructional materials that effectively support learning (e.g., frequent formative assessment opportunities) can appear in educational resources with any copyright license.

The initial empirical results look good for OER, and we should hope they continue to hold. If alternative development models can continue to produce materials that support learning as effectively as the traditional publisher models, there is undoubtedly a positive future for OER. However, if we have to spend $1M per open textbook to achieve effectiveness results on par with commercial publishers, the long-term sustainability of OER is seriously in question.

Can an instructional design-minded instructor provide that missing connective tissue? Absolutely. But how many have the time and motivation to do so? The amount of work required to do it well is one of the major reasons those who can and want to do it also want to be fairly compensated for it. Best-selling authors consistently deliver high-quality, engaging learning experiences that their audience recognizes and seeks out. They do this through complete pedagogical systems that support the educator and learner better than others do, and that’s tough to replicate.

There are two closely related issues here that we can address together using the Drake Equation as a scaffold for our thinking.

The Drake equation estimates the number of civilizations in our galaxy we might be able to communicate with by applying a series of filters, as follows:

N = R* x fp x ne x fl x fi x fc x L; where:

N = the number of civilizations in our galaxy with which communication might be possible (i.e. which are on our current past light cone); and

R* = the average rate of star formation in our galaxy
fp = the fraction of those stars that have planets
ne = the average number of planets that can potentially support life per star that has planets
fl = the fraction of planets that could support life that actually develop life at some point
fi = the fraction of planets with life that actually go on to develop intelligent life (civilizations)
fc = the fraction of civilizations that develop a technology that releases detectable signs of their existence into space
L = the length of time for which such civilizations release detectable signals into space (via Wikipedia)

What does the Drake Equation have to do with OER production and enhancement? It suggests that we might look at issues of faculty time and incentives raised in the op-ed as Drake-like filters:

N = Ce * Pe * T * M; where:

N = number of people who create or improve effective OER; and

Ce = the number of people with sufficient Content Expertise to create or improve effective OER
Pe = the fraction of those people with sufficient Pedagogical Expertise to create or improve effective OER
T = the fraction of those people with Time available to work on OER
M = the fraction of those people with sufficient non-financial Motivation to spend time working on OER

What might this look like in practice? Here is an extremely back of the envelope example for Introduction to Biology:

Ce = 2 teaching faculty per post-secondary institution x 4000 post-secondary institutions = 8000 people
Pe = .5
T = .1
M = .1

N = 8000 * .5 * .1 * .1 = 40 people

I’m confident that both this specific model and these specific numbers are wrong, but I present them in the spirit of Box’s “all models are wrong, but some are useful.” As long as N > 0, extremely interesting things can happen. Even in the case of N = 1 – like the cases of Sal Khan or James Sousa – there can be an incredible impact on teaching and learning. Both Khan and Sousa created thousands of short instructional videos and published them under open licenses. How many more people need to do this work? Certainly not every instructor in the country. The nonrivalrous nature of digital resources (the technical ability to share copies of resources at almost no cost) combined with open licenses (the legal permission to share copies of resources at no cost) means that only a handful of people need to be actively involved in producing or making substantive improvements to OER in order for the public to have free and open access to resources whose effectiveness is on par with those created by commercial publishers.

Perhaps most importantly, the major efforts made by OER workhorses like Khan or Sousa catalyze additional, incremental work by others over time. As Benkler has explained, the smaller the contribution to be made, the more people there are who will have the time and inclination to contribute (c.f. Shirky’s idea of cognitive surplus).

This creates the opportunity for asynchronous, uncoordinated, incremental, continuous improvement that harkens back to Eric Raymond’s notion that “Every good work of software starts by scratching a developer’s personal itch.” Individual instructors make no improvements, small improvements, or large improvements to existing OER based on their own needs and available resources. Some subset of the group that makes changes share those changes back with the community. This kind of “snowball development” is a key characteristic of the most interesting and effective OER.

The future of the sustainable development of effective OER will be characterized by stigmergy. Stigmergy is the watchword for the next decade of OER.

Commercial publishers and those who think along similar lines will immediately point out problems with this alternative development model. The first of these problems relates to the notion of “fair compensation,” incentives, and sustainability raised in the op-ed. This argument harkens back to the classic view of why public goods are underproduced:

Public goods provide a very important example of market failure, in which market-like behavior of individual gain-seeking does not produce efficient results. The production of public goods results in positive externalities which are not remunerated. If private organizations do not reap all the benefits of a public good which they have produced, their incentives to produce it voluntarily might be insufficient…. If too many consumers decide to “free-ride”, private costs exceed private benefits and the incentive to provide the good or service through the market disappears. The market thus fails to provide a good or service for which there is a need. (via Wikipedia)

Like much of classical economics, this model of a world of perfectly rational people acting in their own economic self-interest is simply untrue. Now remember that a model can be wrong, but still be useful. However, in discussing OER this model is neither correct nor useful. It lacks an account of why people volunteer for or donate their time, money, and effort to a range of charitable and other causes, including the creation, improvement, and maintenance of open source software and open educational resources. Theoretically, it’s impossible for anything open to exist at any scale or quality – after all, there is no market incentive for their creation. Yet, somehow, the overwhelming majority of the internet still runs on Linux, Apache, Ruby on Rails, Node, WordPress, and other open source infrastructure. And increasingly, higher education courses are running on an open content infrastructure (i.e., OER).

Unfortunately, while commercial publishers do have a market incentive to invest in the creation and update of educational materials, those incentives are aligned primarily with selling the materials and only secondarily with supporting learning. Much – though certainly not all – of the investment and innovation coming out of commercial publishers over the last decade has been in things like ingenious DRM systems that cause natively nonrivalrous digital content to become artificially scarce so that it can be better monetized; the frenetic pace of creation and release of new editions of textbooks, where the main purpose of the new edition is to undercut the used book market in previous editions; and the creation of digital rental and other “streaming access” type of services where students only license temporary access to content and never own anything, in order to eliminate used markets altogether. Yes, publishers have a financial incentive to create educational materials, but it is not primarily aligned with student learning.

Publishers will also likely come back to the core argument that their production process guarantees “high quality and engaging experiences,” with the subtle implication that other processes either cannot or likely will not. First, this argument is tautological if you have convinced listeners that “high quality” means “the set of proxies for effectiveness that traditional production processes create.” Before there can be any progress made in the educational materials or broader educational technology markets, we have to shed vague notions of “high quality” and have a laser-like focus on effectiveness. When faculty begin basing their adoption decisions on effectiveness rather than publisher, author, or the likelihood that a textbook’s interior photos to win a Pulitzer Prize for Photography, the market will start moving in a direction that better supports learning.

Second, there is a subtle slight of hand that frequently goes unnoticed in the quality argument (even if we assume that quality means effectiveness). The argument goes that student learning will likely suffer because of the probably poor quality of OER. Now, there might be some truth to a similar argument when discussing students’ critical information literacy in their use of resources they discover on the open internet. But who chooses the core instructional resources students will use? Faculty do. If you believe you can honestly argue that faculty don’t have the basic content or pedagogical knowledge necessary to effectively select core instructional materials, then the materials selection process should be very low on your list of concerns for student learning.

How do we get the most current, updated content when we want it?

When it comes to revising and remixing content, OER hold some advantages over the traditional textbook revision cycle. The ability to customize for a specific region or update to reflect recent world events is very academically appealing and can yield more relevant, up-to-the minute content. But who ultimately owns the responsibility for updating and refreshing content? Is it up to each individual instructor? Would department heads be responsible for making the call? How would the quality of these updates be assured? In research intensive disciplines, who is responsible for completing ongoing literature reviews, distilling the best new research, and updating course materials? Who maintains working problem sets over time for disciplines requiring them? And what is the incentive to keep the cycle going? There are some intrepid instructors who would eagerly take this on, but when you consider the implications for content updates at scale, demand can far outstretch capacity. Without clear responsibilities, incentives, quality controls, and a repeatable process for managing content updates, there are substantial costs imposed upon the user of OER for its upkeep.

I believe I addressed all of the questions asked in this paragraph in my concise (!) responses above.

How can we drive down costs for students and for institutions?

This seems to be the most likely question that OER seek to answer. And underpinning this is the presumption that “open” means “free.” But as mentioned above, there are significant costs imposed on both the learner (e.g., pedagogical inconsistency) and the instructor (e.g., material curation and upkeep).

As I have explained at length before, most recently through an analysis of the word “open”s usage in a cluster of interrelated contexts, the definition of open in contexts like “open educational resources” has two critical components – (1) free, plus (2) 5R permissions. There’s a lack of sophistication in the way most people talk about the “free” component of the definition of open, including the op-ed.

All educational resources – whether open or commercial – can be used to support learning only with some effort by the instructor. (We’ll ignore the obvious fact that learners must also exert effort to learn from resources, which is captured in the discussion of effectiveness vs efficacy linked above.) When a publisher releases a new edition of a book and stops selling the old one, forcing an instructor to change texts, or when an instructor chooses to switch from one author’s book to a second author’s book, or when an instructor chooses to switch from commercial materials to OER, there is a significant amount of effort required on the part of the instructor. You may have heard faculty complain about how many “preps” they have this semester. This effort can typically be converted into a cost by using the amount of time an instructor earning a certain amount per hour spent on the task. (I suspect I would disagree with the op-ed author about the relative amount of effort necessary to be ready to teach effectively with commercial materials vs OER. No doubt we would both likely overestimate in favor of our preferred solution.) In this sense, no materials – whether commercial or open – can ever be used without incurring cost. Therefore, in some sense it costs money to use OER in support of learning.

In an entirely different sense, publishers work very hard to ensure that they are able to collect fees from all the students who use their materials. The absolute control with which these rents can be extracted online is one of the main drivers for publishers moving to digital. Legally enforceable Terms of Use can prohibit students sharing account information. The deployment of DRM (plus the DMCA) can make clever ways of accessing materials without paying a crime. There must be a royalty paid for each and every use of publisher materials.

In contrast, by definition OER never require a license fee or royalty to use. You have free, perpetual, and irrevocable permission to retain, reuse, revise, remix, and redistribute OER. Being devoid of any licensing fee or royalty, OER are clearly much more affordable for students than commercial resources. Even at institutions where a small support fee (e.g., $10) is charged per course to provide support to faculty in their use of OER, OER are still significantly less expensive that commercial materials.

Who is responsible for ensuring ADA compliance of OER? How are necessary integrations with student information and learning management systems supported? Who provides technical support to learners and faculty when needed? There is no free lunch, and most OER proponents understand that. That’s why businesses are beginning to sell services around OER, perhaps in anticipation of the day when the foundation, government, and venture capital funding that has kept OER afloat begins to dry up.

The answer to the questions of “who [plays some critical role]” is “anyone who is willing and able.” Often this will be individuals, but I believe there is also significant potential value to instructors in the emergence of organizations that are willing to take on an OER stewardship role – organizations that make it easier for faculty to adopt OER effectively. This belief in the value of such organizations, shared by myself and my partner Kim Thanos, is why Lumen Learning exists. As a rule, Lumen does not create new OER. Instead, we try to provide a simple, supported pathway for faculty members to adopt and teach effectively with OER – including LMS integration, technical support, and many of the functions mentioned above.

But we didn’t create Lumen out of fear that one day foundation, government, or other funding will dry up. If anything, the increasing adoption of policies by governments and foundations requiring that copyrightable materials created by their grantees must be openly licensed (e.g., see the two dozen or so Foundation open licensing policies and this list of seventy-some state and federal open licensing policies) are ensuring that there will be new OER created and shared for a long time to come. Instead, we created Lumen to accelerate the effective adoption of OER – because we believe that OER adoption will greatly improve access, affordability, and learning for students while simultaneously greatly expanding pedagogical freedom for faculty.

OER are a great way to enrich and personalize instruction alongside core instructional resources. But the hidden costs of OER make them a dubious replacement for primary course material, unlikely to deliver substantial savings over proprietary digital solutions in the long run. Low-cost OER make great supplements, but there is real value in core instructional content presented systematically and updated regularly by invested authors. Today’s highest-quality, most in-demand proprietary content is the product of an ecosystem that recognizes and rewards that value. That isn’t likely to change in the future. One good thing about the OER buzz is that it has opened up the conversation about the cost of course materials and our collective responsibility for improving college affordability. We’ll just have to be careful that we’re not sacrificing the quality of the learning experience in the pursuit of lower cost.

I fully agree that “there is real value in core instructional content presented systematically and updated regularly by invested authors.” I simply disagree that the only mechanism for organizing these collective efforts is the promise of royalty payments. Regardless of whether or not the alternative incentive models employed by creators and improvers of OER should be theoretically viable according to standard economic models, these models are viable and they are flourishing. In addition to making great supplements, OER make great replacements for core instructional materials – and the peer-reviewed research on the topic demonstrates that OER are generally at least as effective as the commercial materials they replace. The significant cost savings, comparable student outcomes, and greater pedagogical flexibility facilitated by OER are some of the reasons why dozens of colleges across the country are at this very moment switching their core instructional materials from publisher resources to OER across entire degree programs. These OER-based degrees (or “Z Degrees”), in which general education courses and required courses in the major all use OER instead of commercial materials, can cut the cost to graduate by 25% or more at community colleges. These programs are growing rapidly (most recently with encouragement from Achieving the Dream) and will be important to monitor.

There is indeed an ecosystem of educational materials. Until very recently that ecosystem only contained materials developed by a handful of companies through largely homogenous processes. These processes produced materials that were terrifically expensive, difficult to distinguish from one another, and somewhat draconian in their approach to copyright. Now the ecosystem also includes OER – materials that are developed through heterogeneous processes, are royalty free, are almost infinite in their variety, and provide faculty and students alike with incredible flexibility via open licenses.

It will be interesting to watch the educational materials ecosystem evolve over the next several years. What will the new equilibrium state look like in 2020?