A Response to “OER Beyond Voluntarism”

Well, this has turned into a rather enjoyable conversation. To recap what has unfolded so far:

  • It began with Jose Ferreira inviting me to appear on a panel at the Knewton Symposium,
  • on the panel, I made the claim that in the near future 80 percent of general education courses would replace their commercial textbooks with OER,
  • after the conference, Jose responded to my claim by telling publishers why I was wrong,
  • I responded by explaining that the emergence of companies like Red Hat for OER would indeed make it happen, using the Learning Outcomes per Dollar metric as their principal tool of persuasion, and
  • Michael Feldstein argued that it depends.

Yesterday, Brian Jacobs of panOpen published an essay contributing to the conversation. While I agree that some in the field have yet to pick up on a few of the points he makes, I’m a little perplexed that he would choose to position these points as a response to writing by Michael, Jose, and me. By making these points in a response, he implies that we have yet to understand them. Take this bit for example:

Their comments, though, didn’t tackle what I’ve come to see as the core issue for the OER movement, a foundational assumption that has crimped its progress. The assumption holds that because open-source educational content is like open-source software…its application and uses should follow in a similar way. The short history of the two movements makes clear that this is not the case.

I’ve been accused of many things in my life, but never of missing the difference between open content and open source. As the person who coined the term “open content” sixteen years ago specifically for the purpose of differentiating it from open source, I’ve never had to defend against this particular allegation. Not sure what to say.

Or this:

The OER movement’s almost singular focus on cost can obscure the larger objective — actually getting more students through to graduation while ensuring that they’ve learned (and enjoyed learning) something along the way.

when I spent almost half of the post he is responding to laying out the Learning Outcomes per Dollar metric for empirically measuring the impact of OER use on students’ academic performance. And then demonstrating with actual data from an OER adopter the incredibly powerful ways that OER adoption impacts learning.

Perhaps the article isn’t a response to Jose, Michael, and me at all. Maybe Brian is just using the conversation as an opportunity to underline a few unrelated points he feels need making, and that’s fine. And these little tidbits aren’t what I actually wanted to write about, anyway. Sorry. What I really want to do is unpack and comment on the core argument of the essay. First I’ll disagree, and then I’ll agree.

Disagreeing

As important as [the OpenStax] project is, it doesn’t yet realize the promise of OER as disaggregated high-quality content created and modified from anywhere.

Overworked and underpaid instructors are looking to content and course technology to make their lives easier, not to take on the additional responsibility of managing their own content without financial recognition for that labor.

From these and other portions of the article, I believe Brian’s argument is based on two premises:

  • In order for students to get the full benefit of OER, their faculty need to be aggregating, revising, and remixing OER – really tailoring and customizing it to meet their specific needs
  • This is a lot of additional work for faculty, and they won’t do it unless they are provided with additional incentives

Arguing from these assumptions, he arrives at the following conclusion:

This can be done by charging students nominally for the OER courses they take or as a modest institutional materials fee. When there are no longer meaningful costs associated with the underlying content, it becomes possible to compensate faculty for the extra work while radically reducing costs to students… a system for distributed content development also needs to be accompanied by a system of distributed financial incentives.

So, just stating each step of the argument explicitly to make sure I’m getting it right (hopefully he’ll correct me in the comments if I’m getting it wrong):

  • if we charge students a little when faculty adopt OER,
  • we can use a portion of that revenue to incentivize faculty to do the work of curating disaggregated OER and engaging in the revising and remixing process,
  • (because if we don’t incentivize faculty by paying them, then most will never engage in these activities), and
  • if faculty aren’t aggregating, revising, and remixing disaggregated OER, students won’t get the full benefit of OER.

I largely agree with Brian’s premises, but disagree somewhat with where he takes the argument based on them. (As I’ll argue below, this disagreement is both healthy and a Good Thing.) Here’s where I think the primary differences in our thinking lie.

The “Full” Benefit of OER

First, while I agree in theory that students don’t get the full potential benefit of OER if their faculty don’t engage in the aggregate, revise, and remix process, it’s unclear to me how much benefit students miss out on when faculty simply adopt OER “as is” (though we’re studying this question now). For example, the overwhelming majority of faculty in the college algebra example from my previous post – where passing rates increased from 48% to 60% after faculty switched to OER – did zero aggregating, revising, or remixing. Maybe the change in pass rates would have been even higher if they had, but are we really going to poo-poo an increase of 12 real percentage points in the pass rate? If students are getting much of the potential benefit even when faculty don’t aggregate, revise, and remix, is it worth incurring the additional costs necessary to achieve 100% of the full benefit? This brings us directly back to the Learning Outcomes per Dollar discussion in my previous post. What’s the delta in learning we would place in the numerator? What’s the delta in cost we would place in the denominator?

Why Don’t Faculty Remix?

Second, I disagree with the notion that not getting paid for their time and effort is the primary obstacle to faculty aggregating, revising, and remixing OER. I’ve trained hundreds of faculty in the past two years and have learned some interesting things along the way. One is that the faculty working in the institutions that serve our most at-risk students – those students who would likely benefit the most from OER – are the faculty with the least technical capability. In quite a few cases, these were faculty who needed support for technical tasks as “simple” as attaching a document to an email. Offering them $100 to remix some OER is not going to endow them with the skills – either technical or pedagogical – they need to do this effectively. That takes serious boots-on-the-ground training and support. It can be done, but it’s not a simple matter of offering a faculty stipend. (This also brings us directly back to the Learning Outcomes per Dollar discussion in my previous post.)

Builders, Adapters, and Adopters

Even in cases where faculty adoption of OER was supported by one-time grant funding (i.e., they were getting paid extra), our observation across dozens of campus visits and faculty trainings is that faculty generally fall into one of three categories: builders, adapters, and adopters. Builders have the time, interest, and skill to create, aggregate, revise, and remix OER. Adapters have the time, interest, and skill to make minor tweaks to OER that have been previously packaged in order to work “out of the box.” Adopters simply use OER designed to work right out of the box, just as they found them.

We think the distribution of faculty among these groups is something like 1% builders, 7% adapters, and 92% adopters. (As per our previous research on the number and types of changes faculty made to Flat World Knowledge’s open textbooks, “as with Duncan (2009), we found that the rates of revision and remix were relatively low. Only 7.5% of textbook adoptions over a two-year period were adoptions of custom books. This indicates that while the ability to revise and remix sounds exciting, the number of those who take advantage of this opportunity is relatively small.”) A strategy targeting the 1% – even if it grew to include the next 7% – is unlikely to have the broad impact we all hope OER will achieve. The strategy we’re looking for has to include the 92% without constraining the other 8%.

The Mythical Surplus

Another issue relating to paying faculty is that, as many of us have experienced, the offer of additional funding does not add hours to the day. Many of these faculty are already so overworked and behind on existing commitments that even with a little sweetener they can’t find the time to engage in aggregating, revising, and remixing OER. The entire notion of faculty who would remix if only they were paid assumes a professoriate with surplus time and skill who are looking to maximize their return on the expenditure of that surplus. Unfortunately, that is not the life experience of many faculty. While I freely admit that it’s a terribly hard trap to avoid falling into, this approach seems to disproportionately favor faculty at schools that are much better resourced than their community college cousins.

Incentives, Alignment, and Conflict

My final, and perhaps biggest, issue with paying faculty to adopt OER is the inherent misalignment of incentives it creates. For faculty who previously made their materials choices based primarily on what they thought was best for their students, we now throw money into the mix – “if you choose these materials, we’ll pay you!” And the incentive payment to the faculty member will inevitably be built into the cost which their students pay, raising the price for students in order to financially benefit faculty. Yuck. (Don’t most colleges have conflict of interest policies governing textbook adoptions that directly benefit faculty financially?)

Agreeing

I agree that there are costs associated with adopting OER – someone has to find, vet, properly attribute, load into the local platform, etc. the OER that will be used in classes. Sometimes a faculty member will have the time and skills to do this themselves. Sometimes an institution will provide these kinds of supporting services through the staff of their library and center for teaching and learning. Other institutions won’t have the internal capacity to provide these supports and will have to hire new people or partner with outside organizations for them.

Support Fees

In the latter cases, institutions have to find new sources of funding to pay for those new people or outside support services. There are many ways of doing this. Brian has described the “support fee” model. My experience has been that when you propose to a student “how would you feel if the school instituted a $5 or $10 course support fee in exchange for removing the $170 textbook from the syllabus?”, they happily ask “where do I sign up?” From the student perspective, the economics of this option are hard to argue against.

The INTRO Model

On Monday we’re submitting an article (for a special issue of EPAA) that introduces a new funding model we call the INTRO model – INcreased Tuition Revenue through OER. In this article we use actual enrollment, drop rate, tuition, policy, and other data from a large OER adopting institution to show that:

  • when faculty adopt OER, drop rates decrease significantly
  • when drop rates decrease, the institution refunds significantly less tuition
  • when they refund less tuition, the institution has more funding to spend on things like supporting OER adoption among its faculty

In this particular example we demonstrate that, if the current OER pilot was expanded to all sections of the 20-some courses currently piloting OER, the institution could expect to retain over $100,000 a year in tuition that they’re currently refunding. Some of this new funding could be used to pay for services supporting faculty adoption of OER without charging students.

I’m sure there are other models for funding OER adoption support services out there if we’re creative and open-minded enough to find them.

Parallel Experiments

And I am in total and complete agreement with this statement from Brian’s piece:

What’s needed are lots of entities — for-profit and nonprofit — to experiment with funding models.

YES! We need more experimentation happening, and we need it happening in parallel instead of serially. We can’t all stand around watching the Flat World Knowledge experiment, and only start trying something different when it becomes clear that their approach isn’t quite the right one. As Linus said, in what is possibly my favorite quote:

And don’t EVER make the mistake [of thinking] that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle. That’s giving your intelligence _much_ too much credit.

Even though I disagree with some of Brian’s conclusions (which is why I’m experimenting with a different business model), I absolutely want him out there experimenting with his particular business model. If I’m sufficiently humble, I’ll learn a thing or two from him before it’s all said and done. (If he’s sufficiently humble, Brian may learn something from me, too.) From this learning a new generation of models will emerge and be tested. They will be followed by another, further refined set of models. That’s how the field moves forward in its understanding of how to support OER adoption at scale, and it’s how at least 80% of general education courses will end up adopting OER in place of commercial textbooks.

{ 0 comments }

OpenCon 2014

#OpenEd14 is getting close! For a wide range of reasons, this year’s 11th annual Open Education Conference looks like it will be the best ever. One thing contributing to the awesomeness of this year’s conference is other events organized around the same time in the same area.

One of these events is OpenCon 2014: The Student and Early Career Researcher Conference on Open Access, Open Education and Open Data, organized by SPARC and the Right to Research Coalition As the name implies, this event is really focused on engaging students and early career individuals and helping them become effective advocates in the openness movement. The meeting will run from November 15-17 in Washington, D.C., and the program includes three days of talks, workshops, and in-the-field advocacy experience (leveraging its location in Washington, DC). Of course, a delegation of participants from OpenCon will also attend OpenEd.

Applications are still open until midnight tonight Pacific time – over 1600 applicants from more than 120 countries have already applied. If you fall into the student / early career category, you should definitely apply.

{ 0 comments }

I recently had the wonderful opportunity to participate on a panel about OER at the Knewton Education Symposium. Earlier this week, Knewton CEO Jose Ferreira blogged about ‘OER and the Future of Publishing’ for EdSurge, briefly mentioning the panel. I was surprised by his post, which goes out of its way to reassure publishers that OER will not break the textbook industry.

Much of the article is spent criticizing the low production values, lack of instructional design, and missing support that often characterize OER. The article argues that there is a potential role for publishers to play in each of these service categories, leveraging OER to lower their costs and improve their products. But it’s been over 15 years since the first openly licensed educational materials were published, and major publishers have yet to publish a single textbook based on pre-existing OER. Why?

Exclusivity, Publishing, and OER

The primary reason is that publishers are – quite rationally – committed to the business models that made them incredibly successful businesses. And the core of that model is exclusivity – the contractual right to be the only entity that can offer the print or digital manifestation of Professor Y’s expertise on subject X. Exclusivity is the foundation bedrock of the publishing industry, and no publisher will ever meaningfully invest in building up the reputation and brand of a body of work which is openly licensed. Publisher B would simply sit on the sidelines while Publisher A exhausts its marketing budget persuading the world that it’s version of Professor Y’s open materials are the best in their field. Once Professor Y’s brand is firmly associated with high quality, Publisher B will release it’s own version of Professor Y’s open materials, free-riding on Publisher A’s marketing spend. Publisher A’s marketing efforts actually end up promoting Publisher B’s competing product in a very real way. No, publishers will never put OER at the core of their offerings, because open licensing – guaranteed nonexclusivity – is the antithesis of their entire industrial model. Some playing around in the supplementals market is the closest major publishers will ever come to engaging with OER.

New Models Enabled by OER

However, we are seeing the emergence of a new kind of organization, which is neither invested in preserving existing business models nor burdened with the huge content creation, distribution, and sales infrastructure that a large commercial publisher must support. (This sizable infrastructure, that once represented an insurmountable barrier to entry, is quickly becoming a millstone around the neck of big publishers facing the threat of OER.) The new breed of organization is only too happy to take the role of IBM or Red Hat and provide all the services necessary to make OER a viable alternative to commercial offerings. I had to chuckle a little reading the advice to publishers Jose provides in his post, because that list of services could almost have been copied and pasted my company’s website (Lumen Learning): iterative cycles of instructional design informed by data, integration services, faculty support, etc. I agree wholeheartedly that these are the kinds of services that must be offered to make OER a true competitor to commercial textbooks in the market – but I disagree with the idea that publishers will ever be willing to offer them. That realization is part of what led me to quit a tenured faculty job in a prestigious graduate program to co-found Lumen Learning.

All that said, the emergence of these organizations won’t spell the end of large textbook publishers as we know them. Instead, that distinction will go to the simplest possible metric by which we could measure the impact of the educational materials US students spend billions of dollars per year on: learning outcomes per dollar.

Learning Outcomes per Dollar

No educator would ever consciously make a choice that harmed student learning in order to save money. But what if you could save students significant amounts of money without doing them any academic harm? Going further, what if you could simultaneously save them significant money and improve their learning outcomes? Research on OER is showing, time and again, that this latter scenario is entirely possible. One brief example will demonstrate the point.

A recent article published in Educause Review describes Mercy College’s recent change from a popular math textbook and online practice system bundle provided by a major publisher (~$180 per student), to OER and an open source online practice system. Here are some of the results they reported after a successful pilot semester using OER in 6 sections of basic math:

  • At pilot’s end, Mercy’s Mathematics Department chair announced that, starting in fall 2012, all 27 sections (695 students) in basic mathematics would use [OER].
  • Between spring 2011 [no sections using OER] and fall 2012 [all sections using OER], the math pass rate increased from 48.40 percent to 68.90 percent.
  • Algebra courses dropped their previously used licenses and costly math textbooks and resources, saving students a total of $125,000 the first year.

By switching all sections of basic math to OER, Mercy College saved its students $125,000 in one year and changed their pass rate from 48 to 69 percent – a 44% improvement.

If you read the article carefully, you’ll see that Mercy actually received a fair amount of support in their implementation of OER, which was funded through a grant. So let’s be honest and put the full cost-related details on the table. Mercy (and many other schools) are still receiving the support they previously received for free through their participation in the Kaleidoscope Open Course Initiative. Lumen Learning, whose personnel led the KOCI, now provides those same services to Mercy and other schools for $5 per enrollment.

So let’s do the learning outcomes per dollar math:

  • Popular commercial offering: 48.4% students passing / $180 textbook and online system cost per student = 0.27% students passing per required textbook dollar
  • OER offering: 68.9% students passing / $5 textbook and online system cost per student = 13.78% students passing per required textbook dollar

For the number I call the “OER Impact Factor,” we simply divide these two ratios with OER on top:

  • 13.78% students passing per required textbook dollar / 0.27% students passing per required textbook dollar = 51.03

This basic computation shows that, in Mercy’s basic math example, using OER led to an over 50x increase (i.e., a 5000% improvement) in percentage passing per dollar. No matter how you look at it, that’s a radical improvement.

If similar performance data were available for two construction companies, and a state procurement officer awarded a contract to the vendor that produces demonstrably worse results while costing significantly more, that person would lose his job, if not worse. (As an aside, I’m not aware of any source where a taxpayer can find out what percentage of federal financial aid (for higher ed) or their state public education budget (for K-12) is spent on textbooks, making it impossible to even begin asking these kinds of questions at any scale.) While faculty and departments aren’t subject to exactly the same accountability pressures as state procurement officers, how long can they continue choosing commercial textbook options over OER as this body of research grows?

#winning

Jose ends his post by saying “Publishers who can’t beat OER deserve to go out of business,” and he’s absolutely right. But in this context, “beat” means something very different for OER than it does for publishers. For OER, “beat” means being selected by faculty or departments as the only required textbook listed on the syllabus (I call this a “displacing adoption”). Without a displacing adoption – that is, if OER are adopted in addition to required publisher materials – students may experience an improvement in learning outcomes but will definitely not see a decrease in the price of going to college. Hence, OER “beat” publishers only in the case of a displacing adoption. For publishers, the bar is much lower – to “beat” OER, publishers simply need to remain on the syllabus under the “required” heading.

How are OER supposed to clear this higher bar, particularly given the head start publishers have? OER have only recently started to catch up with publishers in many of the areas where publishers have enjoyed historical advantages, like packaging and distribution (c.f. the amazing work being done by OpenStax, BCCampus OpenEd, Lumen Learning, and others). But OER have been beating publishers on price and learning outcomes for several years now, and proponents of OER would be wise to keep the conversation laser-focused on these two selection criteria. In a fortunate coincidence for us, I believe these are the two criteria that matter most.

OER offerings are always going to win on price – no publisher is ever going to offer their content, hosting platform, analytics, and faculty-facing services in the same zip code as $5 per student. (And when we see the emergence of completely adaptive offerings based on OER – which we will – even if they are more expensive than $5 per student they will still be significantly less expensive than publishers’ adaptive offerings.) Even if OER only manage to produce the same learning results as commercial textbooks (a “no significant difference” research result), they still win on price. “How would you feel about getting the same outcomes for 95% off?” All OER have to do is not produce worse learning results than commercial offerings.

So the best hope for publishers is in creating offerings that genuinely promote significantly better learning outcomes. (I can’t describe how happy I am to have typed that last sentence.) The best opportunity for publishers to soundly defeat OER is through offerings that result in learning outcomes so superior to OER that their increased price is justified. Would you switch from a $5 offering that resulted in a 65% passing rate to a $100 offering that resulted in a 67% passing rate? Would you switch to a $225 offering that resulted in a 70% passing rate? There is obviously some performance threshold at which a rational actor would choose to pay 20 or 40 times more, but it’s not immediately apparent to me where it is.

However, if OER can beat publishers on both price and learning outcomes, as we’re seeing them do, then OER deserve to be selected by faculty and departments over traditional commercial offerings in displacing adoptions.

I was the member of the panel Jose quoted as saying that ’80% of all general education courses taught in the US will transition to OER in the next 5 years,’ and I honestly believe that’s true. The combined forces of the innovator’s dilemma, the emergence of new, Red Hat-like organizations supporting the ecosystem around OER, the learning outcomes per dollar metric, and the growing national frustration over the cost of higher education all seem to point clearly in this direction.

{ 7 comments }