No, Really – Stop Saying “High Quality”

Last week I wrote that we should stop saying “high quality” when discussing learning materials. Some have questioned whether or not that’s true. It is true, and here’s why.

The problem with the phrase “high quality” as used by traditional publishers is that it puts process over outcome. If publishers were basketball players, they would say, “When I shoot free throws, I align my toes with the foul line and square my shoulders to the basket. I slow my breathing and count to 5. I dribble three times, exhale once more, and then shoot, making sure to keep my elbow in and fully extend my arm.” Honestly, who cares? What you really want to know about a basketball player is whether or not he makes his foul shots. You aren’t going to draft him based on his free throw shooting process – you’re going to draft him based on his free throw shooting percentage. If the player you’re vetting shoots underhanded but makes over 90% of his foul shots, you’re going to draft him. The same is true with a salesperson – you don’t care about her sales process, you care about the number of sales she closes. Or with a baseball player – you don’t care about his batting process, you care about his batting average. Or with a network engineer, you don’t care about her specific troubleshooting process, you care about whether your employees can reach the internet or not.

So why, why, why, would we accept a publisher telling us that “high quality” is a function of process and not a function of results? Publishers want “high quality” to mean educational resources that are “written by experts, copyedited by professionals, reviewed by peers, laid out by graphic designers, and provided in multiple formats,” with literally no reference to results. What we need to know is how much do students who use the resources learn? But as a community, faculty largely accept publishers’ claim that “process = high quality” and don’t ask for outcomes or results data as part of our textbook or other materials adoption process.

And this notion is absolutely critical for the field of open education to understand: it is clearly in publishers’ best interest to focus faculty on process rather than outcome. By (1) equating “high quality” with process rather than results, and then (2) creating extremely complex authoring processes they proclaim to be “the industry standard,” publishers are attempting to create a barrier to entry for other would-be creators of educational resources (like many OER authors). “Oh, you can’t afford to replicate our elaborate publication process? That’s too bad, because our process is synonymous with high quality. Ergo, your materials are low quality.” And see? There’s literally no appeal to results in this argument, only slavish devotion to process. It’s a blatant attempt by publishers at keeping fresh competition – including OER – at bay.

In this bizarro world where results don’t matter, resources that produce better learning results than content produced using the traditional process are described as low quality. Huh?!? Encouraging people to talk about results instead of process – encouraging them to avoid nebulous phrases like “high quality” in favor of words like results, outcomes, or efficacy – is about taking back the conversation from publishers and focusing it where it belongs.

Now, I fully believe that resources created through the “traditional process” can effectively supporting learning. But there are two things I don’t believe:

1. That conformance to the traditional process guarantees that every resource created that way will effectively support learning, and
2. That the traditional process is the only process that can result in resources that effectively support learning.

There has to be a recognition by faculty – if not an admission by publishers – that alternate development processes can result in highly effective educational materials. But currently there’s not. It feels a bit like we’re trapped in 2005, still arguing over whether or not the Wikipedia authoring process can create writing as accurate as the Encyclopedia Britannica process. We settled this argument ten years ago. What are we still arguing about?

Postscript. In the comments on my first “Stop Saying High Quality” post, one commenter asked for a concrete example of OER significantly outperforming commercial materials. His comment makes the point of the argument while seeming to completely miss it – “Oh yea? Show us proof we should stop saying high quality!” To oblige the commenter, you can see an example of a college abandoning a Pearson textbook and MyMathLab bundle in favor of OER and the open source MyOpenMath practice system published in Educause Review. (Spoiler alert: Pass rates rise from 48% to 60% from Spring 2011 to Spring 2013.)