Being Clear on “High Quality”

Some readers are misinterpreting my critique of the phrase “high quality” as it’s used with regard to textbooks and other educational resources. Let me reiterate my points and then provide a new example that hopefully sheds more light on what I’m criticizing and what I’m not.

My problems with the phrase “high quality” are two-fold: (1) how the phrase gets equated with a single authoring process to the exclusion of all other authoring processes, and (2) how that usage distracts us from the efficacy conversation.

I do not have any issues with the traditional authoring process itself. As I wrote yesterday, “I fully believe that resources created through the ‘traditional process’ can effectively supporting learning.” Not only do I not have an issue with the traditional process itself, I don’t have an issue with the people or organizations who employ the traditional process – in fact I count some of them as good friends.

To be clear, my first issue is with the way “high quality” is often equated with the traditional process and that process only. According to this usage, if you don’t follow the traditional authoring process it is literally impossible for you to create “high quality” materials. This restrictive usage serves to lock out alternative processes from competing in the marketplace. The usage makes it impossible, for example, for commons-based peer production models to result in “high quality” resources (because “high quality” was previously defined as “adhering to the traditional process”). That’s just plain wrong. We need to create more openness to alternative authoring models in the minds of faculty. The traditional process can create effective learning materials, but so can other processes. These alternative processes likely share some common functions with the traditional process – like quality reviews – but the specifics of how those functions are performed can differ significantly across processes (e.g., focused review by a smaller number of experts versus ongoing review by a much larger number of peers) and still result in effective educational materials.

My second issue is the way this misuse of “high quality” omits any consideration of results, and tends to distract us from the efficacy conversation we need to be having. As this series of posts demonstrates, “high quality” can mean many things. But for the sake of our students, the one thing we most desperately need it to mean is “effective.” It’s quite simple – in order to be considered “high quality” educational materials must be effective. We need to help faculty understand that the phrase “high quality” is often disconnected from measures of efficacy when used by publishers to describe textbooks and other instructional materials, and we need them to help bring effectiveness back into the conversation.

Let me give an example to clarify exactly what I’m calling out and exactly what I’m not.

Take OpenStax as an example. These guys are doing God’s own work, creating a catalog of open textbooks for high enrolling courses using the most open of all the Creative Commons licenses. They seem to use a fairly traditional authoring process, and that’s absolutely fine. Again, the process that any person or organization uses is entirely outside the scope of this critique.

What is absolutely relevant to the critique is this – I’ve never heard OpenStax badmouth OER that were created using other processes solely because they were created using alternative processes. Even though they appear to use a fairly traditional process internally, they don’t try to perpetuate the “high quality if and only if traditional process” myth. As you would expect from leaders in the open educational resources community, they’re open minded about the possibility of other authoring processes resulting in effective materials.

The second point to make in the context of my critique of the way “high quality” gets misused is that OpenStax actually cares about efficacy. In fact right now there’s a huge banner on their homepage inviting people to participate in an efficacy study being conducted by a highly respected third party (the OER Research Hub). OpenStax aren’t trying to distract people from the efficacy conversation; on the contrary, they’re dedicating homepage real estate to trying to get people more involved in the conversation.

I provide the OpenStax example in order to clarify that (1) I’m not criticizing the traditional authoring process itself, and (2) just because an organization uses a more traditional authoring process doesn’t mean that they’re making the mistakes that I think the field at large needs to stop making. My critique of “high quality” is simply that it is being defined far too narrowly – unnecessarily excluding alternate authoring processes and omitting effectiveness. Perhaps rather than needing to stop using the phrase “high quality” we instead need to engage in a campaign to “redefine high quality” so that the phrase emphasizes efficacy and is more accepting of diversity in authoring processes.