When the “Wiki Way” = Poor Quality

Open Education News points to a Scientific American article covering the California Learning Resource Network’s reviews of 16 open science and math textbooks for coverage of CA state standards. These reviews support schools making adoption decisions about whether or not open textbooks are of sufficient breadth and quality to be formally adopted in place of commercial textbooks.

Brendan Borrell, the SA article’s author, points out that “the front-runners [in the CLRN reviews] were typically written by just one or several authors, and the one major organization that has fully embraced a Wiki approach failed to impress CLRN reviewers.” This could have been, and in fact was, predicted long before.

A number of years ago, the USU Center for Open and Sustainable Learning commissioned noted open source expert Yochai Benkler to write a monograph applying his “comons-based peer production” model to educational resources. The result was Common Wisdom: Peer Production of Educational Materials. In this 28 page monograph, Benkler argues that the distributed, “Wikipedia model” of content production does not work for textbooks:

Textbooks that look and feel like textbooks, and, more importantly, that comply with education department requirements, are not quite as susceptible to modularization as an encyclopedia or a newsletter like Slashdot. The most successful book on Wikibooks, for example, is the cookbook. But the cookbook had 1301 “chapters” as of July of 2005. In other words, each module was effectively a single recipe. In this, it is much more like Wikipedia, with discrete, small contributions as the minimal module. Real textbooks appear to reside somewhere between a novel and an encyclopedia in the degree to which they can be modularized, or at least in the degree of effort required to integrate the modules into a coherent whole recognizable as a textbook…

At the moment, however, no working project has in fact implemented a platform that modularizes the work in sufficiently fine-grained chunks to allow a large pool of contributors. As I have elsewhere discussed in great detail, the size of the potential pool of contributors – and therefore the probability that the right person with the right skills, motivation, and time will be available for the job – is inversely related to the granularity of the modules. The larger the granules the more is required of each contributor, the smaller the set of agents who will be willing and able to take a crack at the work. On the other hand, the granularity is determined by the cost of integration—you cannot use modules that are so fine that the cost of integrating them is higher than the value of including the module. The case of textbooks seems to be, at present, precisely at the stage where the minimal granularity of the modules in some projects – like FHSST – is too large to capture the number of contributions necessary to make the project move along quickly and gain momentum, whereas the cost of integration in others, like WikiBooks, is so high that most of the projects languish with a module here, and module there, and no integration.

Yochai’s argument is part of the reason Flat World Knowledge uses the “a few expert authors model” for its open source textbooks, as opposed to a come-one-come-all volunteer-based approach. CA’s initial review of the open high school textbooks available today seems to bear Yochai’s arguments out.