A New Kind of Media Comparison Study

I’ve written about this before, but here we go again…

In educational research there is a long and storied history of people conducting studies along the lines of “is video-based instruction more effective than audio-based instruction?” or “is text-based instruction more effective than audio-based instruction?” or “”is video-based instruction more effective than text-based instruction?,” etc. This pointless family of research has a name, the “No Significant Difference Phenomenon,” and even has it’s own website: http://www.nosignificantdifference.org/. From the website:

This website has been designed to serve as a companion piece to Thomas L. Russell’s book, “The No Significant Difference Phenomenon” (2001, IDECC, fifth edition). Mr. Russell’s book is a fully indexed, comprehensive research bibliography of 355 research reports, summaries and papers that document no significant differences (NSD) in student outcomes between alternate modes of education delivery…. The primary purpose of the NSD website is to expand on the offerings from the book by providing access to appropriate studies published or discovered after the release of the book.

Hundreds of “horse race” studies comparing alternate modes of education delivery show us that nothing interesting happens in these studies. Indeed, careful forethought will demonstrate that we should expect to find nothing interesting in these kinds of studies. And yet eager graduate students and younger faculty continue to conduct them.

Unfortunately, I’m hearing more and more people talk about a new generation of media comparison studies – “License Comparison Studies.” These absolutely pointless studies would ask questions like “do CC BY-NC-SA licensed materials teach more effectively than traditionally copyrighted and licensed materials?” or “do CC BY-SA licensed materials teach more effectively than CC By-NC-SA materials?” Again, careful forethought will demonstrate that we should expect to find nothing interesting in these kinds of studies. Please, if you’re in a position to discourage these kinds of studies, save all of us the trouble and embarrassment by steering your students and colleagues in another direction.

3 thoughts on “A New Kind of Media Comparison Study”

  1. I keep Russell’s book on my office desk not because I reference it so frequently, but because it’s handy in conversations with inimical faculty and administrators.

  2. I try — but it isn’t so easy to dissuade people from the idea that no matter what we’ve seen via comparison studies in the past X is the technology/medium/whatever that will change everything. What you’re describing here seems to take it to a new level. What is the argument for licensing having an impact on instructional effectiveness?

  3. I’ve found that the best way to approach these kinds of topics is that technology equals tools. Tools are tools, nothing more. It’s not even what you do with them that matters as much as how and when you use them. I think the best education example of this is the publication use of the internet.

Comments are closed.