I’ve written about this before, but here we go again…
In educational research there is a long and storied history of people conducting studies along the lines of “is video-based instruction more effective than audio-based instruction?” or “is text-based instruction more effective than audio-based instruction?” or “”is video-based instruction more effective than text-based instruction?,” etc. This pointless family of research has a name, the “No Significant Difference Phenomenon,” and even has it’s own website: http://www.nosignificantdifference.org/. From the website:
This website has been designed to serve as a companion piece to Thomas L. Russell’s book, “The No Significant Difference Phenomenon” (2001, IDECC, fifth edition). Mr. Russell’s book is a fully indexed, comprehensive research bibliography of 355 research reports, summaries and papers that document no significant differences (NSD) in student outcomes between alternate modes of education delivery…. The primary purpose of the NSD website is to expand on the offerings from the book by providing access to appropriate studies published or discovered after the release of the book.
Hundreds of “horse race” studies comparing alternate modes of education delivery show us that nothing interesting happens in these studies. Indeed, careful forethought will demonstrate that we should expect to find nothing interesting in these kinds of studies. And yet eager graduate students and younger faculty continue to conduct them.
Unfortunately, I’m hearing more and more people talk about a new generation of media comparison studies – “License Comparison Studies.” These absolutely pointless studies would ask questions like “do CC BY-NC-SA licensed materials teach more effectively than traditionally copyrighted and licensed materials?” or “do CC BY-SA licensed materials teach more effectively than CC By-NC-SA materials?” Again, careful forethought will demonstrate that we should expect to find nothing interesting in these kinds of studies. Please, if you’re in a position to discourage these kinds of studies, save all of us the trouble and embarrassment by steering your students and colleagues in another direction.