I appreciate the usefulness of open educational resources in supporting informal learning as much as anyone. I also care very deeply about the adoption and use of open educational resources in formal education settings. The kinds of things I lay awake at night worrying about differ depending on which of the two I’m thinking about when I go to bed.
The more people I talk to, the more convinced I am that OER has failed to establish a digestible value proposition for formal education. For better or worse, many people caught up in the day-to-day vortex of teaching, advising, mentoring, and grading don’t have the spare time to problematize publisher-school power relations, realize the virtue of local control of curriculum materials, or fully appreciate the transformative benefits of transparency.
We need to refine our messaging if we mean to impact formal education – particularly in K-12 where so many curricular decisions are made “above” the individual teacher. Perhaps our messaging can take a cue from the intersection of the current, outcomes-obsessed political climate and the slashing of school budgets in response to global economic realities. Perhaps we should begin discussing a “golden ratio” of open educational resources that compares (1) (differences in outcomes) with (2) (differences in cost) when a OER are used instead of traditional, proprietary educational materials.
(1) I’ve written at some length about why we should anticipate the delta in learning outcomes to be near zero when comparable open educational resources and proprietary educational curriculum are measured against one another. When teachers actively take advantage of the local control provided by OER licensing and engage in substantive adaptation / localization exercises, we can reasonably hypothesize an improvement in student performance. Either way, I believe we can anticipate the “differences in outcome” factor to be zero or positive. The appropriate unit for this factor is probably a standard deviation.
(2) Differences in cost need to be accounted for completely. Time spent reviewing traditional textbooks and other curriculum materials should be compared to time spent finding OER. The costs of purchasing or licensing traditional materials, distributing at beginning of term, collecting at end of term, and storing / managing between terms should be compared to the costs of storing, standards aligning, etc. open educational resources. Costs of keeping OER up-to-date should be compared with textbook replacement costs or annual licensing fees for online curriculum. Et cetera. The appropriate unit for this factor is probably percentage change in the organization’s curriculum spend.
That gives us a golden ratio of OER that looks something like:
change in performance (as standard deviation) : change in money spent on curriculum (as percentage)
Now, it is terribly important to note that a great finding like [+0.2 : -7%] is only applicable to the specific open educational resources studied – THE FINDING DOES NOT EXTEND TO ALL OER. However, if we could demonstrate either (a) stable performance and money saved, or (b) performance gains and money saved, several times across different grade levels and subject matters, then we would have an argument that formal education would have a very difficult time ignoring. If we can’t show one of these two outcomes, we should seriously reconsider our work in the field.
Second, and perhaps even more importantly, I don’t think I know of any OCW or OER projects looking seriously at either of these factors (though the recent CMU OLI paper in JIME is obviously headed in the right direction). If you know of any, please drop a comment below.
What do you think? Should OER have to “put up or shut up”? If so, what metrics would you use besides learning gains and cost?
This post surely deserves more thought than my immediate response, and I know I’m completely ignoring the main (and obviously most important) point of the post but I can’t ignore this: I see a theoretical problem with your “golden ratio” metric. Whenever a metric is a ratio, the first thing that should be examined is the denominator. A very small denominator can unreasonably inflate the metric. Since the denominator of your metric is a difference (as a percentage), all one must do to show a fabulous metric is to find a situation where the cost for OER and the cost for PEC (Proprietary Educational Materials) are extremely close. As the denominator of your metric (even as a percentage) approaches zero, the more outstanding the overall ratio becomes! And in the (very unlikely) case where the cost of OER and PEC is equal, then the value of your metric is undefined.
You may counter or hypothesize that OER will obviously cost less than PEC, even after accounting for cost of finding, cataloging, distributing, etc. This also is a problem with your metric, because if both of these hypotheses hold (0 to positive outcome and less cost for OER), then the value of your metric will be negative (as in your example +.2/-7%). The combination of the two problems could then result in a fabulous negative metric. I don’t think that was your plan. Is it possible that I’ve misinterpreted something here?
Can I get philosophical here? If not, ignore the comment.
My wife and I currently home school two of our four children. The two in public ed will be coming home next year – all four have come to the decision themselves.
The more comfortable we become with the concept of home schooling, the more we realize how inexpensive it gets when it is done as a collaborative effort.
Each of our children learns in a different way. They are also fascinated by different topics. We use their interests to teach them the concepts that we know that they need. There is much math art and music as there is in architecture – if you teach it that way.
I am also an adjunct college professor. In each of my roles as a teacher, I spend more time finding material that has already been prepared as I do preparing material from scratch. Why reinvent the wheel?
Referring back to the questions in the post, we must first analyze the expense of traditional education. We must analyze the time spent creating a homogenized system.
This week (fun week between Thanksgiving and Christmas break) my 12-year-old son is using the microscopes in school. These are $600 microscopes. The teacher waits until fun week to get them out. They are doing scientific projects, but they are not being scientific.
Tonight we sat down at the Internet and I showed him magnified images of all kinds of elements, objects and organisms under standard and electron microscopes. In class they had only looked at prepared slides and their own hairs.
My son wants a microscope now so that he can explore for himself since they didn’t have time to do anything fun in class with it.
I found it strange that the teacher waits until fun week to get out the microscopes to learn with, but doesn’t let the students learn by playing and experimenting. In the end his real fascination came from viewing images on the Internet, not actual images from the microscope he used in class.
Education is far less expensive when it is tailored toward actual learning. Teaching is far less expensive when it is done via collaboration. We must bust the myth that there is only one way to teach, one way to learn and that we have it all figured out.
As soon as I began teaching my college courses I realized that students are handicapped. They have been told exactly what is expected of them from their earliest years. Now they don’t know how to act when a professor or boss asks them to go out and learn then come back with a great report. They get so caught up in the number of pages, sticking to the style guide and font size that they turn in crap. Sure, it meets the requirements in style and length but it is absent of any proof that they actually learned anything in the process.
My last assignment was simple – Predict the future of social media and Internet communications. I asked them to use 5-7 sources and make it longer than 3 pages. That’s all I wanted. They grilled me for 20 minutes in class about the structure I was looking for, the types of sources they could use, etc. Some days I leave more frustrated over the system than any of my students.
Oh, and I think that Open Educational Resources are, by definition, any resources that allow a student to learn and progress in the given field. As soon as we define them, certify them and regulate them, all of our problems come back.
Skills, not certification will matter most to the next generation of employers. Proof of education will be measured in abilities and experiences. Of course there are exceptions to this, but why should a mechanic prove that he went to mechanic school when he has been watching and making YouTube auto repair videos since he was 12? Why should an art teacher prove they have been to art school when they have an online gallery that is selling works for $5,000 each?
The teachers of tomorrow will be those who succeeded pursuing a passion that lead to an education not the other way around. Passionate teachers will lead to more passionate students and the cycle continues with ever-increasing knowledge, increased productivity, a stronger economy and a happier society.
The institutional cost issue gets lost at higher ed where students carry the financial burden for most of their resources. I think it’s very important to infuse some empathy for that stress into colleges and universities.
“what metrics would you use besides learning gains and cost?” I don’t know about “besides” but in addition, “costs” could also reflect *student’s* costs, not just institutional costs (I think this is what Jeremy was getting at above.)
Another problematic value proposition on which people have tried to sell formal OER projects is “extending marketing reach” – presumably if one could show that there was some direct benefit to institutions’ registrations from having made their resources open, this would also be part of a “digestible value proposition.” Problematic, because this (along with framing these as “projects” rather than core philosophies and practices) is arguably one of the things that keeps some OER from becoming part of core teaching and learning practice.
In any case, I agree wholeheartedly – too often we have not made a convincing argument *to the people who actually need convincing* on why they should open up and share. I think ‘ratios’ can help. I think changing the frame of the conversations (e.g. ending “use monopoly,” not enforcing “copy rights”) is just as important, if not more. Once you accept a different set of frames, the arguments are a lot easier to make 😉
Kudos for the “Golden Ratio” – it follows in the footsteps of the activity-based costing methods used in distance/online learning where – yes indeed – all activities and their costs, including cost to students and other stakeholders are ideally included.
I think part of the challenge with the OER value proposition is that we are attempting to define it as the value of an incremental change in the existing institutional context. Again, lessons from costing and impact assessment in distance/online, tend to show no significant difference and some slight positives. The full value of the innovation is not realized until it is used in a context that can take full advantage of it. In online/distance those are institutions that build their operations around the affordances of the innovation for cost savings and outcome improvement. For examples of these institutions see the Transparency by Design initiative http://www.wcet.info/2.0/index.php?q=TransparencyByDesign
In the institutional context, perhaps OER is less about the value of publishing openly (which enables reuse), and more about the value of teaching and learning openly – a situation where OER is an element, but not fully sufficient on its own to create significant value.
re: (though the recent CMU OLI paper in JIME is obviously headed in the right direction)
Will you take a minute and clarify why you think this study is headed in the right direction? It is a DE to f2f comparison study with a no significant difference finding (shocking*). Why is it beneficial continue DE vs f2f comparison studies under OER (online) vs f2f?
* been there, done that too many times over:
Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004). How Does Distance Education Compare With Classroom Instruction? A Meta-Analysis of the Empirical Literature. Review of Educational Research, 74(3), 379-439. doi: 10.3102/00346543074003379
Lockee, B., Burton, J. K., & Cross, L. (1999). No comparison: Distance education finds a new use for ‘No significant difference’. Educational Technology Research and Development, 47(3), 33-42. doi: 10.1007/BF02299632
Because, as I recall, the system enabled them to complete the course with no significant difference in learning outcomes in half the time (eight weeks). Now, this has absolutely nothing to do with openness, but they were at least asking an interesting question. (Holding performance constant, can we improve the time factor?)
One thing that comes to mind is a blog post I just read earlier today about a mental trap that creative individuals can get stuck in. http://www.fantopro.com/blog/2009/12/is-your-life-so-divided-up-you-cant-succeed.html Perhaps that article will shed some light on how to communicate the value proposition to others.