Bloom is Brilliant

Rereading Ben Bloom’s 1968 “Learning for Mastery” I am once again impressed and humbled by his ability to cut straight to the absolute core of the issue. I don’t know that I’ve ever heard these points made more clearly or more persuasively. Absolutely inspiring.

If students are normally distributed with respect to aptitude for some subject (mathematics, science, literature, history, etc.) and all the students are provided with the same instruction (same in terms of amount of instruction, quality of instruction, and time available for learning), the end result will be a normal distribution on an appropriate measure of achievement. Furthermore the relationship between aptitude and achievement will be relatively high…

Conversely, if the students are normally distributed with respect to aptitude, but the kind and quality of instruction and the amount of time available for learning are made appropriate to the characteristics and needs of each student, the majority of students may be expected to achieve mastery in the subject. And, the relationship between aptitude and achievement should approach zero…

Carroll’s (1963) view [is] that aptitude is the amount of time required by the learner to attain mastery of a learning task. Implicit in this formulation is the assumption that, given enough time, all students can conceivably attain mastery of a learning task. If Carroll is right, then learning mastery is theoretically available to all, if we can find the means for helping each student.

 

{ 2 comments }

Adopting OER is Better for Everyone Involved

I’m continuing to learn an incredible amount as I work with Lumen Learning, supporting institutions as they go through the process of replacing traditional textbooks with Open Educational Resources (OER), and as I simultaneously continue my work with the Open Education Group conducting empirical research on the effects of OER adoption by faculty. While I’m learning many things down “in the weeds” of implementation, at a higher level I’m understanding more deeply and appreciating more thoroughly how the adoption of OER in place of traditionally copyrighted educational materials is literally better for everyone involved. Adopting OER in place of traditional textbooks truly is:

  • Better for Students
  • Better for Faculty
  • Better for Institutions
  • Better for Society

Here’s a sample of what I’m learning together with these marvelous teams of people.

Better for Students

There are a number of ways in which OER can be – and are – better for students. If we consider two very coarse-grained measures – learning outcomes and cost – there are only nine theoretically possible outcomes of OER adoption. Those outcomes are comprised of the matrix of better, similar, or poorer learning outcomes against higher, similar, or lower costs. And because the cost of OER is always substantially lower than the cost of traditional textbooks, there are only three practically possible outcomes:

  • Lower costs and poorer learning outcomes
  • Lower costs and similar learning outcomes
  • Lower costs and better learning outcomes

Either of the scenarios in which lower costs are paired with similar or better outcomes is a scenario in which OER adoption is better for students. From a purely theoretically perspective, even if the student outcomes associated with OER adoption are simply randomly distributed across adoptions, OER will be better for students twice as often as not. And of the dozen or so empirical studies on the impact of OER on student outcomes which the Open Education Group’s John Hilton presented at #OpenEd14 (a paper based on the presentation is currently under review), every single study showed one of these two results – either lower costs with similar learning outcomes or lower costs with better learning outcomes. So far, the empirical evidence is unanimous in demonstrating that OER are better for students.

When we delve into the nuance of the coarse-grained construct “learning outcomes,” there are a number of specific metrics which the adoption of OER can be reasonably hypothesized to impact. After a moment’s reflection it becomes clear that none of these are rocket science – they are all imminently straightforward effects that we would fully expect to come with OER adoption, and a growing body of evidence indicates that they do indeed come with OER adoption.

Drop / Withdraw Rate. In an almost universally cruel twist of fate, financial aid checks frequently arrive after the Add/Drop deadline. If a student drops a course before this deadline, nothing appears on his transcript and his tuition (or a substantial portion of it) is refunded. For the millions of students who rely on financial aid dollars to purchase course textbooks, this timing presents a conundrum. They’re three weeks into the course, and already a little behind because they haven’t been able to consistently do the readings and the homework. Should they cut their losses and drop the course now, while there’s still time to get a refund and prevent a W or F from appearing on their transcript? Or will they be able to catch up and still pull a decent grade once their financial aid check arrives and they can buy the required textbook(s)? Many students play it safe and drop at this point. However, if the faculty member has adopted OER, there is no getting behind dilemma related to the cost of textbooks. Every student has all the materials they need to succeed from the first day of class. And when students stay in class rather than dropping and trying again next semester, they stay on pace to graduate. Our first research paper on this topic (currently under review) studies 10,000 students across 20 courses and shows that students in sections using OER dropped at a statistically significantly lower rate than students in sections using traditional textbooks.

C or Better. When one group of students has access to all the required course readings and homework assignments and another group of students does not, the students without access to the necessary course materials will likely suffer negative academic consequences. When a faculty member adopts OER, every student in the course has free and unfettered access to all course materials from day one. When faculty adopt a typical textbook, evidence suggests that more than 22% of students frequently forego purchasing these required materials due to their cost, and another 26% occasionally do not make the purchase. When 20-30% of students in a class don’t have the required materials, this state of affairs is going to pull the grade distribution downward. In another article we currently have under review, including 15,000 students across 8 institutions, a comparison of OER sections and traditional sections shows statistically significantly more students receiving a C or better in classes where faculty adopted OER in place of traditional textbooks.

Combined Effect of Drop / Withdraw and C or Better Effects. These two effects interact in a very positive way which may not be immediately obvious. Not only are a greater proportion of students receiving a C or better when faculty adopt OER, but a larger percentage of the group that originally enrolled in the course is staying through to the end and receiving those C or better final grades. These two effects amplify one another to create a greater net benefit in terms of learning outcomes gains for students.

Enrollment Intensity. Many community colleges (the context where we do most of our work) do not have a “full-time” enrollment status for purposes of tuition, where after a student pays for 12 credits any additional credits are free. (As opposed to many universities, where 12 and 18 credits cost the same amount, because 12 credits or more is considered “full-time.”) This means that community college students are paying additional tuition for each course they take. When a student saves the price of a textbook (or two, or three) because their faculty adopt OER, they might elect to reinvest that savings by enrolling in an additional course – helping them get a little ahead on the path to graduation. In the same paper described in the C or Better section above, we found that students whose faculty assign OER take over 2 credits more in the semester they enrolled in the OER section than students enrolled in sections using traditional textbooks. We also found that these students who were assigned OER in the fall enrolled in over 1.5 credits more in the spring semester than their counterparts who were assigned traditional textbooks in the fall did.

Cost savings. Textbooks are ridiculously, possibly immorally expensive. (Liz Weir has a great post today explaining why.) When faculty adopt OER they decrease the cost of education. They decrease the need for additional student loan debt. They decrease the amount of pressure and worry students labor under while trying to learn in the disciplines. (In a strict psychometric sense, are textbook costs a construct-irrelevant source of difficulty? Perhaps.)

Summary. When a faculty member chooses OER in place of a traditional textbook, a wide range learning benefits accrue to students while they’re saving a significant amount of money by not paying for textbooks. Clearly, OER are better for students.

Better for Faculty

Re-professionalizing teaching. Major textbook publishers have worked very hard to reduce the instructional materials curation process to a multiple choice exercise for faculty. “Teaching Introduction to Biology for Nonmajors? You just need to choose between the Pearson versions, the McGraw versions, and the Cengage versions of the book. We’ll even send you free review copies! There’s no need to worry your pretty little head about content. Leave that to us!” One might argue that the publishers provide a valuable service, saving faculty the time and trouble of reviewing content from multiple sources, selecting the best bits, and orchestrating these into a functioning whole. However, these skills – the abilities to critically review content, to choose what’s “best” for your specific students in your specific context, and to combine these in instructionally effective ways – are all skills that were once core to the teaching profession. Jason Pickavance of Salt Lake Community College argues persuasively that, knowingly or not, the publishing industry is responsible for the large scale de-professionalization of teaching by providing faculty with “easy outs” from engaging in these activities. I suspect publishers do it somewhat knowingly, because an entire generation of faculty without these critical skills become wholly dependent on (we might say addicted to) instructional materials created, reviewed, selected, and assembled by publishers. (It is true that many publishers provide a “custom book” service where faculty can mix and match chapters from different books in the publisher’s catalog, but publishers provide these services primarily for the purpose of undercutting the used book market and not as conscious effort at re-professionalizing faculty.) Despite the illusion of choice presented by a handful of titles, constant revisions of these books driven partially by “competitive analysis” assures that the table of contents and topical treatments are essentially identical across all these “different choices.”

Adopting OER is a completely different experience. While faculty may begin by selecting an open textbook, this selection does not need to be a professional and intellectual dead end. Every word, every image, every example, every definition, and every other aspect of the book is open to localization, adaptation, remixing, and improvement by faculty. And while a publisher’s library of content available for reuse in a custom book may reach into the tens of thousands of options, conservative estimates place the number of OER at over 800 million. Here faculty have a greatly expanded content base to draw from, the ability to choose their own editing tools, no copyright clearance process to navigate, and the most lightweight copyright compliance regime imaginable (e.g., meeting an attribution requirement). There’s also no DRM or other completely artificial barriers to usability placed on the content.

Then again, faculty might choose to ignore others’ compilations of OER into open textbooks altogether and build their own collection of individual OER from the ground up. And, on the other end of the spectrum, if a faculty member wants to simply adopt an open textbook and use it just like they did their previous commercial textbook, they have that option, too.

Pedagogical Freedom. Adopting OER instead of traditional textbooks significantly expands the academic freedom of faculty members in terms of pedagogy. There are a wide range of activities and assignments that can be made in the context of OER that simply cannot be made when a traditional textbook has been selected. For example, faculty can assign students to find OER that speak more directly and clearly to them about a course topic than current material, with the promise that the best finds will be incorporated into the official course materials with attribution. Students can also write their own material, or shoot their own videos, or record their own interviews, etc., with a similar guarantee. Immediately these activities change from being “disposable assignments” which students invest little time in and immediately throw away on return (like response essays), and are transformed into activities with real value that will be used and admired by their peers and win them both personal satisfaction and a small amount of fame. In short, adopting OER allows faculty to then invite students to become co-producers of knowledge rather than passive recipients. The permission to make that invitation simply does not exist when faculty adopt traditional textbooks.

Summary. OER provide faculty with the choice to maintain their pedagogical status quo (while saving students money and possibly improving outcomes), as well as the choice to pair truly customized instructional materials with innovative pedagogical experiences for students, and a range of choices in between. Because they increase faculty’s ability to be true teaching professionals while expanding their pedagogical degrees of freedom, OER are better for faculty.

Better for Institutions

Increased Tuition Revenue through Open (INTRO). I described above how changes in Drop / Withdraw rates, in which students whose faculty assign OER instead of traditional textbooks drop their courses less often, can keep these OER using students on track to graduate. This change in drop behavior also has a very real financial impact on the institution. When students stay in their classes instead of dropping, institutions hold on to tuition dollars instead of refunding them. In the same article described in the Drop / Withdraw Rate section above, we projected the financial impact of this effect on an institution at the beginning of a large-scale OER adoption program. Using the change in drop rate at the end of the OER programs’s first pilot year, the number of students in the program, historical divisions among in-state and out-of-state students, and accompanying credit hour rates for in and out-of-state students, we estimated that the institution will hold on to an additional $300,000 per year in tuition revenue once the OER adoption program moves from pilot to full implementation.

Performance-based Funding. A number of states have moved to performance-based funding models, where institutions receive additional state funding (or are able to hold on to unused funds at year’s end) based on the institutions’ ability to hitting key performance goals. As the NCSL report linked in the last sentence shows, many of these key performance indicators are students success goals. These include students hitting key credit milestones (completing 15 credit hours, completing 30 credit hours), retention rates, and degrees awarded. As argued above, OER likely have a significant role to play in helping institutions improve their ability to reach these goals. There is evidence that OER adoption by faculty keeps students in class, improves their chances of passing, and encourages them to take more courses more quickly. Each of these effects contributes directly to improving metrics that are increasingly tied to institutional funding.

Summary. There are precious few measures institutions can take to increase revenue that will also improve student outcomes and increase academic freedom. OER are better for institutions.

Better for Society

I’ve stretched a bit here to include “society” in my list. I can’t honestly say this is something we have direct evidence for yet. However, it’s easy to imagine that simultaneously decreasing student debt while increasing student success has positive effects on society. It’s easy to imagine that re-professionalizing teaching will have positive impacts on society. It’s easy to imagine that finding mechanisms to increase funding to our educational institutions will have positive impacts on society. Verifying this hypothesis will take a significantly longer amount of time and significantly more data than we’re in possession of now. However, there are clear explanatory mechanisms here that could understandably lead a reasonable person to arrive at these same hypotheses. While not yet empirically justified, the theoretical justification for these hypotheses is strong, and I’m excited to gather and analyze data that will bolster empirical arguments along these lines.

{ 4 comments }

Frances Bell has started a Google Doc collecting historical information about cMOOCs. I’m reposting my contributions to the doc (about my own cMOOCs) here on opencontent.org so I can find them again in the future if the Google Doc ever goes away.

Year: 2007
Where: USU, INST 7150, Intro to Open Education
Audience: Those interested in learning more about Open Education
Archive.org Link

Course Design:

  • Students included both formal students earning credit at USU and students from around the world participating for free
  • Students who completed the course and requested a Certificate of Completion received a certificate
  • Course syllabus was presented in a wiki which students could (and did) edit
  • Readings and videos were on the public web
  • Each student maintained a blog where their writing and assignments were posted publicly
  • A course OPML file was used to aggregate all student writing for easy reading in RSS Readers
  • The course wiki included a master list of participants, including names, institution (if any), email address, and blog address
  • Clusters of students created affinity-based sub-groups with mailing lists, etc.

Year: 2009
Where: BYU, IPT 692R, Intro to Open Education
Audience: Those interested in learning more about Open Education
Archive.org Link

Course Design:

  • Students included both formal students earning credit at BYU and students from around the world participating for free
  • Course was designed as a massively multiplayer online game
  • Students had to choose a Character Class to play during the term. Each class specialized in a different area of knowledge (IP and licensing, business models, history and philosophy, etc.) and had a separate Skills Tree (syllabus)
  • Assignments were structured as quests. The first quests could be completed by individuals, but later quests required a range of skills that required different character classes to collaborate.
  • Quests resulted in Experience Points, which translated into player Levels. Levels translated into final grades.
  • An attempt was made to encourage the creation of Guilds (sub-groups of players) that would compete against each other (e.g., on XP earned), but this failed.
  • Readings and videos were on the public web

Year: 2012
Where: BYU, IPT 692R, Intro to Openness in Education
Audience: Those interested in learning more about Open Education
Archive.org Link

Course Design:

  • Students included both formal students earning credit at BYU and students from around the world participating for free
  • Mozilla Open Badges were awarded to students who completed course challenges
  • Badges translated into grades
  • Readings and videos were on the public web
  • Each student maintained a blog where their writing and assignments were posted publicly
  • FeedWordpress was used to centrally aggregate all student writing to an Updates section of the site
{ 0 comments }