Of Progress, Problems, and Partnerships

In 2012 Kim Thanos and I founded Lumen Learning because, through our Gates-funded work on the Kaleidoscope Project, we had seen first-hand how hard it was for faculty to replace publisher materials with OER. The 2000s were an inspiring decade as institutions and individuals created and published a huge amount of openly licensed educational materials (e.g., MIT OCW, Wikipedia, Khan Academy), but in 2010 it was difficult to find a faculty member who had made the switch. It seemed like lots of people wanted to publish and share their own OER, but no one wanted to use anyone else’s.

Actually, plenty of faculty wanted to use OER – they just didn’t want to use them badly enough to fight through all the obstacles that involved.

The difficulties faculty had – and still have – adopting OER is a huge problem for several reasons. First, when faculty don’t adopt OER there’s no opportunity for OER to save students money. Second, when faculty don’t adopt OER there’s no opportunity for OER to facilitate new forms of pedagogy that invigorate both teaching and learning. And third, when faculty don’t adopt OER there’s no opportunity for OER to support significant improvements in student learning.

These are the problems Lumen is still chasing today. How can we help as many faculty as possible adopt OER, leading directly to significant student savings? How can we help those faculty who have made the choice to adopt OER wrap their heads around the pedagogical benefits enabled by OER? How can we help faculty and students use OER in ways that will result in improvements in measures like final grade, completion rate, persistence rate, and graduation rate?

It’s been a humbling and amazing experience. We’ve learned some things about working effectively with faculty through the course redesign and OER selection process. About how to carry forward the best work of previous faculty to a new group of faculty so they can stand on the shoulders of those who came before and go much faster. About hosting, managing, and integrating OER into LMSs where faculty and students already are. About license vetting and CC attribution management. About using data to support the continuous improvement of both content and assessments (watch for more on this topic in weeks to come). About the intersection of instructional design, personalization, learner agency, metacognition, and behavioral economics (nudging). About building and running entire degree programs on nothing but OER.

As we’ve learned more we’ve been able to help more. We more than tripled the number of students we supported from 2014 (~10,000 students) to 2015 (~35,000 students), and more than tripled again from 2015 to 2016 (~110,000 students). Across those three years these students saved almost $15 million. Peer-reviewed research has shown, for a range of outcomes, that when we support students using OER they perform the same or better than their peers using publisher materials. (And the most exciting research, looking in much more depth at our Gates-funded work on personalization and OER, is still to come.) And faculty are slowly coming to appreciate the range of novel things (like renewable assignments) that OER adoption enables in their pedagogy. There’s a mountain of work left to do, but it feels like we’re off to a solid start.

But it’s really just a start. While Kim is magically staying the same age, I’m getting older – and neither of us is really interested in running a cool little project that improves things for a small fraction of postsecondary students and faculty. Neither are the other amazing people who’ve found their way to Lumen over the years. (And did I mention we currently have five positions open?) To only lightly edit the language from my original application for the Shuttleworth Fellowship back in 2012:

We want to push the field over the tipping point and create a world where OER are used pervasively throughout schools, colleges, and universities. In our vision of the world, OER supplant traditional textbooks for all courses at all levels. Organizations, faculty, and students at all levels collaborate to create and improve an openly licensed content infrastructure that dramatically reduces the cost of education, increases student success, and supports rapid experimentation and innovation.

I’m keenly aware that this kind of system-level change can’t be facilitated by a single person or a single organization. This scale of change requires collaborative efforts on the part of many, many people and organizations. So what kind of collaboration could move the field meaningfully closer to this vision of a world where OER are the default?

First, what do we know about why OER aren’t already the default? According to the most recent Babson survey of faculty about OER, the biggest obstacles to OER adoption in higher ed are:

  • the majority of faculty don’t know that OER exist, and
  • for the minority who do know about them, OER are too hard to find.

So how do you address that problem at a scale large enough to make a difference?

Today Lumen announced a major new partnership with Follett. Follett operates more than 1,200 local campus bookstores, and they’ve made significant investments in tools, processes, and people to make it easy for faculty to review and adopt course materials. The new partnership will integrate Lumen’s open courseware offerings into Follett’s systems, making them super easy for faculty at about 1/3 of all US higher education institutions to find, review, and adopt. The partnership also adds, for the first time, the option for students to pay Lumen’s course support fee rather than the institution. (Previously our model only allowed institutions to pay these fees, and that has made it difficult for some schools to work with us.) This new option makes it possible for individual faculty to choose to work with Lumen, which hasn’t been possible due to the institutional focus of our model. Now there’s an easy way for everyone to work with Lumen – from an entire institution to a single faculty member. The press release has all the details.

I’m incredibly excited about this partnership because I think it will go a long way toward overcoming the biggest obstacles to OER adoption Babson identified. It should result in millions more students and faculty using OER and enjoying the wide range of benefits they provide. And it will provide a terrifically interesting case study of a new OER adoption model that we haven’t seen or studied before, hopefully inspiring more creative thinking in this space.

“We’ve only just begun,” as the song says. We won’t realize our entire vision by taking this one step, but this one step certainly moves us forward.

Personalization in Lumen’s “Next Gen” OER Courseware Pilot

For almost three years Lumen Learning has been helping faculty, departments, and entire degree programs adopt OER in place of expensive commercial textbooks. In addition to saving students enormous amounts of money we’ve helped improve the effectiveness of courses we’ve supported, as we’re demonstrating in publications in peer-reviewed journals co-authored both with faculty from our partner schools and other researchers. We’re making great friendships along the way. It’s been absolutely amazing.

Last year we received one of seven grants from a Bill and Melinda Gates Foundation competition to create next generation personalized courseware. We’ve spent the last year working with something like 80 faculty from a dozen colleges across the country co-designing and co-creating three new sets of “courseware” – cohesive, coherent collections of tools and OER (including some great new simulations, whose creation was led by Clark Aldrich, and newly CC licensed video from the BBC) that can completely replace traditional textbooks and other commercial digital products.

As part of this work we’ve been pushing very hard on what “personalized” means, and working with faculty and students to find the most humane, ethical, productive, and effective way to implement “personalization.” A typical high-level approach to personalization might include:

  • building up an internal model of what a student knows and can do,
  • algorithmically interrogating that model, and
  • providing the learner with a unique set of learning experiences based on the system’s analysis of the student model

Our thinking about personalization started here. But as we spoke to faculty and students, and pondered what we heard from them and what we have read in the literature, we began to see several problems with this approach. One in particular stood out:

There is no active role for the learner in this “personalized” experience. These systems reduce all the richness and complexity of deciding what a learner should be doing to – sometimes literally – a “Next” button. As these systems painstakingly work to learn how each student learns, the individual students lose out on the opportunity to learn this for themselves. Continued use of a system like this seems likely to create dependency in learners, as they stop stretching their metacognitive muscles and defer all decisions about what, when, and how long to study to The Machine. This might be good for creating vendor lock-in, but is probably terrible for facilitating lifelong learning. We felt like there had to be a better way. For the last year we’ve been working closely with faculty and students to develop an approach that – if you’ll pardon the play on words – puts the person back in personalization. Or, more correctly, the people.

It’s About People

Our approach still involves building up a model of what the student knows, but rather than presenting that model to a system to make decisions on the learner’s behalf, we present a view of the model directly to students and ask them to reflect on where they are and make decisions for themselves using that information. As part of our assessment strategy, which includes a good mix of human graded and machine-graded assessments, students are asked to rate their level of confidence in each of their answers on machine-graded formative and summative assessments.


This confidence information is aggregated and provided to the learner as an explicit, externalized view of their own model of their learning. The system’s model is updated with a combination of confidence level,  right / wrong, and time-to-answer information. Allowing students to compare the system model of where they are to their own internal model of where they are creates a powerful opportunity for reflection and introspection. 

We believe very strongly in this “machine provides recommendations, people make decisions” paradigm. Chances are you do, too. Have you ever used the “I’m Feeling Lucky” button on the Google homepage?



If you haven’t, here’s how it works. You type in your search query, push the I’m Feeling Lucky button, and – instead of showing you any search results – Google sends you directly to the page it thinks best fulfills your search. Super efficient, right? It cuts down on all the extra time of digging through search results, it compensates for your lack of digital literacy and skill at web searching, etc. I mean, this is Google’s search algorithm we’re talking about, created by an army of PhDs. Of course you’ll trust it to know what you’re looking for better than you trust yourself to find it.

Except you don’t. Very few people do –  fewer than 1% of Google searches use the button. And that’s terrific. We want people developing the range of digital literacies needed to search the web critically and intelligently. We suspect – and will be validating this soon – that the decisions learners make early on based on their inspection of these model data will be “suboptimal.” However, with the right support and coaching they will get better and better at monitoring and directing their own learning, until the person to whom it matters most can effectively personalize things for themselves.  

Speaking of support and coaching, we also provide a view of the student model to faculty and provide them with custom tools (and even a range of editable message templates written from varying personalities) for reaching out to students in order to engage them in good old-fashioned conversations about why they’re struggling with the course. We’ve taken this approach specifically because we believe that the future of education should have much more instructor – student interaction than the typical education experience today does, not far less. Students and faculty should be engaged in more relationships of care, encouragement, and inspiration in the future, and not relegated to taking direction from a passionless algorithm.

A Milestone

This week marks a significant milestone for Lumen Learning, as the first groups of students began using the pilot versions of this courseware on Monday. Thousands more will use it for fall semester as classes start around the country. This term we’ll learn more about what’s working and not working by talking to students, talking to faculty, and digging into the data. We’ll have an even more humane, ethical, productive, and effective version of the courseware when we come out of the pilot in Spring term. And an even better version for next Fall. (We’re really big on continuous improvement.)

This stuff is so fun. There’s nothing quite like working with and learning from dozens of smart people with a wide variety of on the ground, in the trenches experience on the teaching and learning side, and being able to bring the results of educational research and the capabilities of technology into that partnership. You never end up making exactly what you planned, but you always end up making something better.

Clarifying the 5th R

There have been a number of responses to my decision to introduce a 5th R – “Retain” – to my 4Rs framework. Bill, Darren, and Mike have responded, among others. Some parts of the responses lead me to believe that I wasn’t entirely clear in my initial statement, so let me try to clear a few things up.

The original 4Rs were not an attempt to create a new group of permissions that open content licenses needed to support. Many open content licenses, from the CC to the GFDL to the OPL, already granted the rights to reuse, revise, remix, and redistribute long before I created the 4Rs framework. I created the 4Rs framework specifically for the purpose of helping people understand and remember the key rights that open content licenses grant them.

The right to Retain – i.e., make, own, and control – copies of openly licensed content has always been a right granted by open content licenses. Generally speaking, it is impossible to revise, remix, or redistribute an openly licensed work unless you possess a copy of the work. As Mike pointed out, the right to retain is strongly implied in open licenses, but never called out directly. Consequently, it has never been addressed directly in the discourse around open.

I maintain my original purpose for creating the 4Rs framework in adding “Retain” and arriving at 5Rs. The purpose of the framework is to help people understand and remember the key rights that open content licenses grant them. It was becoming increasingly clear to me that both producers and users of open content were unaware of, or forgetting to consider, this critical right to Retain.

As I said above, at least 3 of the original 4Rs are impossible to do without the right to Retain. This makes Retain a fundamental or foundational right, and yet it is completely ignored in the discourse around open. This is why I felt the need to call attention to Retain, and this is why I now place it at the head of the list of 5Rs – retain, reuse, revise, remix, and redistribute.

Thinking along these lines – about how the systems we design can proactively enable people to exercise their right to Retain – has already proven extremely useful to me personally. (As I wrote about recently, we’re in the middle of designing a new OER transclusion system at Lumen.) If the right to Retain is a fundamental right, we should be building systems that specifically enable it. When you specifically add “Enable users to easily download copies of the OER in our system” to your feature list, you make different kinds of design choices. (Unfortunately, you can see all around you that many of the designers of OER systems either failed to think about how to enable users to exercise their right to Retain, or have purposively taken specific actions to prevent users from exercising it.)

Postscript. The name of my blog is “Iterating Toward Openness.” This name is meant to very publicly demonstrate my shortcomings as a thinker about and practitioner of “open.” I didn’t understand everything I needed to about open when I kicked off the open content work in 1998, and I still don’t understand enough about it today. My goal is to be constantly (if incrementally) refining and improving my understanding and appreciation of open. The change from 4Rs to 5Rs reflects one such improvement in my understanding. Who knows – maybe another seven years from now I’ll add a 6th R.