tl;dr – Go play around with generativetextbooks.org and let me know what you think.
Earlier this year I began prototyping an open source tool for learning with AI in order to explore ways generative AI and OER could intersect. I’m specifically interested in trying to combine the technical power of generative AI with the participatory power of OER, in order to both increase access to educational opportunity and improve outcomes for those students who access it. I did some preliminary writing on this topic back in July of 2023, calling the artifacts that result from combining generative AI and OER “generative textbooks” and have continued to ruminate on the topic.
I wanted the tool to exploit the many-to-many relationship between topics and study techniques. That is, I wanted to leverage the fact that you can study one topic using many different study techniques, and you can also use one study technique to study many different topics. For example, you can study chapter 1 using both flash cards and practice quizzes, and you can use flash cards to study both chapter 1 and chapter 2. Both the topics to be learned, and the activities learners engage in to learn them, can be mixed and matched (to some extent).
From a participatory / democratizing perspective, it was important to me that anyone who could author an open textbook could also author a generative textbook. The tool needed to provide a no-code, type-words-in-a-box experience like Pressbooks for authors. But what exactly would they author?
- groups of short, specific statements of what learners should learn (i.e., learning objectives),
- summaries of the topics learners should learn (these are not intended for students to read and learn from, they’re intended to provide extra context to the model to improve its accuracy), and
- activities students can do in order to learn.
At some point my friend (and very talented software engineer) Josh Maddy got involved. What we ended up creating might be called an educational prompt template management system. In addition to the learning objectives, topic summaries, and activities we added a “book-level” prompt stub that can be used to establish tone, personality, voice, response format (e.g., Markdown), etc. across the entire generative textbook. Consequently, if you were going to create a generative textbook with ten “chapters,” you would create:
- one book-level prompt stub,
- ten groups of learning outcomes,
- ten aligned summaries of chapter topics, and
- some number of learning activities.
To study with the system, a learner selects a generative textbook, then selects a topic to study, and then selects a way to study. The information associated with their selections is then aggregated into the prompt template format and the completed prompt is passed to an LLM to kick off the learning activity.
Because we’re committed to openness, we open sourced the tool itself, used open weights models, and added support for attribution and license information. The first version of the prototype sent the prompt to an open weights model hosted on Groq (currently my favorite host of open weights models) via their API. This design makes it easy to swap in a range of different open weights models, including ones you might be hosting locally. (I recognize, though, that setting up local models is likely beyond the capability of the majority of students I hope will benefit from a tool like this, and that creating a truly delightful “done-for-you” experience is beyond the scope of this prototype we’re making. But I think running this whole toolkit locally is a problem that could be solved with some grant funding.)
Early Feedback
Earlier this summer, while I wasn’t ready to show the prototype to people I felt like the design and development work had clarified my thinking enough that I could have meaningful conversations about the ideas underpinning the work. Consequently, I had several conversations with US-based college and university educators about AI. I suppose I shouldn’t have been surprised, but one theme emerged loud and clear from those conversations:
Instructors are significantly more interested in AI tools being free for students to use than they are interested in whether or not the tools are open.
While the prototype was all open source and used open weights models, accessing those models via the API costs money. (In the future, when we’re able to connect the tool to locally running models, we can bring this API-based approach back.) But for now we needed to change course on the prototype design. For a while it seemed like there was no way to do provide the capabilities we wanted to provide in a way that could be free for students.
Then we struck upon a solution. It would degrade the user experience somewhat, but would allow learners to use the tool for free. That solution? At the last step in the process, rather than passing the completed prompt to an open weights LLM via an API, simply copy the prompt to the user’s clipboard and forward them to the LLM of their choice. When they get there, they just type “CTRL-V” or click “Edit > Paste” and hit enter.
There are actually some benefits to this approach beyond not having to charge people per token to use the tool. First, it lets students use the very best models in the world instead of the open weights models which, though terrific, lag behind the proprietary models in terms of quality. Second, if a student’s institution has an institutional LLM that all learners have access to and have experience using, they can use that familiar tool for their work. And finally, if they don’t have a paid account (either personally paid or institutionally paid), students can work up to the free limits of one model and then easily switch over to a different model to continue their learning.
But there are downsides to this approach as well. Beyond the user experience being a little disjointed, this approach makes it difficult to capture analytics data for continuous improvement or in support of research (though I’m working on some ideas to overcome this limitation). There may also be privacy concerns if the free usage tier(s) of the model(s) learners choose to use don’t have strong privacy assurances.
Making It Public
The prototype is still just that – an unpolished experiment as opposed to a polished product. But it’s ready for you to play with now. Two notes to consider as you do:
First, it should be said – and it should be said over and over again – that a tool like an educational prompt template management system will only support learning effectively if the individual template components are well written. The objectives need to be clear, the summaries need to be comprehensive and accurate, and the activities need to be grounded in rigorous research about what actually supports student learning. (An activity prompt that adapts to a student’s “learning style” isn’t going to help anything.) “Garbage in, garbage out” was never truer than it is in the context of LLMs. This tool is, in many ways, just a place for people to easily host and manage their prompts. So think about this as primarily a technology demo – I haven’t invested a lot of time and effort in the demo content. (I’ve just borrowed some open content from Lumen and OpenStax and quickly built a couple of demo activities.) But there’s enough there that you should be able to get a sense for what might be possible if we pushed on this a little harder.
Second, I don’t think these generative textbooks are ready to be adopted as primary course materials just yet – the tool would need a lot more functionality before you could consider that. I do think, however, that it makes for extremely interesting supplemental materials, and that’s the way I’ll be using them in my teaching this semester.
So please go play around with generativetextbooks.org and let me know what you think. You can try the learner experience without logging in, but you’ll need to login with Google to play with the authoring tools. (And if you want to play around in the source code, it’s on Github.) And many thanks to Lumen Learning for supporting this work!