Since the advent of ChatGPT it seems like everyone is talking about Bloom’s 2 sigma problem. The quick version is this: the average student who is taught using a combination of (1) one-on-one (or small group) tutoring and (2) mastery learning performs about two standard deviations better then the average student taught in a typical classroom setting. The “problem” in Bloom’s 2 sigma problem is that, while we know that this dramatic improvement in student learning is possible, we don’t know how to implement it at scale. We can barely afford one instructor for 30 students - there’s no way we can afford full-time individual tutors for each student. So, since Bloom and colleagues published this finding in the 1980s, many people have been working on this challenge included in their article: ...

Prelude The extraordinary woman who mentored me through graduate school and co-chaired my PhD committee, Dr. Laurie Nelson, frequently talked to me about the idea of “current best thinking.” Characterizing something as your “current best thinking” gives you permission to share where you are in your work while simultaneously making it clear that your thinking will still evolve in the future. It is critically important to remember that both open education and generative AI are tools and approaches - they’re means to an end, methods for accomplishing a goal or solving a problem. I’m interested in solving problems of access and effectiveness in education. I think open education and generative AI have a lot to offer toward solutions to these problems. But I want to, from the outset, caution all of us (myself included) against becoming enamored with either open education or generative AI in and of themselves. As they say, you should fall in love with your problem_, not your_ solution_._ ...

This is the video recording of my recent talk, “Why Open Education Will Become Generative AI Education.” I previously published some of the content in written form as How Generative AI Affects Open Educational Resources and Why Generative AI Is More Effective at Increasing Access to Educational Opportunity than OER. https://youtu.be/WpcE7ihlUDo?feature=shared&t=224

This is the opening section of my September 19, 2024 presentation, Why Open Education Will Become Generative AI Education. I’m pre-posting some of the presentation content due to the very active conversation the announcement of the presentation has created. Last week I posted the middle section of the presentation, How Generative AI Affects Open Educational Resources, in which I described how we need to move beyond narrow thinking about how generative AI impacts our work with traditional OER and begin thinking more broadly about the power generative OER. ...

This is the middle section of my September 19, 2024 presentation, Why Open Education Will Become Generative AI Education. I’m pre-posting some of the presentation content due to the very active conversation the announcement of the presentation has created. Next week I hope to post the first section of the presentation, which outlines the reasons why people who care deeply about affordability, access, and improving outcomes should consider shifting their focus away from OER (as we have understood it for the last 25+ years) and toward generative AI. Or, using the language I introduce below, from “traditional OER” to “generative OER." ...

All of the analysis, design, and development that happen in conjunction with retrieval augmented generation (RAG) and fine-tuning of large language models is instructional design. The instructional design process often begins with an analysis of the learner(s). What do they already know? What skills do they already have? What prior knowledge can we assume will be there, so we can design our instruction in a way that successfully builds on their understanding? In the case of LLMs, “prior knowledge” is the set of capabilities models get from their pre-training. When we begin the process of prompting, fine-tuning, or setting up retrieval augmented generation (RAG), we’re not starting from scratch. The first step is understanding the set of capabilities in your base model the same way you would want to understand the prior knowledge of your target learners. ...

A significant amount of time, effort, and resources go into training large language models (LLMs) to follow instructions. In fact, after the initial pre-training step, many models are specifically instruction-tuned in order to make them better at following instructions. If you’ve ever been poking around Huggingface and wondered why some models have “Instruct” in their name (like Llama-3-8B vs Llama-3-8B-Instruct), this is why. While a wide range of prompt engineering frameworks exist, they all have one thing in common: they help you write clear, detailed, thorough, accurate instructions for an LLM to follow. LLMs can complete simple tasks given only simple instructions (“Write a poem about a sunny day”), but in order to complete more complicated tasks they need more detailed instructions (e.g., see this 820 word ‘Updated Tutoring Prompt’ by Ethan Mollick that instructs the LLM to act as a tutor). Because many models are specifically instruction-tuned as part of their training process, clearer instructions generally result in better outputs from the model. ...

Over four years ago I described what I called the Musician’s Rule. The key insight behind the Musician’s Rule can be grasped by reflecting on two short scenarios. Imagine what would happen if: a person with no musical training is given a $1M Stradivarius violin and asked to play it. a person with a graduate degree in violin performance and decades of experience playing in recitals and concerts is given a $30 middle school orchestra rental violin and asked to play it. Which music will be the most enjoyable? Take a minute and really try to imagine what each of those mini-concerts would sound like. One hundred times out of one hundred, the music made by the person with training and experience sounds the best. ...

As recently as the early 1990s getting your thoughts and ideas out into the world at any kind of scale required finding a publisher that would agree to publish your work. Publishers served as gatekeepers, deciding what was worth publishing and what wasn’t, typically publishing only material they thought would be commercially successful. And of course publishers typically required authors to sign over the copyright in their works to the publishers. Finding a publisher who would publish your work was incredibly rare - like being struck by lightning - and the process of publishing anything was expensive and complicated. Publishing was so difficult, in fact, that the overwhelming majority of people never even *imagined* doing it themselves. ...

Large language models (LLMs) make it possible for faculty to rapidly create a wide range of formative and summative assessments for their students. And, as we hear about so often, students can also use LLMs to write their essays and complete other assignments. (Apparently, when faculty use AI to create assignments, it’s a “productivity gain.” But when students use AI to complete assignments, it’s “cheating.” But that’s a topic for another day.) Reflecting on several conversations I’ve been part of at the SUNY CIT conference this week led me to realize an important principle about the symmetry of AI in assessment. LLMs are equally powerful tools for both faculty and students. Speaking solely about what is technologically possible (and not what is ethically appropriate), we might summarize this principle by saying: ...