Open-Source

For those interested in issues around agentic AI and assessment, I’m excited to announce the launch of the CHEAT Benchmark (https://cheatbenchmark.org/). The CHEAT Benchmark is an AI benchmark like SWE-Bench Pro or GPQA Diamond, except this benchmark measures an agentic AI’s willingness to help students cheat. By measuring and publicizing the degree of dishonesty of various models, the goal of this work is to encourage model providers to create safer, better aligned models with stronger guardrails in support of academic integrity. ...

tl;dr - Go play around with generativetextbooks.org and let me know what you think. Earlier this year I began prototyping an open source tool for learning with AI in order to explore ways generative AI and OER could intersect. I’m specifically interested in trying to combine the technical power of generative AI with the participatory power of OER, in order to both increase access to educational opportunity and improve outcomes for those students who access it. I did some preliminary writing on this topic back in July of 2023, calling the artifacts that result from combining generative AI and OER “generative textbooks” and have continued to ruminate on the topic. ...

This is the middle section of my September 19, 2024 presentation, Why Open Education Will Become Generative AI Education. I’m pre-posting some of the presentation content due to the very active conversation the announcement of the presentation has created. Next week I hope to post the first section of the presentation, which outlines the reasons why people who care deeply about affordability, access, and improving outcomes should consider shifting their focus away from OER (as we have understood it for the last 25+ years) and toward generative AI. Or, using the language I introduce below, from “traditional OER” to “generative OER." ...

The Open Source Initiative, which has been the steward of the Open Source Definition since 1998, is leading a multi-stakeholder initiative to define Open Source AI. The definition is currently in its eighth draft, with the goal of finalizing the definition by October, 2024. There are only a few families of models that would qualify under the current draft definition: Mixtral - Weights via Huggingface: https://huggingface.co/mistralai (Apache License) - Technical paper: https://arxiv.org/abs/2401.04088 (CC BY) ...