“Or Equivalent”

My colleague and friend Gideon Burton (of silva rhetoricae fame among other things) and I have been discussing badges lately. To date the open education movement has focused almost exclusively on the production and sharing of content. Significant opportunities exist to reform or reinvent other, non-content portions of the education ecosystem with the support of open content.

One of the areas ripest for innovation is alternative certification of informal learning. Hence, the recent excitement about badges. Badges have incredible potential for providing a viable alternative to the traditional system of credits most universities are tied to by accreditors. It seems to me that there is a critical need for someone to demonstrate that badges are a viable alternative to the traditional accreditation process.

There will doubtless be thousands of badges dedicated to pseudo-academic, hobby-like learning (stargazing, pie making, amateur radio). However, because the gold standard for learning credentials is acceptability by employers, any meaningful badges demonstration project will have to operate in this space. And of course, open content has an important role to play in supporting the widespread adoption of badges as officially accepted credentials.

We are considering a badge demonstration project comprised of three stages. The high-level vision of the project is this: Many job descriptions include a requirement like “BA or BS in EE/CS/CE or equivalent experience.” We want to create a collection of badges that a top employer, like Google, will publicly recognize as “equivalent experience.” This goes straight for the jugular, demonstrating that badges are a viable alternative to formal university education. The timeline below uses Google as an example, but we would be happy to work with any high-profile employer. We haven’t yet reached out to Google or any other employer. Let me repeat, WE’RE NOT CURRENTLY WORKING WITH GOOGLE, THEY’RE JUST INCLUDED BELOW AS AN EXAMPLE.

I’m posting what we’re thinking to get your overall feedback and see if you can suggest any big name employers who might be willing to partner with us.

Stage One – Planning Stage

– Work with Google HR and other Google employees to identify a core set of competencies that would qualify a person to work at Google (e.g., many network engineering positions include the “or equivalent” language).

– Create one or more badges corresponding to each of the competencies identified through conversations with Google.

– Establish and fund an advisory board of recognized experts in the selected area and forward-thinking psychometricians to help create and validate performance assessments and grading rubrics aligned with each badge.

– Review the badges, competencies, performance assessments, and grading rubrics with the team at Google. Secure a commitment from Google to hold this collection of badges as “equivalent” to a BS for purposes of hiring.

– Go / no go decision for moving on to stage two.

Stage Two – Pilot Period

– Hire and train a pool of qualified graders who are capable of quickly and accurately marking the performance assessments. Train these individuals (and refine rubrics as necessary) until achieving acceptable levels of inter-rater reliability in grading of the performance assessments is achieved.

– Stand up the necessary technical infrastructure for awarding badges to successful applicants (in partnership with Mozilla?).

– Launch a website with:
– The official statement from Google regarding their willingness to review applicants submitting these badges as credentials
– The complete list of badges, related competencies, performance assessments, and grading rubrics (all openly licensed)
– Names and affiliations of advisers and partners
– A clear / simple process for submitting performance assessments
– An initial list of OER (e.g., OLI courses) and Q/A services (e.g., StackOverflow.com or OpenStudy) which will help individuals develop the skills necessary to obtain the badges

– Provide a mechanism (wiki?) for allowing users to contribute links to new OER and online communities aligned to specific badges
– Scholarship the first X individuals who apply for badges (very minimal nuisance fee (e.g., $5) to the individual to have their assessment graded)

– Evaluate the success of stage two (e.g., number of applicants, success rates in achieving badges, sanity check costs for providing the assessment service, evaluate the success of applicants in Google’s application process).

– Go / no go decision for moving to stage three.

Stage Three – Implementation

– Establish a sustainable financial model for charging as-small-as-possible fees for marking assessments and awarding badges. Begin exploring crowd-sourced, non-game-able models for marking assessments in order to bring costs down further.

– Expand pool of partner employers (e.g., Microsoft, Apple) and explore the option of having employees of partners mark the assessments. This insures quality on their side and eliminates cost on ours.

– Establish advertising partnerships with colleges offering relevant online courses for students who need extra help earning badges (perhaps an adapted WGU model?) to support core infrastructure over the long term

– Combine these and other business models to generate enough revenue so that (1) the marking service can be free in addition to all the badge related materials being openly licensed and (2) employers will respect and recognize the badges resulting from the process.

The bolded items above really represent one version (and certainly not the only one) of the complete package – open content, open learning support, and open badges that help you demonstrate competence to an employer.

Anyway, thoughts? Feedback? Ideas about who would want to partner?

Comments on this entry are closed.

  • What I don’t understand is why badges are seen to be so sexy. Why do you choose a ‘badge’ to validate learning? Do we all have to turn into boy scouts now and try to get a shallow reward, developed by software developers because it fits with their world view of computer games and Mozilla extensions?

  • Rob

    What I don’t understand is why degrees are seen to be so sexy. Why do people choose an embossed “piece of paper” to validate learning? Do we all have to turn into academically-branded human resources and try to get a shallow reward, developed by elitist academics because it fits their world view of people as widgets on a conveyor belt who pay to be branded for spending time with them?

    Seriously, though – who cares if they are called “badges” or some other word for open credentials. The point is that there isn’t currently a very good way to show you know what you’re doing other than to have a branded piece of paper that claims you do.

  • David, I didn’t mean to be unpleasant in my Twitter responses to you. I’m still grateful to you for agreeing to be a WatchKnow (now WatchKnowLearn, and now in the able hands of Dr. Joe Thomas) advisory committee member. I didn’t mean my remarks personally or even especially confrontationally–I was just giving you my honest reaction, in response to your call for comment. I care about this because I’ve written about something similar, and as someone who has started an ambitious project that got away from him, I see significant potential for that happening here. I also care because I really think that something like this may well lie in our future. So let me develop my points more fully.

    1. About the word “badge.” As the above discussion makes plain, this talk of “badges” marks this whole endeavor as one started by boys, or former Boy Scouts. On a marketing point, I would worry about losing some traction with the female majority of the college-going public. Speaking for myself, I don’t like the talk of “badges” because this implies that hard-won credentials are merely bragging rights, or a mark of authority, of the sort Boy Scout badges or police badges are. I have a Ph.D. but I don’t put my diploma on display like a “badge”; that’s bad taste. Also, you aren’t reporting much thinking about how this whole project will be received by academia; academics are not apt to find “badges” very compelling.

    2. So, let’s talk about the whole idea of academia confronting this endeavor. Far be it from me to speak for academe (you’re the paper-publishing professor, not me), I think this deserves some consideration.

    Let’s begin here. You say, “the gold standard for learning credentials is acceptability by employers.” I’m not sure what this means, exactly. Is this a statement about what degree-seeking students (the customers of universities) believe, what “society in general” believes, or what is really, in fact, the highest conceivable standard in your own personal opinion?

    When it comes to evaluating someone’s B.A. in Philosophy, do you want to say that “the gold standard” is “acceptability by employers”? Would that mean that I am a better philosopher if my degree has a better chance of getting me hired? Speaking as a formerly under-employed philosophy major, that sounds utterly ridiculous. A degree is, objectively speaking, supposed to indicate some actual level of intellectual attainment in the field. Surely you don’t mean to say that, if an employer hires me for such-and-such a degree, that indicates that I have reached that level of attainment in the field? Of course it doesn’t mean that.

    3. Speaking as someone who hires people from time to time, to help with my projects, what I’m looking for depends entirely on the job. When I was looking for a voiceover person, the absolute only thing I cared about was the quality of her performance. But when I was hiring editors for an educational website, I was looking for the ability to write, as well as do or understand other things that a college education trains to do or understand. In that case I required a college degree and in fact was strongly preferring an M.A. in the relevant subject area.

    Suppose I were hiring ten years from now and, lo, a dozen candidates lacked an M.A. but had “badges” that, the candidates (or some organization) claimed, was “equivalent” to an M.A. So how do I evaluate this claim of equivalence? My mind might have already been made up (I know that various companies are hiring such grads with no more complaints, so far, than they have about their recent college grads). But if it is up to me, I am going to look at the process whereby the “degree equivalent” is granted.

    If a “badge” is the sort of thing that by common practice almost anybody can define, and then claim, then I’m not likely to take it seriously, and most others won’t either. In other words, the badge is a credential and a credential has to have, well, credibility. If supposed credentials are granted as easily as diploma mill “degrees,” the whole endeavor will–obviously, I think–not get off the ground. Some geeks might go about claiming to have all sorts of “badges,” but when it comes to hiring, I will ignore such self-claimed badges.

    Your blog makes it very clear that you don’t propose a system in which badges are self-claimed. You want badges awarded by some sort of objective body. That is, of course, as it should be. (By the way, why not Excelsior College? They’ve been doing this, although they award things that they call “degrees” instead of “badges.”) But what do you imagine this objective body would look like?

    The impetus behind your proposal, and the Mozilla white paper (which I read a version of a while ago), does not seem so much to be to make credentialing cheaper and more lightweight, as it is getting it away from academe altogether. Well, why? Come on now–do you really think that a Google employee is going to be able to evaluate a “badge” portfiolio better than a professor of computer science, who has long experience doing exactly this sort of thing?

    4. You mention that badge evaluation might be, somehow, “crowdsourced.” This is a startling claim. It is one thing to crowdsource an encyclopedia article or software. We know why those work, at least as well as they do work. Why on earth think that evaluating credentials is something that could be accomplished Wikipedia-style? Perhaps (I doubt this, but just suppose) you are proposing that people vote on whether a person has made the grade. Well, I obviously don’t know, but I seriously doubt you’ll get many volunteers to do the hard work of portfolio evaluation–unless identified personalities are involved and the evaluation is not done in “blind” fashion. I mean, if people can get “badges” by getting a thumbs up from some benchmark number of people in a community of practice–boy, count me out. You mention gameability; that very suggestion positively screams gameability, precisely because personalities are involved. If the crowdsourcing is suitably double-blind, I doubt you’ll get many volunteers. In my experience, people volunteer for the fun stuff. They don’t volunteer for the real gruntwork; you have to pay people for that.

    I’m guessing that you think that grading should be kept as independent of personalities as possible, and moreover, you agree with me that employers and graduate schools etc. are going to care, a lot, about the quality of the “badge” evaluation program. Well, it seems pretty obvious to me that this is going to end up calling in the professionals–the experts, in whatever field. After all, if I’m looking at two candidates, one of which was credentialed by some outfit I’ve never heard of, and the other of which was credentialed by Harvard, well, I’m going to respect the “badges” of the latter quite a bit more, won’t I? Surely the “badges” as recognized by more prestigious institutions will be worth more. And even if, in the move to bring this sort of system into the online world, there is some shuffling of players (say, Harvard makes the switch to an online badging system, but Yale is late to the game and fails), that won’t stop there from being competitive credentialing.

    I think a lot of the starry-eyed dreaminess about this whole venture stems from the fact that it would seem to take credentialing out of the hands of elites and put into the hands of the masses. Well, I really don’t think that’s going to happen. Why should it? Indeed, what about your project, insofar as it is plausible, puts credentialing in the hands of the masses?

    Well, the credentialing isn’t put into the hands of the masses. It is put into the hands of people who are willing to take money from people to evaluate exams, portfolios, etc., in exchange for small and large credentials. The only thing that makes this more “open” is the fact that one does not have to enroll as a student at an institution in order to get the credential. And that is something I’ve supported, wholeheartedly, at least since 1995.

  • Peter

    Interesting post David. I must admit I am somewhat uncomfortable with the increasing commodification of education, but this process has been going on for some time now in higher education. Receiving credit is crucial for people to gain employment and with the wealth of OER available, I think providing a low-cost alternative such as this is a necessary endeavor.

    I am currently investigating how badges might work within a blended model of delivery systems (e.g. community radio, mobile, online) so that perhaps basic qualifications in maths or bookkeeping could be offered and stored on an individual’s mobile or a biometric card. I spent several years in the Indian microfinance sector and given the high amount of migrants in the informal labor sector, I hope that having this type of ‘mobile CV’ might assist them in finding jobs.
    If anyone else is interested in this area, please do get in touch.

  • Hi, David. Long time no see.
    I like your bold proposal. There is, indeed, a need. Your initial plan seems strong. (As we all know it is likely to morph once underway, but it seems like a good roadmap and it conveys your proposed path well.)

    As far as connecting with other strong corporate partners, I suggest you talk to Emily Dirocco, with the National Alliance of Manufacturers. (You may remember her as one of the panelists during the MacArthur competition announcement.) NAM has been thinking in this direction for years and I suspect that she could direct you to some enthusiastic partners. I’ll send you her contact info in an email, rather than posting it here.

    Best wishes as you move ahead with this idea.

  • Rita, why are badges necessarily “shallow”? A PhD could be seen as a badge – there is usually no grade that comes with it, but it’s something that you have to work rigorously for years to attain. Of course, you could also receive smaller, more granular badges. Time will tell if badges really will make a huge difference, but I think they have the potential.

    David, a very well thought-out plan, and I’d definitively like to see it put into place. It addresses two things I’ve been thinking about, first that badges might be more applicable to work places that look for specific competencies (web developer etc), than to work places that are looking for something more vague – “a BA in any subject”… And second the idea of working with some large companies, especially ones that value creative self-motivated employees, and have a public statement that they are open to more non-traditional forms of learning.

    The challenge I can see, coming from a background in higher ed theory, where we read about credentialism etc, is that for many companies, a BA is not really about what you learnt (as proven by the many ads that require you to have a BA in anything), it’s more about being able to get out of bed every morning, fill out all the forms, follow all the instructions, etc. And of course the university you went to might be important – not so much what you learnt, again, but perhaps who you met, or just the fact that they admitted you (in a way companies are outsourcing selection to universities — if you’re good enough for Harvard, or Stanford, you’re good enough for us — or at least good enough to go to the next round)…

    I think this will become difficult, when you ask companies to outline what they are looking for in a BA – many might not have thought about it too much. (Again, Google, being a pragmatic technology company, might be easier to work with here than many others).

    But a very exciting process, and something absolutely worth attempting!

  • You should look at what http://stackoverflow.com/ and companion sites are doing with badges. There is no grader …. As you participate in the community you are upvoted. with these votes you are given power to moderate the site etc etc. Because these merits really do reflect what you know and how you interact with a community employers are looking for you. The site has integrated this and has job postings. Really a great way to find and show you know what you are doing.