It seems like many people struggle to understand how the Open Badge Infrastructure works. Here’s an analogy that I’ve recently found helpful.

Say your friend buys you an Amazon or iTunes gift card for your birthday. When your friend buys the gift card, they are required to provide your email address, both so that (1) the store knows where to send the gift card and (2) the store can verify you’re you when you come to claim the gift card. After your friend completes the purchase, you receive an email containing a special code. To redeem the gift card, you go to a website, verify your identity, and enter the code. After you enter the code, a certain amount of credit appears in your account, which you can spend however you like.

Badges are “redeemed” in much the same way. Say you participate in an online course. When you complete the requirements (e.g., pass the final test), the systems prompts you for your name and email address both so that (1) the systems knows where to send the badge and (2) they can verify you’re you when you claim the badge. After you’ve completed the course requirements, you are presented with a special code (which in the case of a badge is a URL). To redeem the badge, you take this code to a Backpack site, verify your identity, and enter the code. You may even be automatically sent to the site, and only need to verify your identity in order to redeem the badge. Either way, the badge then appears in your Backpack and you can do whatever you like with it at that point.

With apologies to psychometricians who may read, let me set some vernacular context for additional thoughts (prompted originally by Dan Hickey‘s, and then Alex Halavais’, writing) regarding my own thinking on badges and assessment.

It is beyond argument that we cannot crack open a learner’s head, insert a magnifying glass, and make direct, error-free observations of what the learner “knows.” Since we can’t actually take a “direct” measure of what someone knows, we collect evidence that allows us to increase or decrease our beliefs about the likelihood that they know, or are able to do, something.

For example, I can’t fMRI your brain in order to see if you are able to multiply two three digit numbers. However, if you successfully multiply two three digit numbers I will start to believe that you know how to do it. If you do it three times in a row without making a mistake I will believe it more. If you do it 100 times in a row without error I will have a very strong belief that you “know how” to multiply these kinds of numbers.

The same, high-level “direct measurement is impossible so we settle for gathering evidence” argument applies to all sorts of knowledge and skills, from multiplying, to naming state capitals, to troubleshooting TCP/IP networks, to arranging orchestral scores, to interpreting and critiquing a new philosophical work.

Assessment, then, is about having people engage in activities that provide this kind of evidence.

This evidence can be used in a number of ways. It can lead others to believe that you are qualified for employment, or it can lead others to believe that you will succeed in graduate school. It can lead you to believe that you don’t need to study for the final exam anymore, or it can lead you to believe that you’re ready to sign up for that Udacity class. How evidence is used and who it is used by is a related – but separate – issue from the extremely thorny process of helping learners create the most valid, reliable body of evidence possible.

To me, a badge – which strictly speaking is a few lines of JSON and a PNG image – is a form of evidence. However, these two files stored on a server are clearly NOT an activity (like writing a 1000 word compare and contrast essay) that results in evidence.

If you think about it, not only is the badge not an activity, it is also not the evidence (e.g., artifact) directly created by engaging in the activity. The activity of comparing and contrasting the North and South’s motivations for engaging in the civil war does not result directly in a badge. This process results directly in an essay (for example). After a learner has engaged in the activity and created the evidence, someone judges the essay and then represents their beliefs about what the person knows – based on the evidence – by awarding or not awarding a badge.

Those of you who have poked around in the JSON know that the word “evidence” is used in exactly this way inside the badge file. The common way of thinking about this in the badge world is “Ms. Third Party, if you don’t believe the person really deserved this badge you can click through and look at the evidence yourself!” But -importantly – this is the same evidence that led a different third party to believe that the learner deserved the badge in the first place.

So that’s a lot of explanation to say that we design (1) an activity, which results in (2) evidence, which is (3) judged, and if judged sufficient is awarded a (4) badge.

You see that a badge is a proxy for evidence, which evidence itself is a proxy for what a person “actually knows or can do.” We provide second-order proxies like badges, GPAs, and ACT scores so that every future person who is interested in your ability doesn’t have to grade your essays, review your portfolios, and view the video of your
performance assessment themselves. While these second-order proxies provide lossy compression (they contain less detail and less information), they greatly increase the efficiency of decision making processes later on. Imagine trying to narrow a 300 person applicant pool without these second-order proxies (with only access to their original evidence / artifacts).

All this rambling to say that I hope that as a community we will commit to being agnostic with regard to (1) the activity, (2) the evidence, and (3) the judgment. Regardless of whether these three steps are radically modern or terribly traditional, there is no a priori reason that any arbitrary configuration of these could not result in a (4) badge. In fact, the technical approach Mozilla has taken to badges assures that this agnosticism is possible. Only social pressure could close this door Mozilla has architected open.

This is what I was trying to get at the other day when I said that badges are credentials and not assessments. To me, an assessment is the (1) the activity, (2) the evidence, and (3) the judgment. Whether the “thing” awarded out the back end of that process is a grade, a certificate, a pat on the back, or a badge, these second-order proxies are credentials and not the actual assessments. Perhaps this is just a difference in terminology. I hope to find out…

I believe we need to be very careful in the way we talk about badges. Badges are not assessments. A badge is something you receive after you successfully complete an assessment. The actual assessment could take the form of generic multiple-choice questions, a performance assessment, a portfolio evaluation, a construct-aligned bundle of context-dependent items, or whatever. If the person successfully completes this assessment, then they receive the credential. Badges are not assessments; badges are credentials – badges are things we award to people who pass assessments.

Many smart people – people who actually know what they’re talking about – are being less specific than they might in the conversation about badges. If we aren’t vigilant with our vocabulary, the general public will start thinking that badges are supposed to be assessments. If this happens, they’ll start expecting something from badges that badges can never deliver. Confusion early in the adoption process – especially among decision makers – will be a killer for the entire movement.

Also, it’s desperately important that we keep badges cleanly separated from assessments. What we don’t want to do is have badges inseparably tethered to a singe type of assessment in the public mind. You can imagine comments like, “Badges? That’s only good for performance assessment. I could never award badges in my context because I use more traditional assessments.” This is exactly what we don’t want. We want badges to be as broadly used as possible – which means we need everyone to understand that badges are compatible with any and every kind of assessment.

I don’t have a horse in the assessment game – I’m not trying to guarantee that traditional assessments continue to dominate education. I’m also not trying to radically reinvent assessment by inventing new task or item types and rewriting item response theory. I just want to protect the “open” in “open badges.” The badge system should be open from both a technology perspective as well as from a pedagogy / assessment perspective. “Open” means making sure we don’t exclude anyone – whether they are traditionalists or reformers.

I very much want to see badges succeed. But if we set badges up as something they aren’t (i.e., assessments), success will be impossible. Everyone involved in this endeavor – myself included – needs to all stay on message: “badges are credentials, not assessments.”