Now this model of educational programming does not have as specific data as you will have- how long on a website and how many clicks. But it does take in to consideration the specific skills exhibited by the student. It also requries the teacher to adapt their pedagogy and (perhaps) curriculum to enable the student to succeed.

However this is a difficult pedagogical process. It requires a lot of effort and skill on the part of the teacher. It also takes time. And, even with good data, it is not always possible to make the educational plan work well for the student. And may not be supportable in higher education.

There is a considerable body of knowledge in this are that you might consider looking at.

Co-opting metrics from related fields is a way to quickly bootstrap this kind of research, addressing the concerns of @Mark Smithers and @Stian Haklev about the relative paucity of useful existing metrics in education. For example, email marketers talk about email “opens”, “click throughs” and “conversions”. These three concepts apply when a marketing email message contains some kind of “call to action”. Did people look at the message, did they click through to some landing page, did they do the thing the landing page prompted them to do? These questions can also be asked of educational resources.

The thing is, on the back of relatively simple metrics like this, some fairly sophisticated analytics can be applied, to derive information about the relative effectiveness of different messages and campaigns. I am not a numbers wonk, but I sense that it would be useful to try to bastardize these kinds of analytics for educational contexts. Even the failure of this effort would tell us something about what it is we need to measure in terms of “educational analytics”.

Great post!

Here’s a quote from Stephen Jay Gould that relates to your post. It’s from his 1977 book Ever Since Darwin.
“… ecology is the study of organic diversity. It focuses on the interaction of organisms and their environments in order to address what may be the most fundamental question in evolutionary biology: ‘Why are there so many kinds of living things?’ … During the first century of Darwinism, ecologists pursued this question with little success. In the face of life’s overwhelming complexity, they chose the empirical route and amassed storehouses of data on simple systems in limited areas. Now, nearly twenty years after the centennial of Darwin’s Origin of Species, this poor sister among evolutionary disciplines has become a leader. Spurred by the efforts of scientists with a mathematical bent, ecologists have built theoretical models of organic interaction and appplied them successfully to explain data from the field. We are finally beginning to understand (and quantify) the causes of organic diversity.”

Ecologists apparently devoted 120 years to gathering descriptive data before it became really useful. This seems like a normal trajectory for scientific domains. I suspect some educators would take exception to your characterization of their research results as garbage. But your point is well taken. We’re still in the early decades of education’s 120 year trek.

Nice post. Very fun.

Gary