This is an archive copy. The original article was posted on the P2PU blog on 9 Dec 2012. You can find it here.
I like data and evidence. My father is a civic engineer and I spent (too) much of my youth traipsing around building sites learning about foundations, structural beams, weight distribution and finite elements. I loved it (most of the time). Many of my father’s friends are architects and engineers and I grew up immersed in their way of thinking about the world. Always seeking to understand inner structures and relationships, predicting behaviors, and designing solutions.
I mention all this because my personal history may explain my ambivalent position with respect to the new field of learning analytics and efforts to assess student learning. It may explain why it has taken me longer than others to fully acknowledge how difficult (and potentially futile) it is to apply a measurement mindset to learning and education. But also, why I can’t get myself to agree with those who argue it is not even worth trying.
Bad Analytics
Heisenberg’s uncertainty principle explains that by measuring one property of a particle we reduce our ability to measure other properties. The other properties seem to disappear. That sounds roughly like what is happening in learning analytics, with the addition of a bitter twist. We are measuring the wrong things. And the right ones disappear.
A lot of what people measure is driven by what we can measure, not necessarily by what we should measure. This is true not just in learning. And in some fields it is not a problem, because the difference between what is important and what we can measure is small. In learning that difference is huge.
For building projects there are measurements that make sense. We can start with measurements that will help us determine if a building will fall over in strong wind. And there are lots of other indicators that we can track, such as energy consumption, ventilation, noise, levels of social interaction its inhabitants experience that all make some level of sense. It’s worth noting that some of these relate to personal preferences and many are conventions that have changed over time. But even in buildings, none of these measures fully explain how a building feels. Some buildings make you feel at home and safe. Others let your mind wander, yet others calm you down and inspire awe. A Pattern Language is a fantastic book about towns, buildings and communities. It looks at some of the design principles which shape how we feel about particular space. Its approach feels like a good direction for learning analytics, grounded in evidence and experience, but respectfully preserving the magic that is unmeasurable.
Unfortunately in learning, the things we can measure are very different from the things that are important. Even the baseline is hard to define. What is the equivalent of a building that is standing up and not collapsing? Is it the number of students who pass a standardized test, who graduate high-school, or complete college? These indicators seem straightforward, and familiar, but they were designed as nothing more than proxies for a vague set of skills and experiences, many of which were directly influenced by the needs of the industrial economy. That economy is crumbling in front of our eyes, but the education system seems more resilient to change than old steel mills and sweltering plants.
If we try to remember an experience in our lives that we associate with good learning or education, few of us will think of these indicators. More often than not we will remember inspiring moments and role models, times when were gripped by curiosity about seemingly random things (“I collected rocks, like a geologist … at age 5″), new ideas that came into our minds and stayed there. We remember people that influenced and helped us, or people we influenced and helped. We recount successes that are often very personal and local and may seem irrelevant to others. And we remember that learning felt good. None of which lend themselves to easy measurement.
There is another group of people who talk about really hard exams they had to pass, and how proud they are of their grit and persistence. That’s also interesting, but a little tangential for this post.
Why Analytics are Nevertheless Important
The people who measure, too often measure the wrong things. But the people who care about the right things, too often don't have good alternatives. As a result, we start caring about the things we measure, rather than capturing the things we care about.
We need a pattern language of learning, to describe principles that are grounded in evidence and experience. Many of us have a natural understanding of what it means to learn, but I am frustrated that we have been unable to present a more compelling alternative to the engineer’s tables of data. Poetic descriptions of the magical feeling of learning, and anecdotal stories of moments when it happened are not enough.
The stakes are rising. Education technology is booming and millions of funding from private and public sources are being pumped into a new field, called learning analytics. It aims to track student progress at the micro level, and mine the aggregate data from millions of students, to identify where they get stuck – and how to get them unstuck. The goals are laudable, but the implementations reduce learning to an engineering problem, in which efficiency can be optimized. The magic is lost.
The reason I care is that great learning opportunities are not evenly distributed within communities, countries, and across the world. In order to give more people better opportunities to learn, we have to understand where and how the magic happens, and how technology can help us scale it.
The Art of Learning
My father is an engineer, but my mother sang, played the piano, and, if she felt like it, kicked her shoes off and danced. Her heart was full of stories and adventures, which she shared with the world around her. She made people laugh and cry, and even though she was never formally an artist, I can’t think of a better term to describe the way she lived her life.
Learning is full of magic and wonder, when we approach it with an artist’s sensibility and passion, instead of observing with an engineer’s eye for efficiency. The latest innovations in learning are seeking inspiration in the wrong place.
The Art of Learning is an old practice that we can remember and re-imagine, rather than replace, with new technology. Understanding its patterns will help us design better spaces for more people to experience it.