New study measures the wrong things to show negative effect of digital tools in the classroom

A new study by MIT colleagues (the actual study / the guardian’s post about it) on the negative effects of digital tools in the classroom is a good example of why I am frustrated with a lot of education research.

The authors show that students in an introductory Economics course at West Point Academy performed significantly worse on class assessments if they were allowed to use laptops, phones or tablets during the classes. That seems shocking, and will–no doubt–be used by those who are critical of digital learning technology to suggest we should abolish these tools from the classroom.

We all know that having digital tools around and using them to access the Internet can be distracting, and there is plenty of evidence that context switching is the number one killer of sustained cognitive activity. But there are issues with this type of study.

It measures success as exam scores. That’s what we usually do in education. Yet we don’t challenge the quality of the assessment enough. Because for many assessments there is at most a short-term and relatively tenuous relationship between scores and actual learning. And in fact, the study finds that there was only a negative effect when looking at the results from those assessments that are least able to recognize deeper learning and understanding: “permitting laptops or computers appears to reduce multiple choice and short answer scores, but has no effect on essay scores”.

Secondly, the computers were not used to support the learning activities in any thoughtful way. They were simply permitted as note taking devices. So, the study really shows that note taking on a computer or laptop is more distracting than note taking on a piece of paper, when sitting in an Economics class. However, even The Guardian, picked up a very different conclusion, that digital tools decrease academic performance.

Maybe we should inverse the study’s conclusion and instead of pointing the finger at distracted students, challenge professors, and redesign curricula, to make learning more meaningful and engaging. The course was based around a standard economics textbook and online assessments. We don’t see negative effects on learning when kids are developing Scratch projects, or building Minecraft worlds, and computers have much less power to distract when learners are working in a maker space on physical projects. However, if they are sitting in a lecture, listening to a professor going through the chapters of a textbook, they are more likely to look for distraction. Who can blame them?