Jesper Balslev, research consultant and Ph.D., Copenhagen School of Design and Technology

Published by “Unblackthebox”, December 2021

For the last six years I have studied policy papers on the value, effects, and/or the potentials of digital learning technologies in education. It is a fascinating field to study. Especially one enigma has emerged: how self-conscious is “political” or “institutional” thinking really? Is there somewhere in the political sphere (at global levels, at EU-levels, or at national levels) a unifying consciousness (or just an intelligent database) that synthesizes the evaluations, the whitepapers, the strategies, the visions, and the reports for the benefit of future writers and researchers – or are the systems populated with groundhog-day writers, who start praising the potentials anew, every day?

Corona

Reports show that companies behind products like Microsoft Teams[1] and Zoom[2] have experienced remarkable growth rates the last couple of years. This, of course, is caused by the choice of schools and educational institutions to use their platforms to compensate for the inability to teach physically during lock-down periods.

Corona lockdown has been a gift for the learning sciences, in the sense that it created an ideal test situation over a longer period. On the same day, in Denmark, practically the whole nations’ pupils and students, from first grade to university students, were subjected to a similar educational situation: taught virtually through Microsoft Teams or Zoom platforms. This, one might think, ought to provide a convincing empirical base, to answer a range of questions, that were more difficult to answer before, when the use of digital platforms was more sporadic, differed from school to school and from municipality to municipality.

In the following I will try to find out whether political investments in ICT are warranted by evaluations, in the wake of Corona, i.e. through the analysis of thirteen evaluation reports[3] published between April 2020 and February 2021.

Some of them are published by higher education (HE) institutions, others by different consortia of HE-institutions, some by unions, some by government-funded think tanks, and finally by government funded evaluation institutes and educational boards (“styrelser”). Some have commissioned the services of consultancies, some are “home-crafted”; methodologically most of the reports use survey methods.

Let us go through some of the most startling findings, starting with the negative findings:

  • Motivation: in a survey sent out to 12.000 pupils, two thirds indicated that remote teaching is lessmotivating than normal. In another study, 40% of the students expressed that they had difficulties in maintaining concentration and motivation in virtual environments. 
  • Teaching: The danish consultant house Rambøll concluded that teachers found it difficult to gauge the students’ reactions in virtual environments, making it hard to practice adaptive teaching. A study published by the danish institute of evaluation made the startling observation that only 4% of pupils preferred online teaching. An evaluation commissioned by a consortium of 9 HE-institutions, concluded that most students found that their experience of online-teaching was negative. 56% of teachers made the assessment that the use of virtual tools “under normal circumstances” would create lesser learning. 7/10 experienced less joy when using virtual platforms.
  • Socially: The same study concluded that there is a loss of sense of belonging and loss of collective engagement. Another study concluded that pupils are “pressured on their social well-being” – 92% of the pupils missed their friends, 70% missed the teaching, and 60% missed their teachers. Nearly half of the students felt lonely in pure, virtual environments. 

On the positive side:

  • A segment of pupils appreciated the fact that classes started on time, and that they were not disturbed by their classmates.
  • All institutions succeeded in maintaining educational offerings, despite physical lock-down.
  • Flexibility and less time used on transport together with more efficient meetings has been cited as a positive effect.
  • The students mentioned flexibility and videos that can be revisited, as a positive aspect
  • 70% of the pupils found that the teachers did well under Corona, and a corresponding share of parents agreed with their children: The teachers handled the situation with great skills and enthusiasm.  

Prior to Corona, larger synthesizing meta-analyses concluded that 

  • “The results […] show no appreciable improvements in student achievement in reading, mathematics or science in the countries that had invested heavily in ICT for education”.[4] 
  • ”The implications from these findings suggest that we should not expect large positive (or negative) impacts from ICT investments in schools or computers at home.”[5]  
  • “However, 60 years of evaluation data show no major quantum leap in the impact of technology on learner outcomes. Most of the current technological interventions in schooling remain average or below in their ability to enhance student learning—when the technology is used in schools and classrooms.”[6]

Conclusion:

The corona-evaluations do not alter that image: it is difficult to claim that digital tools have caused improvements in student achievement, we cannot expect positive or negative impacts from ICT investments, it is safer to expect that technological interventions remain average or below in their ability to enhance student learning.

Evaluations show that notions of “digital generations” that are especially motivated by technology is impossible to corroborate. It is a myth that we should now safely bury, once and for all. It is also difficult to corroborate that technology facilitates adaptive teaching, or that technology is intrinsically motivating. 

On the other hand, findings seem to stress that (perceptions of) successful learning presupposes physical meetings, collective teaching, and many social aspects of education. Schools have been, and should remain social institutions.

Secondly, we should resist the temptation to think that different evaluations, made by different actors, in different settings and that measure on different parameters are commensurable. 

  • Synthesizing research requires coordination between actors, and problems must be addressed one a time. The evaluation practice in Denmark, consists of a flurry of actors measuring each their point of interest, rarely building on the efforts of others. 
  • It is difficult to gauge the effectiveness of Edtech if it co-exists with commercial offerings (Facebook, Instagram, Youtube etc.) in the same interface as learning platforms and learning materials. 
  • We need analogue control groups. The danish evaluations should be compared with schools that have used, or experimented with, analogue, remote teaching methods. This would create a clearer picture of the value of digital learning environments.
  • The potential of digital non-use should be explored to address the many sources of frustration. Potential to improve remote education by implementing non-use of commercial platforms, or to decrease the scope, the scale or the intensity of digital platforms needs to be investigated. 
  • Finally: There is no sense in evaluating if consequences are not drawn! Results seem to show that digital learning environments consistently have failed to meet a lot of the hype from vendors throughout the decades. This should lead to a more systematically skeptical practice, vis-à-vis the adoption of Edtech, and risks should be shared between vendors and educational institutions. 

[1] https://www.theverge.com/2021/4/27/22406472/microsoft-teams-145-million-daily-active-users-stats

[2] https://www.bbc.com/news/business-56247489

[3] They can be downloaded here: https://jesperbalslev.dk/10-rapporter-om-digital-laering-under-corona/

[4] OECD. (2015). Students, Computers and Learning. https://doi.org/10.1787/9789264239555-en

[5] Bulman, G., & Fairlie, R. W. (2016). Technology and education: computers, software, and the internet. Cambridge, Massachussets.

[6] Not all that glitters is gold – can education technology finally deliver? Hattie, J. & Hamilton, A. A Joint Publication / Corwin & Cognition Education Group (2020).

Relateret: Video-conferencing fatigue