The EdTech Revolution Has Failed

The case against student use of computers, tablets, and smartphones in the classroom

Introduction from Jon Haidt and Zach Rausch:

When smartphones and social media platforms swept into teens' lives in the early 2010s, schools experienced their own digital revolution, with 1-to-1 laptops, tablets, and iPads becoming staples in classrooms across the Western world. (1-to-1 means one device for every student.) A decade later, the revolutionary optimism is fading. One OECD review found that most educational technology (EdTech) has not delivered the academic benefits once promised. Meanwhile, global test scores in math, science, and reading have been plummeting, as you can see in Figure 1 below. These trends were exacerbated by the COVID-19 pandemic, but they began in the early 2010s, just as digital devices were being placed on students’ desks. 

Global PISA Test Scores in Decline

Figure 1. Declines in math, reading, and science scores averaged across the 38 OECD countries. Image source: The Atlantic, from the OECD.

In The Anxious Generation, we advocated for phone-free schools. We made the case that locking up phones from bell-to-bell would improve student performance and focus, and the quality of students’ in-person relationships. (That seems to be happening, at lightning speed, and with very positive effects.) However, the impact of EdTech more broadly on student outcomes was unclear to us. Since publication, we've engaged with leading experts on the subject, and we are now publishing a series of posts to address questions that many educators, parents, and students are asking: Is EdTech actually better than traditional learning methods? When is it helpful, and when do the distraction effects overwhelm the pedagogical benefits? At what age (if any) should students start using iPads and Chromebooks in the classroom? And is the sudden proliferation of 1-to-1 devices partially responsible for the global decline in test scores over the last decade?

In the first post in our series, Everyschool co-founder Amy Tyson addressed five major myths about screens in school. In this post, neuroscientist and educator Jared Cooney Horvath goes further, arguing that the EdTech revolution has been a failure. He explains why schools across Europe and Southeast Asia are moving away from student-facing digital technology, favoring more traditional methods—and why American schools would benefit from considering a similar shift.

Jared is the author of six books and over 60 research articles. He currently serves as an honorary research fellow at St. Vincent’s Hospital in Melbourne and leads LME Global, a team dedicated to making the latest brain and behavioral research accessible to teachers, students, and parents. In December of last year, Jared reached out to Jon about a review article he was writing. Jon read an early draft of the article, found it to be powerful and well sourced, and invited Jared to consider publishing a shortened version of it here at After Babel. We hope you find it as thought provoking as we did.

— Jon and Zach

p.s. We are looking for a variety of perspectives on this important topic. We would especially value hearing from those who can make a strong case for the benefits of 1-to-1 digital tech in the classroom, ideally backed by research. If you can make such a case, let us know in the comments below and submit an essay to zach@anxiousgeneration.com.

---

The EdTech Revolution Has Failed

by Jared Cooney Hovrath

In May of 2023, schools minister Lotta Edholm announced that Swedish classrooms would aim to significantly reduce student-facing digital technology and embrace more traditional practices like reading hardcopy books and taking handwritten notes. The announcement was met with disbelief among pundits and the wider international public: why would an entire country willingly forgo those digital technologies which are widely touted to be the future of education? 

In a recent survey, 92% of students worldwide reported having access to a computer at school. In New Zealand, 99% of schools are equipped with high-speed internet while in Australia the student-to-computer ratio has dipped below 1:1 (meaning there are more computers than students in school). In the U.S., government expenditure on EdTech products for public schools exceeds $30 billion annually.

With these numbers, you’d think EdTech had clearly demonstrated a strong beneficial impact on student learning.

Think again.

(In this article, I’m defining EdTech as any student-employed, internet-connected digital device; this includes computers, laptops, tablets, cell phones, and smart watches. I am not discussing or evaluating the use of digital devices by teachers.)

An OECD international review of the impact of computers in education reports:

“Students who use computers very frequently at school do a lot worse in most learning outcomes…And perhaps the most disappointing finding of the report is that technology is of little help in bridging the skills divide between advantaged and disadvantaged students.”

After reviewing 126 research studies exploring technology-based education interventions, the global research center J-PAL concluded:

“Initiatives that expand access to computers…do not improve K-12 grades and test scores. [Furthermore], online courses lower student academic achievement compared to in-person courses.”

A recent analysis investigating the impact of computers on reading performance among K-12 students across the U.S. concludes, “…even small daily amounts (30min) of use of digital devices in classrooms are negatively related to scores on a reading comprehension test.” A similar analysis of learning moderators within university settings concludes, “…expanding the use of…technology at the expense of other forms of instruction is likely to have detrimental effects on achievement.” Yet another set of analyses reported that investing in air conditioning has a more beneficial impact on student learning than investing in a laptop for every student (ES = 0.21 vs 0.16)

“But wait,” some readers might say: “if giving every student a laptop has an effect size of 0.16, then surely this means digital tools can improve learning!”

Let’s take a closer look at the data.

Situating the Data

Since the 1980s, a number of meta-analyses (and meta-syntheses pooling these analyses) have been conducted exploring the impact of digital technologies within varied fields of learning. What do the weighted mean effect sizes show?

• Math: ES = 0.33 (22 meta-analyses / 1060 Studies / 1464 effect sizes)¹

• Literacy: ES = 0.25 (17 meta-analyses / 736 Studies / 1547 effect sizes)

• Sciences: ES = 0.18 (6 meta-analyses / 391 Studies / 567 effect sizes)

• Writing Quality: ES = 0.32 (6 meta-analyses / 75 Studies / 85 effect sizes)²

• Specific Learning Needs: ES = 0.61 (10 meta-analyses / 216 Studies / 275 effect sizes)³

At first blush, this looks very promising: seeing as each effect size is larger than zero, surely this means EdTech is working, right?

Not quite.

In 2023, educational statistician John Hattie released Visible Learning: The Sequel. In this major work, he analyzes over 2,100 educational meta-analyses exploring 357 different moderators affecting learning within the typical classroom. What he found was equal parts surprising and predictable: nearly everything has a positive impact on student learning. In fact, of the 357 included learning moderators, only 33 reported a negative effect size (this includes things like abuse, malnutrition, illness, and mental disorders). In other words, 91% of everything a teacher does can be said to improve learning.

Although this finding has doubtless been influenced by publication bias and the file-drawer problem, one additional practical explanation concerns engaged time-on-task. Students spending any amount of time explicitly focused on learning novel material would be expected to demonstrate improvement compared to students spending no time focused on learning novel material, regardless of the specific tool or technique employed. 

Seeing as nearly every strategy serves to increase engaged time-on-task (this includes employing air conditioning during hot days), then comparing the impact of digital technologies to ‘zero’ makes little sense. So, when considering effect sizes within education, what is a reasonable baseline? Acknowledging that this is a tricky metric within social science, yet remains the strongest representation of quantified impact we possess within education, what effect size would constitute ‘meaningful’ improvement?

Data suggests that in order for students to maintain a 50th percentile rank nationwide, they must improve an average of 0.42 standard deviations per year (calculated using standardized reading and math data across K-12); anything below this will likely lead to declining rank and vice versa. A similar analysis places this value at 0.46, suggesting an effect size of around 0.44 would be a reasonable educational baseline. As a secondary estimate, when the effect size from all 357 moderators noted above are pooled, Hattie reports the average value sits at a less conservative effect size of 0.4. In fact, he calls this a ‘hinge-point’ and recommends that only those tools and/or strategies with values above this level can be seen to ‘work best’ and should be considered for mass inclusion across education, as those have the greatest chance to deliver the greatest impact to the greatest number of students.

Using the less conservative baseline of 0.4, the aforementioned meta-analyses look much weaker. In fact, the only realm within which digital tools seem to be meaningfully beneficial is in the realm of specific learning needs (a topic we will discuss later in this piece). Use of these tools outside of this context may well be driving learning; unfortunately, that learning will be slower, less robust, and likely lead to a drop in rank compared to other, more powerful, non-digital methods.

The fact that dozens of meta-analyses comprising thousands of calculated effect sizes has done little to quell excitement over digital technology in education suggests that we may be dealing less with an issue of evidence and more with an issue of wishful thinking. So let’s step back from the data and explore why it is that digital technology in students’ hands may be doing more harm than good.

Multitasking Is Really Bad for Learning

To understand the primary reason why digital devices make learning so difficult, we need to take a quick cognitive tangent. Rest assured, we will return to EdTech and tie this together shortly.

A commonly used phrase amongst educational researchers is, “Attention is the gateway to learning.”

Perhaps the best way to conceive of human ‘attention’ is as a filter. Much like 3-D glasses, which only allow certain wavelengths of light to reach the retinas, attention only allows relevant information to pass into conscious awareness; irrelevant information is blocked out.

What, then, determines if a particular bit of information is relevant?

Much like board games, every task we undertake comes with its own unique set of rules that dictate what actions are required for success. For instance, to successfully read these words, your ‘reading ruleset’ dictates you must move your eyes from left to right, hold each word in memory until the end of each sentence, use your fingers to scroll, etc.

Whenever we engage with a task, the relevant ruleset must be loaded into a small area of the brain called the Lateral Prefrontal Cortex (LatPFC). Whatever ruleset is being held within this part of the brain will ultimately determine what the attentional filter deems relevant or irrelevant.

Here's the problem: the LatPFC can only hold onto one ruleset at a time.

This means anytime we attempt to consciously undertake two tasks simultaneously, the brain simply cannot handle this; the best it can do is quickly jump back-and-forth between tasks, swapping out the ruleset within the LatPFC each time.

This jumping between tasks incurs three significant costs.

Time: It takes the brain about 0.15 seconds to swap out a ruleset, during which time all external information stops being consciously processed and learning slows considerably (this process is called task-switching cost).

Accuracy: Whenever we jump between tasks, there is a brief period of time when the two rulesets conflict and general performance suffers (this process is called cross-talk dual-task interference).

Memory: Memories are typically processed by a part of the brain called the hippocampus. However, when jumping between tasks, memories are more often processed by the striatum (an area of the brain linked to reflexive processes), ultimately leading to the formation of subconscious memories which are difficult to access and utilize in the future.

In sum, multitasking is among the worst things that students can do for learning and memory. When we multitask, we go slower, our accuracy drops, and our learning decreases significantly.

Why does any of this matter?

What is the Primary Function?

The primary function of any tool can be thought of as the main task or job said tool is believed to perform: or, as I often consider it, it’s the manner in which 80% of people spend 80% of their time employing said tool. 

Consider a hammer. There are many uses for a hammer: door stop, bottle opener, back scratcher, screw driver. etc. However, 80% of people spend 80% of their time using this tool to hit things – as such, if I were to hand you a hammer, you would begin looking for something to hit.

What is the primary function of a computer? When students sit down in front of a screen, what is the fundamental behavior that immediately jumps to mind?

A pre-Covid survey exploring how US students aged 8-18 utilize digital technologies both inside and outside of school provides the answer (values below are per week):

• 10 hours 44 minutes playing video games

• 10 hours 2 minutes watching television or film clips

• 8 hours 14 minutes scrolling social media

• 7 hours 32 minutes listening to music

• 3 hours 25 minutes doing homework 

• 2 hours 5 minutes doing schoolwork 

• 1 hour 14 minutes reading for pleasure

• 52.5 minutes creating digital content

• 14 minutes writing for pleasure

If we extrapolate and consider a typical U.S. academic school year of 36 weeks, these numbers suggest that students spend 198 hours annually using digital devices for learning purposes, and 2,028 hours annually using those same exact tools to jump around between scatter-shot media content. 

Do you see the issue?

Seeing as the great majority of students spend over 80% of their digital device time using these tools to multitask, the automatic response for a great majority of students using these tools has become multitasking.. Unfortunately, when we attempt to employ digital devices for learning purposes, this primary function quickly bleeds into student behavior. 

This is why, when using a computer for homework, students typically last fewer than 6 minutes before accessing social media, messaging friends, and engaging with other digital distractions. This is why, when using a laptop during class, students typically spend 38 minutes of every hour off-task. This is why, when getting paid as part of a research study to focus on a 20-minute computerized lesson, nearly 40% of students were unable to stop themselves from multitasking. It’s not that the students of today have abnormally weak constitutions; it’s that they have spent thousands of hours training themselves to use digital devices in a manner guaranteed to impair learning and performance. It’s also that many of the apps being run on those devices were carefully engineered to pull young people away from whatever they were doing.

Perhaps we can draw an analogy: imagine I sat a group of people around a jug of beer and asked them to use that beer to learn about buoyancy. This tool (beer) is perfectly suited to the task, and if the group of people were non-drinkers, I would expect no problem achieving my learning goals. However, if the group of people were alcoholics - individuals who had spent thousands of hours training themselves to use beer in a manner that does not concern buoyancy - then the tool would most likely hinder my learning goals.

And perhaps this is the key point: I’m not saying that digital technologies can’t be used for learning; in fact, if these tools were only ever employed for learning purposes, then they may have proven some of the most important academic inventions ever. The argument I’m making is that digital technologies so often aren’t used for learning that giving students a laptop, tablet, or other multi-function device places a large (and unnecessary) obstacle between the student and the desired outcome. In order to effectively learn while using an unlocked, internet-connected multi-function digital device, students must expend a great deal of cognitive effort battling impulses that they’ve spent years honing - a battle they lose more often than not. (of course schools do often try to implement blockers and restrictions, but this opens up an eternal cat-and-mouse struggle, and the mice are very good at finding ways to evade the cat.)  

To be fair, even before digital devices took over the classroom, students struggled with lack of attention, shallow thinking, overconfidence, and other learning problems. However, with digital devices, the frequency of these problems increases. Many schools were already struggling with issues of engagement, so we need to consider what additional harms may have occurred in the early 2010s when so many schools adopted tools that can be described as distraction machines. 

The Three Excuses

Seeing as enthusiasts can’t lean on empirical data to support the adoption of student facing, internet connected digital devices across education, they frequently turn to three seemingly compelling arguments to excuse the lack of clear evidence of effectiveness. When placed in context, however, these arguments lose a lot of their exculpatory power.

Excuse #1: Digital devices have so much potential (or “Tech is the future…”).

It’s easy to get swept up in the promise of computers in education. Returning to the J-PAL review quoted earlier, after concluding that computers do not benefit learning, this group goes on to state:

“Computer assisted learning shows considerable promise…against this backdrop, promising uses of education technology have the potential to support massive inroads in learning.”

Potential is what something could be, what it should be, and what we hope it will be – not what it actually is. This means that arguments from potential do not accurately reflect reality; rather, they reflect the faith, belief, and desires of the speaker. In fact, when people say that computers have the potential to drive student learning, they are tacitly acknowledging that these tools are not currently achieving this aim.

I am not arguing against having hope for EdTech; there is a good chance that somebody will someday create a program that outshines even the best teachers in the world. I am simply pointing out that this has not yet happened, and that promissory arguments are not a solid enough foundation upon which to change educational practice at  mass scale.

Excuse #2: Digital devices are ubiquitous (or “They ain’t goin’ nowhere…”)

Health insurance. Taxes. Video games. Dating, marriage, and divorce. Mortgages. Laundry. Student loans. Litigation. Self-defense. First aid. Pet care. Superannuation. Yoga.

I mention these things to highlight that just because something is ubiquitous does not mean it needs to be explicitly taught in school. Teachers have long recognized that they are not alone in the journey of education, and that many essential concepts are meant to be passed along by parents, peers, society, and life experience.

With that said, an argument could be made that schools should be teaching these ubiquitous subjects; that it’s the responsibility of education to ensure all students are well versed in those things they are certain to encounter in their adult lives. 

I think this argument has some merit.

However, to argue that a topic should be taught is far different than arguing that all things should be taught through that topic. The former is an argument about curriculum; the latter is an argument about pedagogy. For instance, you might believe we should teach table manners to students (curriculum), but that’s different than arguing we should teach all classes in a dining room over dinner (pedagogy).  Determining what to teach and how to teach are two very different considerations.

Here is where the excuse from ubiquity goes awry. Through some linguistic alchemy, the argument “we should teach computer skills” has morphed into “we should teach all skills through a computer.” Teachers around the world are being asked by developers, politicians, and their own principals to adopt digital technologies within their classroom, even if these tools aren’t well suited to the learning goals these teachers desire.

Although teaching computer processes, coding skills, and digital etiquette are worthwhile goals (curriculum), it does not follow that we must reconfigure all academic subjects to achieve this goal. When it comes to effective teaching and learning (pedagogy), we should select the tool best suited to the job; not the tool that is most prevalent. Just as power tools are confined to the woodshop and Bunsen burners are confined to the science lab, perhaps student-facing digital devices should be confined to a dedicated computer lab within school; they should not be freely available whenever and wherever. Or perhaps devices should be disconnected from the internet and/or locked using an LMS system that only allows for specific programs to be employed.  Employing mediated devices or a singular location may ensure that teachers choosing to employ tech have a clear reason for doing so, and students choosing to employ tech have oversight and less opportunity for detrimental multitasking.

Excuse #3: Schools are using digital devices incorrectly (or “It’s all your fault…”)

As referenced earlier, following a three-year analysis of hundreds of thousands of data points and concluding that computers significantly reduce learning, the OECD report went on to state:

“One interpretation of all this is that building deep, conceptual understanding and higher-order thinking requires intensive teacher-student interaction…Another interpretation is that we have not yet become good enough at the kind of pedagogies that make the most of technology.”

In the interest of digging deeper, let’s give this excuse the benefit of the doubt. If computers are in fact being used incorrectly in schools, then what is the correct way to use them? As the OECD report states:

“[Digital technology] is linked to better student performance…when computer software and internet connections help to increase study time and practice.”

In case you missed it, let me rephrase that: learning improves when students spend more time learning. 

This revelation is not unique to computers. Flashcards, workbooks, an empty milk jug, my great aunt Justine… as the work of John Hattie mentioned above reveals, when used to increase study time and practice, literally any tool will improve student performance. The relevant question is whether or not computers and tablets actually do lead to increased learning time. Unfortunately, as we’ve already established, the answer to this seems to be ‘no,’ they are at least as likely to increase time spent on video games, video watching, social media, and listening to music.

When Does EdTech Bring Benefits?

While I am generally critical of the way that EdTech has been put into the hands of students, I do see two ways in which digital technologies can be extremely helpful for teachers and for students. The first concerns teaching. When digital tools are used and controlled by a well-trained teacher to drive pedagogy, then any problems of multitasking and distraction that occur due to student-facing digital technologies can be averted.

However, simply asking teachers to employ computers in their teaching will not, in itself, boost learning. It is essential that teachers continue to select the strongest pedagogical techniques, regardless of whichever tool they choose to employ. For instance, employing PowerPoint slides during a didactic lesson has sometimes been found to impair the amount of verbal information students retain. Again, this is not due to the tool itself; rather, it is determined by the way in which teachers choose to use this tool.

In the words of the great media scholar Neil Postman, “I do not say that [digital technologies], when used properly by a skilled teacher, will not help [learning], but I doubt that it can do better than pencil and paper, or speech itself, when used properly by a skilled teacher.” In other words, digital tools do not obviate the need for sound pedagogy; and sound pedagogy will rarely be dependent upon digital tools.

The second instance where EdTech may be helpful concerns issues of learning abatement. For instance, if learning is abated due to school closures necessitated by environmental disasters (e.g. poor air quality from fires), sociological upheavals (e.g. violent political protests), or health risks (e.g. local epidemics or global pandemics), then it’s worthwhile to employ digital technology as a solution.

With that said, issues of digital distraction, diminished impact, and socioeconomic divide persist or increase during these forms of digital-based distance education. In fact, in a recent survey, 95% of students admit to media multitasking during distance learning,  with 15% admitting going off-task over 30 times per session.

As another example, if learning is abated due deafness, blindness, orthopedic, or other forms of specific learning disabilities that preclude students from engaging with non-digital learning materials, then it’s worthwhile to employ digital technology as a solution.

In the end, if there is no other avenue or option for engaging with learning material besides digital technology, then we are well justified in employing it. However, if there are two or more options for engaging with learning material, then it is best to select the tool that will yield the best results. As we’ve seen, that tool will rarely be digital in nature.

In Conclusion…

Enough.

As mentioned above, we are seeing a shift away from digital reliance in schools across many european and southeast asian countries, not just Sweden. I urge readers to take these shifts seriously and look closely at emerging data in schools that reduce their use of EdTech. If the educational and cognitive research that I have discussed is accurate, then we should see not only improvements in student learning, but also improvements in student relationships, mental health, and physical wellbeing.  

1. Mitchell (1987), Ran et al. (2021), and Benavides-Varela (2020) omitted from analysis due to looking at students with learning needs. Note: Mitchell (1987) was an as yet undigitized dissertation. Meta-analytic results, though, can be found here.

2. Two studies comprising two effect sizes were omitted from Little et al. (2018) for explicitly exploring students with learning needs. Two recent meta-analyses were added: Graham et al. (2012) and Wen & Walters (2022).

3. Mitchell (1987), Ran et al. (2021), Benavides-Varela (2020), and the two omitted studies from Little et al. (2018) were added to this analysis.