Responses to comments on: Social Media is a Major Cause of the Mental Illness Crisis

The effects are large and sometimes bi-directional

This week at the After Babel Substack is different from previous weeks. Today we have two posts, beginning with a guest post from Jean Twenge showing that the big rise in mental illness that began around 2012 does not seem to be caused by increasing pressure at school, as some have proposed. 

We also have this “response to comments” post, written mainly by Zach Rausch, who is the lead researcher for this substack. Zach has a master's degree in psychological science from SUNY New Paltz (Where he studied and worked extensively with Professor Glenn Geher). He handles all the data sets, collaborative review docs, and graphs for this Substack and the two books I’m writing. You will be seeing more posts from Zach in the coming weeks.

— Jon Haidt

---

We (Zach and Jon) have decided to share comments and responses from previous weeks in a new series of brief posts. Today’s post is a followup to Jon’s earlier post: Social Media is a Major Cause of the Mental Illness Crisis, where he showed that the evidence against social media is not just correlational; there are many longitudinal, RCTs and naturalistic quasi-experiments too. This post is for people who read the previous post and had questions or asked questions in the comments section. It’s also for data geeks and people who just like to look at graphs and effect sizes about social media and teen mental health. 

Comment #1: Is social media really worse than other online activities?

The Critique: When you look at studies of digital media use, the correlation between general internet use and poor mental health is as large, or even larger, than the correlation with social media use. How can you explain this?

(Thanks to David Stein)

RESPONSE: David, as usual, your very careful reading finds errors in my (Jon’s) work. (Readers should check out David’s blog, The Shores of Academia.) You are correct that in the datasets used by Orben and Przybylski (“O&P”) and in some of Jean Twenge’s earlier writings, the correlations with “browsing the internet” or “general internet use” are often similar to those for “social media use.” Up until now, I have written that off by thinking that “general internet use” probably includes a lot of social media use. But I can’t back that up, so I’ll need to change what I say. You and I both agree that the problem with using “digital media use” is that it includes “watching television” or “Netflix videos,” which are not nearly as harmful as social media. 

However, my view was formed not just by the two O&P datasets I referred to in the post but by going through the literature reviews in section 5 of the Collaborative Review doc. Most of the time that a literature review or meta-analysis came out that purported to show no correlation with mental illness, when I read it, it turned out to be summarizing studies on digital media use. For example, see Odgers & Jensen (2020) (study 5.10 in our review doc); Ferguson, Kaye, Branley-Bell, et al. (2021) (study 5.17); and Tang, Werner-Seidler, Torok, Mackinnon, & Christensen (2021) (study 5.24). In contrast, those such as Orben (2020) (study 5.7) that focused on social media almost invariably find a larger effect. 

After writing the addiction section of my book, I can also add that I was wrong to focus so tightly on social media platforms. Those are the worst overall, but any app that is designed for “engagement” is designed for addiction. I’ll have more to say in the future about addictions to video games and the internet more broadly.

Comment #2: None of what I see justifies the word “MAJOR” in your title

The Critique: Jon showed many statistically significant effects in the post, but are those effects large enough to justify the title, “Social Media is a Major Cause of the Mental Illness Epidemic in Teen Girls”

(Thanks to Mark Hammer, Psyoskeptic, along with Indy and Anon in Academia for requesting that we post effect sizes). 

RESPONSE (from Zach): A few readers pointed out that Jon did not include effect sizes in his analysis. That’s true; this was not a formal meta-analysis where you weight the individual effect sizes by metrics of quality (such as sample size). However, if you look at the studies in each section, you can see that the studies are not just finding tiny effect sizes that are only significant by virtue of huge sample sizes. In the section on correlational studies, we concluded that the Pearson correlation is in the ballpark of r = .17 or even .20, which is large in comparison to other public health effects. 

Commenter Chris Said did us the favor of posting a link to a superb blog post he created and shared with Jon last year, where he showed that a correlation of .17 is large enough to explain a 50% increase in depression among teen girls. 

Among the experimental studies, the effect sizes are not usually very large, but true experiments are designed to test for causality in a situation that is usually fairly unnatural, such as asking people to reduce or give up social media for a few weeks, during which time people often compensate by doing more of other things on their phones. We don’t look to them to tell us how much of an effect social media has overall. To gauge that, the quasi-experiments are much more informative. What happened to whole communities (in Spain, Italy, and British Columbia) when the community got high-speed internet and the ability to shift into a phone-based lifestyle? Did we see an increase in mental illness of 3%? Or was it more like 10%? Actually, no. Guo found “significantly increased teen girls’ severe mental health diagnoses – by 90% – relative to teen boys’ over the period when visual social media became dominant in teenage internet use.” Arenas found “that fiber penetration significantly increases behavioral and mental health (BMH) cases in adolescents aged 15 to 19 years (one standard deviation increase in fiber penetration increases cases of BMH by 13.3%), and girls entirely drive this effect. We document an exceptionally large effect on cases of self-harm and suicide attempts among girls aged 15 to 19 years (+112.3%), but no significant effect for boys.”

The effects are similar in the other quasi-experimental studies, which you can find in section 3 of the google doc.

Finally, a recent meta-analysis by Lui et al. (2022), “Time Spent on Social Media and Risk of Depression in Adolescents: A Dose-Response Meta-Analysis,” found that across gender, “The risk of depression increased by 13% (OR = 1.13, 95% CI: 1.09 to 1.17, p < 0.001) for each hour increase in social media use.” And when we look at the rates separately for girls and boys, the rates are even more striking:  “...The association [for depression] was stronger for adolescent girls (OR = 1.72, 95%CI: 1.41 to 2.09) than boys (OR = 1.20, 95% CI: 1.05 to 1.37).” In other words, using social media for five hours a day instead of none raises the risk of depression by 65%. Using it for seven hours a day instead of none raises the risk by 91%, almost doubling it. These are large effects, not just statistically significant effects. 

Comment #3: Does causality run in both directions? 

The critique: How do we know the relationship between mental illness and social media is not bi-causal [or bi-directional]? In that, depressed and anxious teens turn to social media more often, not necessarily because the latter causes the former. 

(Thanks to Sam R)

To determine whether social media use causes depression or vice versa, we can look at the longitudinal studies in section 2 of the Collaborative Review doc. Those studies are able to show us what went up first: social media use, followed by increased depression, or the other way around? These studies often check for such reverse-correlation effects, and we find that the results tend to show that (1) those with pre-existing mental health conditions tend to use social media more than those without and (2) when adjusting for pre-existing conditions, social media use still has a negative effect on mental health (although there are exceptions). We can also look at the experimental studies in section 3 of the review doc, which show strong evidence that social media use has a direct negative impact on adolescent mental health (but, to be clear, that does not discount the bidirectional hypothesis).

One commenter on this post, Slaw, offers an insightful perspective on this exact question. Here’s what they wrote:

The writer Freddie deBoer has examined the trend of "mental health chic" in social media extensively, where influencers adopt a mental illness as a centrally defining characteristic and use that as the basis for attracting followers. His fear was that it not only romanticized mental illness but set up echo chambers that were ripe for the spread of social contagion. I have to wonder if there is a correlation between adverse mental health outcomes in girls versus the type of social media they are consuming.

Slaw's comment hits home with the point that the relationships between mental illness and social media are dynamic, complex, and often reciprocal. And so, you are correct to note that we need to more closely consider the bidirectional effects between social media use and mental illness.

---

Below are some additional interesting comments from readers:

COMMENT from Psyoskeptic: I haven't gone through all of the studies but in the way you relate them there seems an obvious point that hasn't been made. In each classification or kind of study, you differentiate those that show effects and those that don't. You don't list any effects that show the mental health benefits of social media. Given that the effect never goes the other direction one of two things must be true. The true effect cannot be 0, or there is something wrong with the measurement. As near as I can tell it never goes positive for boys as well so even there, one of those things must be true.

RESPONSE: Thanks, Psyoskeptic. You make an excellent point. Some studies report that more teens say they enjoy social media than teens who say it hurts them (And some studies do show positive effects for particular kinds of social media use). But when we focus on mental health measures rather than enjoyment, you are right. There are none that I know of that show that heavy users are in better mental health than light users or non-users.

COMMENT from Amber Muhinyi: I understand that the questions are focused on teens, but I wondered what happens when these teenage girls become adults. Do the mental ill health symptoms persist, even if social media use goes down as they settle into adult life and begin families etc?

RESPONSE: Thanks, Amber. We do have an appendix to the Mood Disorders document titled Variation by Age (Including Adults). Please check it out. We explore the rates of depression, anxiety, self-harm, and suicide among all age groups since 2010. You’ll find that in the U.S., younger Americans have much higher rates of diagnosed anxiety and depression relative to older Americans (across sex). Similarly, rates of self-harm are lower (and declining) among Americans above the age of thirty. On the other hand, rates of suicide are higher among both older men and women and have been rising since 2010 (But the relative increases are small compared to the younger cohorts). But, to your primary question, it is too early to know the long-term effects of early social media use on mental health in later adulthood. We’ll keep an eye out for studies of the older members of Gen Z.

COMMENT from Jason Manning: Good post, especially the point about how we should look beyond dose effects. One thought to add: "Boys are doing badly too, but their rates of depression and anxiety are not as high." But their rates of suicide are about four times as high, as are their rates of drug overdose. Women and girls are more likely to express distress in culturally recognized idioms of abnormality in need of treatment -- in traditional societies, spiritual possession, in modern societies, eating disorders, depression, etc. Males are more likely to turn straight to violence and reckless behavior.

RESPONSE: Thanks, Jason (and we love your book). You’re absolutely right, and the high and rising suicide rate for boys is the red alert siren.  We have been focusing on girls because the evidence linking girls’ mental illness to social media is consistently stronger than it is for boys. But we (Jon and Zach) are about to start writing the book chapter on what’s happening to boys. We are teaming up with Richard Reeves, author of the fantastic book Of Boys and Men. Stay tuned; we’ll have a lot to say about boys.

Thanks to everyone who commented. Now, if you haven’t read our most recent Substack guest post by Jean Twenge, please check it out.