I get the need for the science behind the assertion that social media creates anxiety and depression. But smartphones do so much more damage than that. I have run my own experiments with my 5 children and it was stunning to see the difference in their behavior. The first day was awful but by day 2 they started to change and by day 3 they were back to my amazing children. We all talked about the huge difference. This was in 2018. Every person can use their critical thinking skills to make rational decisions about what they experience in their life. You don’t need an expert to tell you. Just as both Jon and Abigail Shrier have written-we have acquiesced our decision making to someone else. We as parents need to take that back. You really do know what is right for your kids.
Hi, I think the point that you might be missing is that not every child has parents as involved in their lives as you are. There are many parents who don't have the time or the means to be as involved in their child's development (or some are uninterested entirely) who give phones to their kids without making the same considerations that you do. Changing the landscape of social media would help them in ways parental figures in their lives are currently not.
I completely agree that it is not just big, bad social media, it is the phones themselves. My daughter (almost 17) can be on her phone 5 hours a day (on a school day!) texting and FaceTiming, constantly checking, constantly waiting for something to happen on her phone.
May I ask, for how long did you take their phones away? And do you mean you physically took them away, or you allowed them a phone but took certain apps off? Thanks!
Physically took their phones away. We also implement phone free days and phone free standards ie when I am driving them around-no phone use, we talk, sit in silence or listen to music.
Go through this article and substitute “social media” with the word “cocaine.” Over and over, we talk about the potential benefits of brief and partial reduction of “social media” use, ignoring the fact that in every imaginable way, social media use is an addiction. So is cocaine use. With that in mind, of course a reduction in either behavior or substance will lead to some improvement in mental health—or none. But we’re missing the point entirely. No cocaine recovery center strives for a brief or partial reduction in use. None. And no parent would strive for a brief or partial reduction in cocaine use at home. And yet here we are talking about brief and partial reduction of social media use because parents lack the courage and information required to eliminate all phone use before the age of 18. The kids simply don’t need the risk of phones, which are destroying the real joy of life for children and parents, as Jon has said.
But here’s the real point. Simply stopping an addiction is never enough, as every recovering addict knows. You must replace the addictive behavior with something constructive. What children need most is unconditional love—the kind without the stain of disappointment and anger. But parents don’t know how to give that. They never got it themselves. Controlling children or enabling them is not loving them. And parents don’t have to figure out unconditional love and guidance on their own. Just go to the free and agenda-free websites RealLove.com and RealLoveParents.com. I have nothing to sell. But I do offer thirty—30—years of intense experience with teaching parents and children all over the world. It’s love they need, not social media or phones or indulgence or entertainment.
I would personally argue that cocaine makes you far more productive than social media, but that's a whole other aside. I do think that you are spot on that excessive social media use being treated as addictive is the correct way to go. However, I would not as far as to equate it with cocaine. Certainly there is no need to restrict social media usage just to medicinal use. The other issue is that unlike cocaine, the makeup of social media is far more malleable and can change form. Is it not better to find a healthier form of social media than to ban it entirely?
This must be Collis 2022, and the problem there was that the 'reduction' group actually ended up increasing its digital screen time:
===
We find that participants in the treatment group substitute social media for instant messaging and do not decrease their total time spent on digital devices.
[...]
Remarkably, although students in the treatment group significantly reduced their social media activities, their overall digital activities overall are not affected but, in fact, exceed those of the control group in block 2 (t-test, p = 0.026). This result indicates that students substituted or even overcompensated their social media usage with other activities.
===
In fact it seems SM time did not decline at all, as students just moved not only to WhatsApp but also to YouTube and TikTok (this is seen only in Supplement data).
Note also that Collis 2022 did not measure MH outcomes like depression or anxiety -- it measured outcomes like 'satisfaction with life'.
Of course, studies are incredibly beneficial and, in some cases essential, for decision-making. However, deciding if social media is problematic is not one of those. Talk to any parent or teacher, and it's probably one of the few things we all happen to agree upon. You can show me all the data in the world to try to convince me otherwise, but it's incongruous with real life experiences and what parents witness daily with their own two eyes.
Even scientists say it's not always wise to "wait for the science." Look where that got us with smoking. It wasn't until reviewing 7,000 articles in the medical literature that the Surgeon General concluded that smoking caused lung cancer and bronchitis in 1964 and then decades more before any meaningful policy and change occurred. We don't have that much time and we currently have it all backward. I think Jon has alluded to this in a previous post, and I agree. Instead of waiting for beyond a reasonable doubt that it's harmful to children, we should be waiting for beyond a reasonable doubt evidence that it's helpful. That it contributes to their thriving and flourishing. Especially with children where we must be more careful and prudent before experimenting on their one precious childhood.
I think your approach to this issue is far too reductive. The idea that one does not need a sufficient threshold for evidence to act with evidence belies the fact that such action would be unhelpful if there is not enough evidence to inform which actions should be taken. Your point towards smoking as an example doesn't undermine the desire for scientific consensus since industry lobbying played a large role in delaying regulations. There are plenty of instances where the opposite was true and more testing was required, take thalidomide for instance.
However, in this case I do believe that there is a concrete link between certain mechanistic uses of social media and negative health outcomes. That being said, that does not mean that all features of social media are responsible. It's a complex and multi-faceted topic with many variables to consider. And while yes, there is a sense of urgency behind this
campaign, trepidation is essential. A poorly constructed response to an issue can make matters far worse if care is not taken.
Appreciate your comment, John. That's my brain - start with the simple. You're correct that a poorly constructed response to an issue can make it worse. I tend to see things on a risk continuum. And I see much less risk to children by taking the "let's just wait" approach than running recklessly ahead or running ahead only partially informed. That might be an overly simplistic view, but that's just how my brain tends to work when it comes to children.
Yeah, I mean we often speak in generalities for the sake of brevity, but too often we forget that these generalities are underscored by several layers of often complex factors. This is a fault committed even by many of the researchers in this field, who speak about broad topics like social media and screen usage without considering that outcomes from these activities are highly mechanistic.
You are right in that we definitely do not need to wait for cross-sectional studies of fully mature adults before we start debating that action should be taken. The difficulty in constructing a plan of action is in creating regulations/reforms that service the needs of the people against the wants of government, corporations, and special interests. That's why I am particularly wary of support from certain institutions because yes, there definitely is a world where this goes astray and we end up with awful legislature like the Gramm-Leach-Bliley Act or the Patriot Act.
Hi Matthew, thank you for taking the time to engage with the meta-analysis and providing your own statistical analysis.
I have a couple critiques of your interrogation. The first is in defending the choice Rausch and Haidt made in sub setting studies by duration. They cite Anna Lembke's work on dopamine addiction as noting that withdrawal symptoms may last for up to two weeks. While the withdrawal hypothesis should be more concretely tested, the decision to stratify by study duration is not entirely arbitrary. Furthermore, any decision to remove studies from consideration such as the Collis and Eggers 2022 is being made on the basis of impropriety in the methodology/experimental design, to which there are many flaws in the case of Collis and Eggers as well as many others, even studies with a positive d.
The second critique is in your approach towards building a regression model. While your coding seems sound, the decision to construct the regression using only the overall effect sizes of each study seems inappropriate. Many of the studies done over a longer period of time were longitudinal, so their effect size is not static. Your regression does not capture the time evolution of effect sizes for some of these extended studies.
I will post some sort of penalized spline regression tomorrow. Dichotomizing into groups is simply not an optimal modeling approach even if you have some vaguely justified cut-off.
I used the effect sizes that Rausch and Haidt used, I will not be constructing a whole new meta-analytic database for this. I’d be happy to analyze it if someone else does though.
A lot of the smaller studies are populated with noise due to ability for natural variance in daily life to obscure any signal. Would you really weight a day's abstinence from alcohol the same as a month's break? If so, then how about a hours? Or just 60 minutes? Surely you would agree that the length of a time series influences its utility.
Also, including Collis and Eggers 2022, a study that did not holistically reduce social media usage, in a model for social media reduction studies seems highly inappropriate.
I’m not sure what this means: “A lot of the smaller studies are populated with noise due to ability for natural variance in daily life to obscure any signal.” If you are referring to sampling error, a random effects model is weighting by sampling variance. If you are referring to weighting by time, I am not going to weight the meta-regression by the independent variable.
I’m using the dataset that Rausch and Haidt used. I am not conducting a new meta-analysis here, I am critiquing Haidt’s re-analysis. If you would like to use a new selection criterion for a meta-analysis then I encourage you to do that. Ultimately, if you don’t have a well-defined and coherent inclusion/exclusion criterion then you are cherry picking studies. I’m sure you could find post-hoc justifications to exclude any one of these studies. But again, this is well beyond the point of my critique.
There are a variety of day-to-day factors that modulate your well-being (mood, sleep, stress, etc.) and performing a day's abstinence of social media without controlling for changes in the plethora of other moderators sets you up for having both high sample variance and sampling variance.
What Rausch and Haidt are arguing is that the Ferguson meta-analysis suffers from a poor inclusion/exclusion criterion. It is not post-hoc to say that a study that does not reduce social media fails to meet the inclusion criteria of having measured social media reduction.
Good point Christos. We will discuss this study in more detail in the next post (in brief, we'll show that digital screen time actually increased in the social media reduction group).
Hi Zach, I was wondering if you would take this approach with the other studies in the Ferguson meta-analysis, even including those who reported a positive effect size.
The problem with meta-analyses is that not all studies are created equal, and that milage varies greatly with methodology. Take for instance the Faulhaber et al. 2023 study.
Despite finding significant changes between the treatment and control group, the treatment group limited social media usage through self-monitoring which is problematic for many reasons: truthfulness about social media usage, reliability in estimating social media usage, etc.
Yes, some of the studies with a negative effect size are methodology flaws, but so do some of the studies that report positive effect sizes. What steps you are taking to balance this out?
These are good points. I agree that there are a number of issues across a variety of studies in the meta, on "both sides." Our focus in this post is primarily to show that there were clear moderators (e.g., duration) that should have been taken into account when conducting the meta.
In the next post, in addition to calculation errors/other moderators, we are primarily concerned about which studies violate the inclusion criteria that was laid out within the meta (and why). This Is why we will discuss Collis in more detail, as there are a number of issues that directly pertain to inclusion. We don't cover all of the studies, just the ones that relate to that specific point.
Independent of our response to this meta-analysis, we are working on a broader review of the experimental studies, where we plan to address some of your points.
Yes, I fully agree that duration is a key modulator as many of the short-term studies seem to be capturing all the noise and no signal. The 2021 Przybylski study was especially egregious and useless. Trying to extrapolate from one-day's abstinence to make claims about how social media generally affects well-being is nigh malpractice.
In your broader review, what demographic factors will you be trying to control for? I know sex is an obvious target, but I was wondering if you are considering other factors like political identification or gender/sexual identity as moderators.
A big issue with the Collis & Eggers study is that it only limited access to Facebook, Instagram, and Snapchat, of which they noted students just substituted those services with other social media applications.
The other main issue is that there was no attempt made to disentangle the well-being effects of social media from the general affects of college life (i.e. midterms and exams) so it is unclear as to how much social media usage was responsible for well-being.
This was very informative, not just on the details of the studies of the meta-analysis in general. (in the interest of promoting screen-minimalism in myself I actually printed it out so I could have a better experience reading it). I'm looking forward to reading the rest of the series.
I'm curious if you're planning to reach out to Chris Ferguson to give him the opportunity to respond to your analysis of his work. I really appreciate how you take a collaborative approach to this in providing people with competing views to share their perspectives alongside your own. I would be interested to hear his response if he took you up on the opportunity to do so.
I’m a Canadian high school educator of 35 years (French& Psych/ Masters in counseling and addiction) who has been teaching and learning about addiction and cell phone dependency and its effects on learning for 10 years. As I head back to the classroom next week, our provincial government in British Columbia has mandated a ban on cell phones, except when the teacher deems it necessary for learning. Teaching grade 11 and 12, I absolutely have to provide access to the Internet. I have a responsibility to teach kids about how to responsibly an ethically navigate AI ,how to interpret research, discern good research from poor research, write their own research papers, and understand peer reviewed studies, to name a few. However, as teachers, we desperately need help and strategies for navigating this incredible challenge. How do we prepare kids for 21st century learning without requiring them to have access to their addictive $1200 smart phones? One answer, of course, is mobile laptop carts in the classroom, which requires funding, which we don’t have.
An added challenge is that most teachers and parents don’t have the education around the importance of compassion and the danger of shaming when talking about cell phone with students. Anybody who’s done any work knows Gabor Mate’s work “ the opposite of addiction is connection”. When parents and teachers shame kids for using their phones ,even when they’ve been repeatedly told they can’t , all we do is drive them deeper into their dependency and increase their isolation from us. This is a huge challenge for teachers navigating cellphones in the classroom. We have to constantly be sure that we’re being respectful and kind when kids pull out their phones and they’re not supposed to. (If you’re not an educator, just think about what that’s like when you see 200 students a day and a large percentage of them have a dependency on their phones and do not realize it at all) . What I do with my students , whenever I see them with their phone out, even though we have a policy for keeping it away unless I ask them to pull it out, ; I quietly ask them “ Do you need to take that ?” and if they do, I ask them to step outside and deal with their issue ( there is no shaming and no scorn in my voice ever! This is super important) . Most of the time when I ask a student if they need to take their call or text, they just look up at me (like they’re coming out of a fog ) and apologize and say “I didn’t realize, sorry Ms W. “. So many of our kids reach into their bags and grab their phones,whenever their minds wander and they’re a little bit bored. In the addiction research, we refer to this as “an external reach for internal well-being”. We need to really truly understand human nature around dependency and addiction . We are dealing with teenagers who are trying to figure out who they are and kindness and compassion are key when we approach kids. Teachers and parents need education around this as we all try to figure out and navigate this challenge of how to self regulate around an incredibly compelling , interesting, complex and possibly for some addictive device.
Hi Trish, it's great to hear to you are taking such an active role in your students usage of technology. Giving your pupils the benefit of the doubt when it comes to using their phones responsible seems like a surefire way to help nurture their personal responsibility.
Laptop carts are usually a mess and not worth the financial burden if their usage is only for simple tasks. Kids coming to school with supercomputers in the pockets saves a lot of money on the side of the school.
What I would be curious about is the resources your school provides in regards to education about AI. Does your institution have measures in place to determine if homework assignments are facilitated or aided by AI? Many universities and colleges are already struggling with keeping it apart from coursework but nonetheless it has persisted and many papers in academia are now being published with passages written by AI. I can only imagine how much harder it may be for high schools to regulate it.
Not scientific, but for the entire month of July I treated my "smart" phone like a land-line phone. The device was plugged into the wall and rested on the fireplace mantle. I didn't use it to listen to podcasts, or YouTube vids, or immediately look at notifications from "X", or hear it ringing as a phone as I worked out in the garden most of the days. In the late afternoon, like checking an old message recording machine from 20 years ago, I'd check for any phone messages and then go about my business. It was bliss and my vitals were all in very healthy ranges. And I was generally content. August 5, I was called back to work and needing to see what union members were asking or reporting on a Facebook page and carrying my "smart" phone around and even just feeling the pinging of notifications from text messages and/or calls has had deleterious effects on my ability to sleep through the night or immediately get back to sleep if I should wake up very early. (To be clear, I don't sleep with the phone. It's not even in the same room with me as I sleep. It's the events of the day, infiltrated by a cell phone and some social media on it that linger in my head.)
My local education union negotiated an MOU with our school district outlining a weak, but at least it's a start, expectation that students' cell/smart phones will be "off and away all day". Response from staff and parents has been mixed, though mostly supportive. I am discouraged that district administration would not take the guidance from Phone-Free Schools Movement more seriously. Wish us success!
Thanks for the article After Babel crew. I look forward to the other installments.
I have read this analysis, the original paper, and also the statistical reanalysis by Matthew Jané. The facts seem very straightforward except one issue - you claim that there are errors in the original dataset. Can you make these claimed errors public? It's a serious claim, and I would like to know exactly what errors you have claimed to find.
I have not checked every study, but I did recalculate a random sample of effect sizes and could not find any errors in Ferguson's table. It is somewhat complex, as he does not specify which effect he considers the "main" effect per se, but I could find an effect size that could reasonably be considered the "main" one for at least Allcot, Hunt, and Lambert's studies (Ferguson appears to have selected the ES that is reported in what the authors of each study consider their primary findings as far as I can tell.)
I like a good weedy discussion, but it's important here not to let the weeds obscure the forest. In effect, social media critics Rausch & Haidt and social-media-critic critic Ferguson agree: social media is a trivial factor in teenage mental health, and quitting social media has only trivial benefits.
Ferguson's 27-study analysis finds a Cohen's d (a statistic of effect size) of +0.08, while Rausch/Haidt cull the most favorable 10 studies and report a d-statistic of +0.20. Applied to the real world, their argument is over whether social media desistance or partial desistance benefits 2% to 8% of subjects above merely random chance. That's like arguing a few points north or south of trivial.
Where Rausch & Haidt, like they & Twenge before, go wrong is in citing medical analogies. We would not refuse to use a drug that proved beneficial for only 2% to 8% of the population. True, but neither would we prescribe that drug to everyone. We don't make everyone take Prozac. We don't take out everyone's appendix. Their medical analogy argues for targeted prescription, not mass bans and curbs. The vast majority of teens tell surveys like Pew Research that social media benefits them in important ways. The small fraction of teens and adults who are harmed by social media can be best addressed individually.
As study designs have improved, longer time periods are assessed, and more factors are considered, social media is found to have smaller, and typically no or beneficial, effects on teens than earlier, more primitive studies suggested. In all cases, the effects on the teenage population are very small in either direction. It's long past time to confront the big factors in teens' depression and anxiety: violent and emotional abuse by parents and adults, parents' mental health and drug/alcohol issues, food insecurity, family financial problems, etc. The obsession with social media is a distraction.
The point laid out here is that the Ferguson meta-analysis did not properly stratify studies by type or duration in addition to including problematic studies as well.
Of course not all social media usage is harmful, but there are definitely ways in which social media is being used as to produce negative outcomes. Rausch and Haidt are promoting restrictions over social media being a net harm to teens. While yes there are things we don't force everyone to do, there are also things we prohibit people from doing. We don't let teenagers vote. We don't let them drive until they're at least 15. There are certainly teens capable of driving when they're 13/14, but we still use a mass ban rather than having individual age minimums.
I would be curious to know where your data for these other factors comes from. By many accounts teen drug use has consistently lowered over the past three decades.
Thanks for your comments. As to the first comment, citing ways in which we restrict teenagers as justification to pile on more restrictions makes no more sense than did denying rights to nonwhite races and women in the past. Policies have to be evaluated on a case-by-case basis, and the two examples you cite – teen driving and voting bans – do not receive favorable long-term evaluations.
I didn’t refer to teenage drug use at all. But on that topic, the 2021 CDC survey https://www.cdc.gov/healthyyouth/data/yrbs/data.htm#nationalyear reports that teens who RARELY or NEVER use social media are many times MORE likely to use heroin, meth, and cocaine, to report more drinking and drug use, and to harm themselves and attempt suicide, than do teens who use social media moderately or heavily. How do Rausch, Haidt, Twenge, Murthy, and those who connect social media use to “negative outcomes” explain that? It sounds like forcing teens to NOT use social media increases their negative outcomes.
What does matter – a lot – are real overdose deaths (55,206 in 2010; 136,928 in 2021) among grownups’ age 25-64 (the ages to be parents, parents’ partners, relatives, and other adults influencing teens), https://wonder.cdc.gov/mcd.html , and hospital emergency cases (far less reliable, but Twenge keeps citing them only for teens) (2.8 million among ages 25-64 in 2010, over 5 million in 2022) https://www.samhsa.gov/data/release/2022-drug-abuse-warning-network-dawn-releases
Drug/alcohol tragedies exploded among parent-aged adults during 2010 to 2021 – exactly when teens became more depressed, and exactly when the CDC reported widespread violent and emotional abuses by parents and household grownups victimizing teens.
I realize After Babel, Haidt’s and Twenge’s books, and Surgeon General websites simply don’t mention these harsh realities – ones they can dodge, but teens have no choice but to face. Let me look at the other comments,
The citation of age-based restrictions was given as evidence that these approaches exist, not necessarily as a justification. I think you can admit that not letting kids vote is not tantamount to apartheid.
I admittedly did think you missed a serial comma and were talking about teen drug alcohol issues. That was my mistake. However, the 2021 YRBS survey measures time spent in front of screens and not social media explicitly. Furthermore, there is no option to state usage as never. When looking at the reported drug uses statistics for people with low screen time usage, you need to recognize that you have culled the vast majority of your dataset, increasing the extent to which the results are affected by measurement error.
That social media usage can lead to negative outcomes does not preclude negative outcomes being reached from other means. When you talk about overdose deaths, you cannot extrapolate the impact on teens simply from a death count. Furthermore the trend of increasing overdose deaths only really started to accelerate vigorously in 2015, which doesn't explain the decrease in teen mental health before hand, nor does it explain the stable period of the 2000s when overdose deaths were still rising monotonically.
Additionally, your evidence of an epidemic of widespread parental abuse from the 2021 ABES does not differentiate between a parent constantly haranguing their kid vs. a single crass remark, nor does it establish an extended pattern of abuse that could somehow trace back to the trend of worsening mental health in the early 2010s.
I appreciate this discussion, but there is a pattern that needs attention. When citing studies claiming to show negative outcomes from social media use, Haidt, Twenge, Rausch, Murthy et al, rest their case on studies and surveys the large majority of which are long out of date (pre-2010), that use dubious definitions and methods, that investigate only social media and still produce very low or mixed effect sizes, and that produce a mere “positive correlation.” Then, when confronted with a mass of counter-research, they either ignore it or turn super-analytical, parsing each nuance, ferreting any complication, and advancing even the most far-fetched analogies based on the tiniest numbers or none at all – they’re the ones who brought up heroin use, for example. They then issue wild claims that “smartphones destroyed a generation” and “social media rewired childhood” and push the most extreme bans.
I wish CDC’s ABES-2021 had asked more precise questions (2023’s appears to), but it remains by far the only usable multi-factorial survey. “Screen time” is a relevant variable, since the same authorities claim TV and video games also damage teenage mental health. Yes, it does differentiate whether parents/adults abuse teens never, rarely, sometimes, most of the time, or always. The definition for parental bullying is more narrowly specific than the broad language used to define cyberbullying (which could include voice telephones) and school bullying (which could include teachers, coaches, etc.), yet Murthy, Haidt, et al, don’t question the numbers for the latter two types.
As for misusing small numbers, look at the huge panic blaming social media for the extremely rare phenomenon of teen (especially vanishingly rare girls’) suicide and self harm when there is not only zero evidence, but actual counter-evidence. The CDC survey has a fascinating result. It does show teens who use screens 5+ hours per day indeed report more sadness, poorer mental health, and more suicide ideation (51%, 37%, 25%) than teens onscreen less than 1 hour a day (37%, 23%, 16%) – but heavy screen users then turn around and report fewer risks: less actual self harm and suicide attempts (3%, 11%) than rarer screen users (5%, 15%). Teens using screens 5+ hours suffer less sleeplessness and lesser increases in drinking and drug use (31%, 15%, 14%) than rarely onscreen teens (37%, 22%, 23%). In a survey with 7,000 participants, these are not small numbers, yet look how rampantly these issues have been distorted.
I don’t know where you’re getting your drug numbers. The CDC Wonder shows that for age 25-64, drug/alcohol overdose deaths soared steadily: 15,325 (2000), 34,861 (2010), 47,328 (2015), 94,376 (2021) and 95,122 (2022). I don’t trust hospital ER data, but annual overdose cases among age 25-64 also skyrocketed into the millions over this period, in turn only the iceberg tip of what teens face among adults around them. Again, these are not small numbers.
Yet, After Babel substackers and Murthy cite the CDC survey and tiny numbers in the few areas where they help their case, then ignore them in the large areas where they refute their case. The preponderance of evidence today (though maybe not in 2010) shows social media is beneficial for teenagers, though not for every individual.
I suggest you go to Mike Males’ substack and read some of his posts. They show how there’s a clear, but often ignored, alternative to the “it’s the phones” hypothesis: that the parents of today’s teens are not alright. His reference to soaring drug abuse is to parent-aged adults, not teens, which is backed up by various studies.
Having now read through some of Mike's work I still remain unconvinced of the youth mental health crisis being owed primarily by bad parenting. While data collection efforts on parental involvement from the side of the child are indeed lacking, that does not mean you can necessarily infer missing data nor assign correlations to said missing data. The data Mike primarily draws upon is from the 2021 ABES, which is limited to that specific cross-section in time. It neither distinguishes chronic from temporary abuse nor does it explain the worsening of youth mental health in the 2010s.
Mike is totally right that some of the issues around parents have been memory holed, the fentanyl crisis receives less press than it did a decade ago, despite having grown into a much bigger issue now. It's not a question as to whether parental well-being can be a confounding variable in the well-being of their children, but this does not exonerate social media from culpability. Even if the current generation of parents are to be blamed, one of the more novel features of Millennials and their upbringing was being the first generation to grow up with social media. Beyond even the work in psychology there a multitude of other studies that focus on how screen usage is modulating brain development.
“It's not a question as to whether parental well-being can be a confounding variable in the well-being of their children, but this does not exonerate social media from culpability.” Agreed. Which is why we need clearly articulated causal models of the data generating process that can be put to the test. Haidt has never presented his theory in a clear, testable way. It’s all vague, narrative-driven, post-hoc hypothesizing. As I write about here:
One thing your DGP model does not acknowledge is that there are often very real feedback loops between causes and effects. Dismissing the link between problematic social media usage and poor mental health as wholly narrative-driven seems reductive in and of itself. While I agree that nuance is often discarded, trivializing results, that does not mean that such a link does not exist.
The other thing your DGP does not consider is maturity. This is a youth-driven trend, and your model does not does not make any distinction towards age.
Just a quick note. Thanks, Craig, for the clarifications.
I do not exonerate social media. When CDC results are analyzed through standard regression, the association between teen depression and social media is 0.44, and screen time is 0.08. That means parental abuse is a moderate correlate and screen time a very small correlate of teens' poor mental health. This is becoming a standard finding -- that heavy social media use is bad for a small fraction of teens and adults while beneficial for the large majority.
But here's the surprise. While parental abuse is associated with vastly more self harm, suicide attempt, drug/alcohol abuse, sleeplessness, violence, etc., among teens, screen use generally is associated with FEWER of these problems. What about screen use helps teens avoid negative outcomes (or is it some other mutual correlate)? Younger teens show virtually identical effects as older teens, although MTF and CDC indicate older teens spend far less time with parents and also report less abuse.
Be careful with "teen brain" studies. the fMRI neuroscannings underlying a lot of past claims have been tossed out as unreliable, while newer, better-designed studies are finding social media and smartphones are not associated with developmental or mental health problems. It's not a zero sum. Teens who socialize well online also tend to socialize well offline.
The point of my post wasn’t to present a model that perfectly captures the DGP. Rather, it was to show (a) how failing to adequately account for confounders can easily lead to spurious effects, and (b) how it’s vital for academics like Haidt to present their theories as explicit causal models so they can be put to the test. It’s a lot easier to continuously shift the goalposts when your theory is shrouded by vague narrative.
Jon and Zach, thank you for providing insight to your research! I wonder, are these studies also investigating the amount of dopamine hits that occur using social media and what that might do to the developing adolescent brain? Could the dramatic increase of use of technology in education in combination with heavy use of social media be to blame as well? In my own research into edtech use in schools, 2012 was also the time that schools started to transition to 1:1 devices which led to an explosion of edtech programs pushed into the education setting. Many of these products were adopted without any research to prove they were better than traditional methods. As a result, kids spend a significant amount of time online during the day which has many negative unintended consequences (i.e. increased emotional outburst, poor social skills, decline in test scores). I believe that this in combination with social media is leading to the increase in mental health but am not aware of any studies that combine these issues, if that even is a possibility.
The one big flaw here is that the results seem to be self reported, or self assessed / subjective based data. At some point, I think it would be worth pursuing some biological data, such as measurement of cortisol and MRI scans. Neuromarketing uses fMRI scans to assess whether products are more or less appealing (refer to work done by Martin Lindstrom who also wrote several books). This very good work that you are doing runs the risk of us arguing over whether social media platforms that have already peaked and are in decline are agents of mental health epidemics or it could possibly be other factors such as decreasing physical fitness, limited exposure to sunshine, decreased in person face to face socialization. You’re doing a great job of threading the needle, but don’t forget that there are some fairly easily measurable biological markers. Hopefully there will be researchers that can bridge the psychological markers with the physiological markers.
Salivary cortisol levels were measured in Vanman 2018, a 1 week FB abstinence experiment, and there was a st. sig. decline. Satisfaction, however, also declined, and *subjective* stress was not affected. Mental health outcomes such as depression or anxiety were not measured.
The authors note:
---
We suspect that the subjective measure of stress used here (the PSS) either may be insensitive to the small changes in stress perceived by participants, or that their perceptions about being cut off from Facebook (e.g., “I’m disconnected!”) in the No Facebook condition influenced their responses on this measure.
I would argue that those factors are all related. Less time spent on social media opens the opportunity for that time to be spent on outdoor exposure, physical activity, etc. Social media is very much a sedentary activity so many of the physiological outcomes are reflective of that.
When it comes to biological data, while I agree that it would be useful for the sake of completeness, the reason why don't see a lot of psychologists using that approach is because many don't have to competency to execute such an approach, having spec'd out of neuroscience at every opportunity or not remembering their education in that field. That being said, there are still many researchers who do investigate screen usage and brain development from this angle.
With a public health concern as serious as this, and the surgeon general is also “on board”, it should be fairly easy to recruit researchers on the biological side. I suggest that Jon Haidt include these concerns during his media appearances and in his written work. This issue is gathering more and more traction, grant money should be easily obtained and the audience for this issue is broad and includes many healthcare professionals.
You should be cautious about the involvement of more powerful institutions. The surgeon general in particular is someone whose motives I would be skeptical of, he has a history of embellishing data to fit his agenda. I've previously documented his lack of integrity on the issue of smoking cigarettes and loneliness, and I would be extremely wary of him wanting to become a leading figure in the discussion on social media.
As I've already shown, there is plenty of biological data, it just exists in a different field than the one Haidt operates. I've already linked John Hutton's excellent work on this topic, but are many others who are already doing the work that you speak of. For sure I would love to see someone with a more concrete focus into brain development be featured as a guest post or join Haidt's team, but I understand why he doesn't make it the focal point of his discussion. Self-evaluations of depression are much easier to interpret than fMRI results for the average person, although if you have the background or the willingness to acquire one, there is a rich and growing body of literature to sink your teeth into.
I’ll agree that the loneliness/cigarette metrics are at best clumsy, at worst a desperate attempt to generate headlines. Quantifying loneliness simply cannot be done. At the same time, I believe health professionals have already incorporated loneliness assessments into their care plans. Jonathan Haidt has a platform, an audience and can reach far more people. I’m quite certain that these fields can complement each other and Jonathan Haidt has been more than willing to collaborate with other professionals. I look forward to hearing more from him and the surgeon general. The 82 page document appears to be written with care and aside from the cigarette comparison the work is sound. Healthcare professionals are not data analysts but they do look holistically at each patient and have intimate first-hand experience that other professions do not. It would be wise to heed his advice and also remember that patient care is a very different career than running numbers and analyzing groups. I’m quite sure he is doing the best he can within the constraints of his training and experience.
He is most senior leader in public health and as such he should be held to the highest standards. We're not talking about a graduate student p-fishing to get results for their supervisor. This is the U.S. Surgeon General saying "above 40%" when the data explicitly says 35%, selectively cherry-picking the most favorable results, citing numbers that can't be traced from the underlying data, as if they sourced from thin air. These behaviors are unbecoming of anyone who works in the field, never mind the head of public health. The mentioned errors most likely could not have occurred naturally.
Murthy has done lots of positive work as I do point out, but nine truths do not excuse a single lie. This cavalier attitude towards deceitful behavior from officials is precisely why there has been such a precipitous decline in trust for public health institutions. It's not as I don't want to be able to trust him, but the more I familiarize myself with his work, the lower my expectations become.
That's not to mention all the baggage he carries around with the political statements he's made and endorsements, which are typically quite unbecoming of a surgeon general. Past surgeon generals like Koop and Carmona had the courage to speak out against their administration when they believed the scientific consensus was at odds with national policy (Elders too, although perhaps she was a bit too candid). Murthy's close ties to this administration make it seem highly doubtful that he would run roughshod of their plans.
Sorry to see this comment diverting so far off topic. We are here to comment on Jonathan Haidt’s work, which is extremely timely and important. The Surgeon General has offered some guidance on the topic that should be reviewed by the public. His report may not be perfect but it is very helpful and presented in a way that is accessible to everyone. Very sorry to hear you are unhappy with his report but it is not appropriate to hijack a thread in order to criticize his job performance. I’m assuming you have not gone through rigorous medical training, residency and practice, we should respect his training and experience.
I get the need for the science behind the assertion that social media creates anxiety and depression. But smartphones do so much more damage than that. I have run my own experiments with my 5 children and it was stunning to see the difference in their behavior. The first day was awful but by day 2 they started to change and by day 3 they were back to my amazing children. We all talked about the huge difference. This was in 2018. Every person can use their critical thinking skills to make rational decisions about what they experience in their life. You don’t need an expert to tell you. Just as both Jon and Abigail Shrier have written-we have acquiesced our decision making to someone else. We as parents need to take that back. You really do know what is right for your kids.
Hi, I think the point that you might be missing is that not every child has parents as involved in their lives as you are. There are many parents who don't have the time or the means to be as involved in their child's development (or some are uninterested entirely) who give phones to their kids without making the same considerations that you do. Changing the landscape of social media would help them in ways parental figures in their lives are currently not.
I completely agree that it is not just big, bad social media, it is the phones themselves. My daughter (almost 17) can be on her phone 5 hours a day (on a school day!) texting and FaceTiming, constantly checking, constantly waiting for something to happen on her phone.
May I ask, for how long did you take their phones away? And do you mean you physically took them away, or you allowed them a phone but took certain apps off? Thanks!
Physically took their phones away. We also implement phone free days and phone free standards ie when I am driving them around-no phone use, we talk, sit in silence or listen to music.
For how long do you take the phones away? Did you take them away randomly, or as a consequence for bad behavior/choices?
We have phone-free Sundays (or at least try).
Sorry for all the questions:) Desperately trying to navigate this when it almost seems too late!
Go through this article and substitute “social media” with the word “cocaine.” Over and over, we talk about the potential benefits of brief and partial reduction of “social media” use, ignoring the fact that in every imaginable way, social media use is an addiction. So is cocaine use. With that in mind, of course a reduction in either behavior or substance will lead to some improvement in mental health—or none. But we’re missing the point entirely. No cocaine recovery center strives for a brief or partial reduction in use. None. And no parent would strive for a brief or partial reduction in cocaine use at home. And yet here we are talking about brief and partial reduction of social media use because parents lack the courage and information required to eliminate all phone use before the age of 18. The kids simply don’t need the risk of phones, which are destroying the real joy of life for children and parents, as Jon has said.
But here’s the real point. Simply stopping an addiction is never enough, as every recovering addict knows. You must replace the addictive behavior with something constructive. What children need most is unconditional love—the kind without the stain of disappointment and anger. But parents don’t know how to give that. They never got it themselves. Controlling children or enabling them is not loving them. And parents don’t have to figure out unconditional love and guidance on their own. Just go to the free and agenda-free websites RealLove.com and RealLoveParents.com. I have nothing to sell. But I do offer thirty—30—years of intense experience with teaching parents and children all over the world. It’s love they need, not social media or phones or indulgence or entertainment.
Agreed till we got to the point about controlling children. Of course controlling children is part of loving children.
That's the parents job to control their children as they learn the many skills needed to be responsible adults.
If you don't control your kids and let them just do whatever they want you are asking for trouble
I would personally argue that cocaine makes you far more productive than social media, but that's a whole other aside. I do think that you are spot on that excessive social media use being treated as addictive is the correct way to go. However, I would not as far as to equate it with cocaine. Certainly there is no need to restrict social media usage just to medicinal use. The other issue is that unlike cocaine, the makeup of social media is far more malleable and can change form. Is it not better to find a healthier form of social media than to ban it entirely?
Informative
This must be Collis 2022, and the problem there was that the 'reduction' group actually ended up increasing its digital screen time:
===
We find that participants in the treatment group substitute social media for instant messaging and do not decrease their total time spent on digital devices.
[...]
Remarkably, although students in the treatment group significantly reduced their social media activities, their overall digital activities overall are not affected but, in fact, exceed those of the control group in block 2 (t-test, p = 0.026). This result indicates that students substituted or even overcompensated their social media usage with other activities.
===
In fact it seems SM time did not decline at all, as students just moved not only to WhatsApp but also to YouTube and TikTok (this is seen only in Supplement data).
Note also that Collis 2022 did not measure MH outcomes like depression or anxiety -- it measured outcomes like 'satisfaction with life'.
Of course, studies are incredibly beneficial and, in some cases essential, for decision-making. However, deciding if social media is problematic is not one of those. Talk to any parent or teacher, and it's probably one of the few things we all happen to agree upon. You can show me all the data in the world to try to convince me otherwise, but it's incongruous with real life experiences and what parents witness daily with their own two eyes.
Even scientists say it's not always wise to "wait for the science." Look where that got us with smoking. It wasn't until reviewing 7,000 articles in the medical literature that the Surgeon General concluded that smoking caused lung cancer and bronchitis in 1964 and then decades more before any meaningful policy and change occurred. We don't have that much time and we currently have it all backward. I think Jon has alluded to this in a previous post, and I agree. Instead of waiting for beyond a reasonable doubt that it's harmful to children, we should be waiting for beyond a reasonable doubt evidence that it's helpful. That it contributes to their thriving and flourishing. Especially with children where we must be more careful and prudent before experimenting on their one precious childhood.
I think your approach to this issue is far too reductive. The idea that one does not need a sufficient threshold for evidence to act with evidence belies the fact that such action would be unhelpful if there is not enough evidence to inform which actions should be taken. Your point towards smoking as an example doesn't undermine the desire for scientific consensus since industry lobbying played a large role in delaying regulations. There are plenty of instances where the opposite was true and more testing was required, take thalidomide for instance.
However, in this case I do believe that there is a concrete link between certain mechanistic uses of social media and negative health outcomes. That being said, that does not mean that all features of social media are responsible. It's a complex and multi-faceted topic with many variables to consider. And while yes, there is a sense of urgency behind this
campaign, trepidation is essential. A poorly constructed response to an issue can make matters far worse if care is not taken.
Appreciate your comment, John. That's my brain - start with the simple. You're correct that a poorly constructed response to an issue can make it worse. I tend to see things on a risk continuum. And I see much less risk to children by taking the "let's just wait" approach than running recklessly ahead or running ahead only partially informed. That might be an overly simplistic view, but that's just how my brain tends to work when it comes to children.
Yeah, I mean we often speak in generalities for the sake of brevity, but too often we forget that these generalities are underscored by several layers of often complex factors. This is a fault committed even by many of the researchers in this field, who speak about broad topics like social media and screen usage without considering that outcomes from these activities are highly mechanistic.
You are right in that we definitely do not need to wait for cross-sectional studies of fully mature adults before we start debating that action should be taken. The difficulty in constructing a plan of action is in creating regulations/reforms that service the needs of the people against the wants of government, corporations, and special interests. That's why I am particularly wary of support from certain institutions because yes, there definitely is a world where this goes astray and we end up with awful legislature like the Gramm-Leach-Bliley Act or the Patriot Act.
Hi Dr. Haidt, I wrote a blog post critique of this re-analysis. You can view it here: https://matthewbjane.com/blog-posts/blog-post-6.html
Hi Matthew, thank you for taking the time to engage with the meta-analysis and providing your own statistical analysis.
I have a couple critiques of your interrogation. The first is in defending the choice Rausch and Haidt made in sub setting studies by duration. They cite Anna Lembke's work on dopamine addiction as noting that withdrawal symptoms may last for up to two weeks. While the withdrawal hypothesis should be more concretely tested, the decision to stratify by study duration is not entirely arbitrary. Furthermore, any decision to remove studies from consideration such as the Collis and Eggers 2022 is being made on the basis of impropriety in the methodology/experimental design, to which there are many flaws in the case of Collis and Eggers as well as many others, even studies with a positive d.
The second critique is in your approach towards building a regression model. While your coding seems sound, the decision to construct the regression using only the overall effect sizes of each study seems inappropriate. Many of the studies done over a longer period of time were longitudinal, so their effect size is not static. Your regression does not capture the time evolution of effect sizes for some of these extended studies.
I will post some sort of penalized spline regression tomorrow. Dichotomizing into groups is simply not an optimal modeling approach even if you have some vaguely justified cut-off.
I used the effect sizes that Rausch and Haidt used, I will not be constructing a whole new meta-analytic database for this. I’d be happy to analyze it if someone else does though.
A lot of the smaller studies are populated with noise due to ability for natural variance in daily life to obscure any signal. Would you really weight a day's abstinence from alcohol the same as a month's break? If so, then how about a hours? Or just 60 minutes? Surely you would agree that the length of a time series influences its utility.
Also, including Collis and Eggers 2022, a study that did not holistically reduce social media usage, in a model for social media reduction studies seems highly inappropriate.
I’m not sure what this means: “A lot of the smaller studies are populated with noise due to ability for natural variance in daily life to obscure any signal.” If you are referring to sampling error, a random effects model is weighting by sampling variance. If you are referring to weighting by time, I am not going to weight the meta-regression by the independent variable.
I’m using the dataset that Rausch and Haidt used. I am not conducting a new meta-analysis here, I am critiquing Haidt’s re-analysis. If you would like to use a new selection criterion for a meta-analysis then I encourage you to do that. Ultimately, if you don’t have a well-defined and coherent inclusion/exclusion criterion then you are cherry picking studies. I’m sure you could find post-hoc justifications to exclude any one of these studies. But again, this is well beyond the point of my critique.
There are a variety of day-to-day factors that modulate your well-being (mood, sleep, stress, etc.) and performing a day's abstinence of social media without controlling for changes in the plethora of other moderators sets you up for having both high sample variance and sampling variance.
What Rausch and Haidt are arguing is that the Ferguson meta-analysis suffers from a poor inclusion/exclusion criterion. It is not post-hoc to say that a study that does not reduce social media fails to meet the inclusion criteria of having measured social media reduction.
A whooping -0,138 in green color for 10 weeks.Needs investigating
Good point Christos. We will discuss this study in more detail in the next post (in brief, we'll show that digital screen time actually increased in the social media reduction group).
Hi Zach, I was wondering if you would take this approach with the other studies in the Ferguson meta-analysis, even including those who reported a positive effect size.
The problem with meta-analyses is that not all studies are created equal, and that milage varies greatly with methodology. Take for instance the Faulhaber et al. 2023 study.
https://tmb.apaopen.org/pub/yvcb5y06/release/2
Despite finding significant changes between the treatment and control group, the treatment group limited social media usage through self-monitoring which is problematic for many reasons: truthfulness about social media usage, reliability in estimating social media usage, etc.
Yes, some of the studies with a negative effect size are methodology flaws, but so do some of the studies that report positive effect sizes. What steps you are taking to balance this out?
Hi,
These are good points. I agree that there are a number of issues across a variety of studies in the meta, on "both sides." Our focus in this post is primarily to show that there were clear moderators (e.g., duration) that should have been taken into account when conducting the meta.
In the next post, in addition to calculation errors/other moderators, we are primarily concerned about which studies violate the inclusion criteria that was laid out within the meta (and why). This Is why we will discuss Collis in more detail, as there are a number of issues that directly pertain to inclusion. We don't cover all of the studies, just the ones that relate to that specific point.
Independent of our response to this meta-analysis, we are working on a broader review of the experimental studies, where we plan to address some of your points.
Yes, I fully agree that duration is a key modulator as many of the short-term studies seem to be capturing all the noise and no signal. The 2021 Przybylski study was especially egregious and useless. Trying to extrapolate from one-day's abstinence to make claims about how social media generally affects well-being is nigh malpractice.
In your broader review, what demographic factors will you be trying to control for? I know sex is an obvious target, but I was wondering if you are considering other factors like political identification or gender/sexual identity as moderators.
A big issue with the Collis & Eggers study is that it only limited access to Facebook, Instagram, and Snapchat, of which they noted students just substituted those services with other social media applications.
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0272416#sec009'
The other main issue is that there was no attempt made to disentangle the well-being effects of social media from the general affects of college life (i.e. midterms and exams) so it is unclear as to how much social media usage was responsible for well-being.
This was very informative, not just on the details of the studies of the meta-analysis in general. (in the interest of promoting screen-minimalism in myself I actually printed it out so I could have a better experience reading it). I'm looking forward to reading the rest of the series.
I'm curious if you're planning to reach out to Chris Ferguson to give him the opportunity to respond to your analysis of his work. I really appreciate how you take a collaborative approach to this in providing people with competing views to share their perspectives alongside your own. I would be interested to hear his response if he took you up on the opportunity to do so.
I’m a Canadian high school educator of 35 years (French& Psych/ Masters in counseling and addiction) who has been teaching and learning about addiction and cell phone dependency and its effects on learning for 10 years. As I head back to the classroom next week, our provincial government in British Columbia has mandated a ban on cell phones, except when the teacher deems it necessary for learning. Teaching grade 11 and 12, I absolutely have to provide access to the Internet. I have a responsibility to teach kids about how to responsibly an ethically navigate AI ,how to interpret research, discern good research from poor research, write their own research papers, and understand peer reviewed studies, to name a few. However, as teachers, we desperately need help and strategies for navigating this incredible challenge. How do we prepare kids for 21st century learning without requiring them to have access to their addictive $1200 smart phones? One answer, of course, is mobile laptop carts in the classroom, which requires funding, which we don’t have.
An added challenge is that most teachers and parents don’t have the education around the importance of compassion and the danger of shaming when talking about cell phone with students. Anybody who’s done any work knows Gabor Mate’s work “ the opposite of addiction is connection”. When parents and teachers shame kids for using their phones ,even when they’ve been repeatedly told they can’t , all we do is drive them deeper into their dependency and increase their isolation from us. This is a huge challenge for teachers navigating cellphones in the classroom. We have to constantly be sure that we’re being respectful and kind when kids pull out their phones and they’re not supposed to. (If you’re not an educator, just think about what that’s like when you see 200 students a day and a large percentage of them have a dependency on their phones and do not realize it at all) . What I do with my students , whenever I see them with their phone out, even though we have a policy for keeping it away unless I ask them to pull it out, ; I quietly ask them “ Do you need to take that ?” and if they do, I ask them to step outside and deal with their issue ( there is no shaming and no scorn in my voice ever! This is super important) . Most of the time when I ask a student if they need to take their call or text, they just look up at me (like they’re coming out of a fog ) and apologize and say “I didn’t realize, sorry Ms W. “. So many of our kids reach into their bags and grab their phones,whenever their minds wander and they’re a little bit bored. In the addiction research, we refer to this as “an external reach for internal well-being”. We need to really truly understand human nature around dependency and addiction . We are dealing with teenagers who are trying to figure out who they are and kindness and compassion are key when we approach kids. Teachers and parents need education around this as we all try to figure out and navigate this challenge of how to self regulate around an incredibly compelling , interesting, complex and possibly for some addictive device.
Hi Trish, it's great to hear to you are taking such an active role in your students usage of technology. Giving your pupils the benefit of the doubt when it comes to using their phones responsible seems like a surefire way to help nurture their personal responsibility.
Laptop carts are usually a mess and not worth the financial burden if their usage is only for simple tasks. Kids coming to school with supercomputers in the pockets saves a lot of money on the side of the school.
What I would be curious about is the resources your school provides in regards to education about AI. Does your institution have measures in place to determine if homework assignments are facilitated or aided by AI? Many universities and colleges are already struggling with keeping it apart from coursework but nonetheless it has persisted and many papers in academia are now being published with passages written by AI. I can only imagine how much harder it may be for high schools to regulate it.
Not scientific, but for the entire month of July I treated my "smart" phone like a land-line phone. The device was plugged into the wall and rested on the fireplace mantle. I didn't use it to listen to podcasts, or YouTube vids, or immediately look at notifications from "X", or hear it ringing as a phone as I worked out in the garden most of the days. In the late afternoon, like checking an old message recording machine from 20 years ago, I'd check for any phone messages and then go about my business. It was bliss and my vitals were all in very healthy ranges. And I was generally content. August 5, I was called back to work and needing to see what union members were asking or reporting on a Facebook page and carrying my "smart" phone around and even just feeling the pinging of notifications from text messages and/or calls has had deleterious effects on my ability to sleep through the night or immediately get back to sleep if I should wake up very early. (To be clear, I don't sleep with the phone. It's not even in the same room with me as I sleep. It's the events of the day, infiltrated by a cell phone and some social media on it that linger in my head.)
My local education union negotiated an MOU with our school district outlining a weak, but at least it's a start, expectation that students' cell/smart phones will be "off and away all day". Response from staff and parents has been mixed, though mostly supportive. I am discouraged that district administration would not take the guidance from Phone-Free Schools Movement more seriously. Wish us success!
Thanks for the article After Babel crew. I look forward to the other installments.
I have read this analysis, the original paper, and also the statistical reanalysis by Matthew Jané. The facts seem very straightforward except one issue - you claim that there are errors in the original dataset. Can you make these claimed errors public? It's a serious claim, and I would like to know exactly what errors you have claimed to find.
I have not checked every study, but I did recalculate a random sample of effect sizes and could not find any errors in Ferguson's table. It is somewhat complex, as he does not specify which effect he considers the "main" effect per se, but I could find an effect size that could reasonably be considered the "main" one for at least Allcot, Hunt, and Lambert's studies (Ferguson appears to have selected the ES that is reported in what the authors of each study consider their primary findings as far as I can tell.)
Re: Church attendance improving anxiety, depression, anger, etc
Has anyone differentiated between:
1
I need Jesus
2
Haidt says I should "come to Jesus" and I'll "give in to my gullible self"... aka "consciously feeding the elephant"...?
3
Same as #2 but MY DARK SENSE OF HUMOR FINDS 99% OF HUMAN FLAWS TO BE HUMOROUS
I like a good weedy discussion, but it's important here not to let the weeds obscure the forest. In effect, social media critics Rausch & Haidt and social-media-critic critic Ferguson agree: social media is a trivial factor in teenage mental health, and quitting social media has only trivial benefits.
Ferguson's 27-study analysis finds a Cohen's d (a statistic of effect size) of +0.08, while Rausch/Haidt cull the most favorable 10 studies and report a d-statistic of +0.20. Applied to the real world, their argument is over whether social media desistance or partial desistance benefits 2% to 8% of subjects above merely random chance. That's like arguing a few points north or south of trivial.
Where Rausch & Haidt, like they & Twenge before, go wrong is in citing medical analogies. We would not refuse to use a drug that proved beneficial for only 2% to 8% of the population. True, but neither would we prescribe that drug to everyone. We don't make everyone take Prozac. We don't take out everyone's appendix. Their medical analogy argues for targeted prescription, not mass bans and curbs. The vast majority of teens tell surveys like Pew Research that social media benefits them in important ways. The small fraction of teens and adults who are harmed by social media can be best addressed individually.
As study designs have improved, longer time periods are assessed, and more factors are considered, social media is found to have smaller, and typically no or beneficial, effects on teens than earlier, more primitive studies suggested. In all cases, the effects on the teenage population are very small in either direction. It's long past time to confront the big factors in teens' depression and anxiety: violent and emotional abuse by parents and adults, parents' mental health and drug/alcohol issues, food insecurity, family financial problems, etc. The obsession with social media is a distraction.
The point laid out here is that the Ferguson meta-analysis did not properly stratify studies by type or duration in addition to including problematic studies as well.
Of course not all social media usage is harmful, but there are definitely ways in which social media is being used as to produce negative outcomes. Rausch and Haidt are promoting restrictions over social media being a net harm to teens. While yes there are things we don't force everyone to do, there are also things we prohibit people from doing. We don't let teenagers vote. We don't let them drive until they're at least 15. There are certainly teens capable of driving when they're 13/14, but we still use a mass ban rather than having individual age minimums.
I would be curious to know where your data for these other factors comes from. By many accounts teen drug use has consistently lowered over the past three decades.
https://yrbs-explorer.services.cdc.gov/#/graphs?questionCode=H41&topicCode=C03&location=XX&year=2021
Thanks for your comments. As to the first comment, citing ways in which we restrict teenagers as justification to pile on more restrictions makes no more sense than did denying rights to nonwhite races and women in the past. Policies have to be evaluated on a case-by-case basis, and the two examples you cite – teen driving and voting bans – do not receive favorable long-term evaluations.
I didn’t refer to teenage drug use at all. But on that topic, the 2021 CDC survey https://www.cdc.gov/healthyyouth/data/yrbs/data.htm#nationalyear reports that teens who RARELY or NEVER use social media are many times MORE likely to use heroin, meth, and cocaine, to report more drinking and drug use, and to harm themselves and attempt suicide, than do teens who use social media moderately or heavily. How do Rausch, Haidt, Twenge, Murthy, and those who connect social media use to “negative outcomes” explain that? It sounds like forcing teens to NOT use social media increases their negative outcomes.
What does matter – a lot – are real overdose deaths (55,206 in 2010; 136,928 in 2021) among grownups’ age 25-64 (the ages to be parents, parents’ partners, relatives, and other adults influencing teens), https://wonder.cdc.gov/mcd.html , and hospital emergency cases (far less reliable, but Twenge keeps citing them only for teens) (2.8 million among ages 25-64 in 2010, over 5 million in 2022) https://www.samhsa.gov/data/release/2022-drug-abuse-warning-network-dawn-releases
Drug/alcohol tragedies exploded among parent-aged adults during 2010 to 2021 – exactly when teens became more depressed, and exactly when the CDC reported widespread violent and emotional abuses by parents and household grownups victimizing teens.
I realize After Babel, Haidt’s and Twenge’s books, and Surgeon General websites simply don’t mention these harsh realities – ones they can dodge, but teens have no choice but to face. Let me look at the other comments,
The citation of age-based restrictions was given as evidence that these approaches exist, not necessarily as a justification. I think you can admit that not letting kids vote is not tantamount to apartheid.
I admittedly did think you missed a serial comma and were talking about teen drug alcohol issues. That was my mistake. However, the 2021 YRBS survey measures time spent in front of screens and not social media explicitly. Furthermore, there is no option to state usage as never. When looking at the reported drug uses statistics for people with low screen time usage, you need to recognize that you have culled the vast majority of your dataset, increasing the extent to which the results are affected by measurement error.
https://cces.gov.harvard.edu/news/perils-cherry-picking-low-frequency-events-large-sample-surveys
That social media usage can lead to negative outcomes does not preclude negative outcomes being reached from other means. When you talk about overdose deaths, you cannot extrapolate the impact on teens simply from a death count. Furthermore the trend of increasing overdose deaths only really started to accelerate vigorously in 2015, which doesn't explain the decrease in teen mental health before hand, nor does it explain the stable period of the 2000s when overdose deaths were still rising monotonically.
https://nida.nih.gov/research-topics/trends-statistics/overdose-death-rates#Fig1
Additionally, your evidence of an epidemic of widespread parental abuse from the 2021 ABES does not differentiate between a parent constantly haranguing their kid vs. a single crass remark, nor does it establish an extended pattern of abuse that could somehow trace back to the trend of worsening mental health in the early 2010s.
I appreciate this discussion, but there is a pattern that needs attention. When citing studies claiming to show negative outcomes from social media use, Haidt, Twenge, Rausch, Murthy et al, rest their case on studies and surveys the large majority of which are long out of date (pre-2010), that use dubious definitions and methods, that investigate only social media and still produce very low or mixed effect sizes, and that produce a mere “positive correlation.” Then, when confronted with a mass of counter-research, they either ignore it or turn super-analytical, parsing each nuance, ferreting any complication, and advancing even the most far-fetched analogies based on the tiniest numbers or none at all – they’re the ones who brought up heroin use, for example. They then issue wild claims that “smartphones destroyed a generation” and “social media rewired childhood” and push the most extreme bans.
I wish CDC’s ABES-2021 had asked more precise questions (2023’s appears to), but it remains by far the only usable multi-factorial survey. “Screen time” is a relevant variable, since the same authorities claim TV and video games also damage teenage mental health. Yes, it does differentiate whether parents/adults abuse teens never, rarely, sometimes, most of the time, or always. The definition for parental bullying is more narrowly specific than the broad language used to define cyberbullying (which could include voice telephones) and school bullying (which could include teachers, coaches, etc.), yet Murthy, Haidt, et al, don’t question the numbers for the latter two types.
As for misusing small numbers, look at the huge panic blaming social media for the extremely rare phenomenon of teen (especially vanishingly rare girls’) suicide and self harm when there is not only zero evidence, but actual counter-evidence. The CDC survey has a fascinating result. It does show teens who use screens 5+ hours per day indeed report more sadness, poorer mental health, and more suicide ideation (51%, 37%, 25%) than teens onscreen less than 1 hour a day (37%, 23%, 16%) – but heavy screen users then turn around and report fewer risks: less actual self harm and suicide attempts (3%, 11%) than rarer screen users (5%, 15%). Teens using screens 5+ hours suffer less sleeplessness and lesser increases in drinking and drug use (31%, 15%, 14%) than rarely onscreen teens (37%, 22%, 23%). In a survey with 7,000 participants, these are not small numbers, yet look how rampantly these issues have been distorted.
I don’t know where you’re getting your drug numbers. The CDC Wonder shows that for age 25-64, drug/alcohol overdose deaths soared steadily: 15,325 (2000), 34,861 (2010), 47,328 (2015), 94,376 (2021) and 95,122 (2022). I don’t trust hospital ER data, but annual overdose cases among age 25-64 also skyrocketed into the millions over this period, in turn only the iceberg tip of what teens face among adults around them. Again, these are not small numbers.
Yet, After Babel substackers and Murthy cite the CDC survey and tiny numbers in the few areas where they help their case, then ignore them in the large areas where they refute their case. The preponderance of evidence today (though maybe not in 2010) shows social media is beneficial for teenagers, though not for every individual.
I suggest you go to Mike Males’ substack and read some of his posts. They show how there’s a clear, but often ignored, alternative to the “it’s the phones” hypothesis: that the parents of today’s teens are not alright. His reference to soaring drug abuse is to parent-aged adults, not teens, which is backed up by various studies.
Having now read through some of Mike's work I still remain unconvinced of the youth mental health crisis being owed primarily by bad parenting. While data collection efforts on parental involvement from the side of the child are indeed lacking, that does not mean you can necessarily infer missing data nor assign correlations to said missing data. The data Mike primarily draws upon is from the 2021 ABES, which is limited to that specific cross-section in time. It neither distinguishes chronic from temporary abuse nor does it explain the worsening of youth mental health in the 2010s.
Mike is totally right that some of the issues around parents have been memory holed, the fentanyl crisis receives less press than it did a decade ago, despite having grown into a much bigger issue now. It's not a question as to whether parental well-being can be a confounding variable in the well-being of their children, but this does not exonerate social media from culpability. Even if the current generation of parents are to be blamed, one of the more novel features of Millennials and their upbringing was being the first generation to grow up with social media. Beyond even the work in psychology there a multitude of other studies that focus on how screen usage is modulating brain development.
“It's not a question as to whether parental well-being can be a confounding variable in the well-being of their children, but this does not exonerate social media from culpability.” Agreed. Which is why we need clearly articulated causal models of the data generating process that can be put to the test. Haidt has never presented his theory in a clear, testable way. It’s all vague, narrative-driven, post-hoc hypothesizing. As I write about here:
https://open.substack.com/pub/craigsewall/p/haidts-mirage?r=1jg7sp&utm_medium=ios
One thing your DGP model does not acknowledge is that there are often very real feedback loops between causes and effects. Dismissing the link between problematic social media usage and poor mental health as wholly narrative-driven seems reductive in and of itself. While I agree that nuance is often discarded, trivializing results, that does not mean that such a link does not exist.
The other thing your DGP does not consider is maturity. This is a youth-driven trend, and your model does not does not make any distinction towards age.
Just a quick note. Thanks, Craig, for the clarifications.
I do not exonerate social media. When CDC results are analyzed through standard regression, the association between teen depression and social media is 0.44, and screen time is 0.08. That means parental abuse is a moderate correlate and screen time a very small correlate of teens' poor mental health. This is becoming a standard finding -- that heavy social media use is bad for a small fraction of teens and adults while beneficial for the large majority.
But here's the surprise. While parental abuse is associated with vastly more self harm, suicide attempt, drug/alcohol abuse, sleeplessness, violence, etc., among teens, screen use generally is associated with FEWER of these problems. What about screen use helps teens avoid negative outcomes (or is it some other mutual correlate)? Younger teens show virtually identical effects as older teens, although MTF and CDC indicate older teens spend far less time with parents and also report less abuse.
Be careful with "teen brain" studies. the fMRI neuroscannings underlying a lot of past claims have been tossed out as unreliable, while newer, better-designed studies are finding social media and smartphones are not associated with developmental or mental health problems. It's not a zero sum. Teens who socialize well online also tend to socialize well offline.
The point of my post wasn’t to present a model that perfectly captures the DGP. Rather, it was to show (a) how failing to adequately account for confounders can easily lead to spurious effects, and (b) how it’s vital for academics like Haidt to present their theories as explicit causal models so they can be put to the test. It’s a lot easier to continuously shift the goalposts when your theory is shrouded by vague narrative.
Jon and Zach, thank you for providing insight to your research! I wonder, are these studies also investigating the amount of dopamine hits that occur using social media and what that might do to the developing adolescent brain? Could the dramatic increase of use of technology in education in combination with heavy use of social media be to blame as well? In my own research into edtech use in schools, 2012 was also the time that schools started to transition to 1:1 devices which led to an explosion of edtech programs pushed into the education setting. Many of these products were adopted without any research to prove they were better than traditional methods. As a result, kids spend a significant amount of time online during the day which has many negative unintended consequences (i.e. increased emotional outburst, poor social skills, decline in test scores). I believe that this in combination with social media is leading to the increase in mental health but am not aware of any studies that combine these issues, if that even is a possibility.
The one big flaw here is that the results seem to be self reported, or self assessed / subjective based data. At some point, I think it would be worth pursuing some biological data, such as measurement of cortisol and MRI scans. Neuromarketing uses fMRI scans to assess whether products are more or less appealing (refer to work done by Martin Lindstrom who also wrote several books). This very good work that you are doing runs the risk of us arguing over whether social media platforms that have already peaked and are in decline are agents of mental health epidemics or it could possibly be other factors such as decreasing physical fitness, limited exposure to sunshine, decreased in person face to face socialization. You’re doing a great job of threading the needle, but don’t forget that there are some fairly easily measurable biological markers. Hopefully there will be researchers that can bridge the psychological markers with the physiological markers.
Salivary cortisol levels were measured in Vanman 2018, a 1 week FB abstinence experiment, and there was a st. sig. decline. Satisfaction, however, also declined, and *subjective* stress was not affected. Mental health outcomes such as depression or anxiety were not measured.
The authors note:
---
We suspect that the subjective measure of stress used here (the PSS) either may be insensitive to the small changes in stress perceived by participants, or that their perceptions about being cut off from Facebook (e.g., “I’m disconnected!”) in the No Facebook condition influenced their responses on this measure.
---
I would argue that those factors are all related. Less time spent on social media opens the opportunity for that time to be spent on outdoor exposure, physical activity, etc. Social media is very much a sedentary activity so many of the physiological outcomes are reflective of that.
When it comes to biological data, while I agree that it would be useful for the sake of completeness, the reason why don't see a lot of psychologists using that approach is because many don't have to competency to execute such an approach, having spec'd out of neuroscience at every opportunity or not remembering their education in that field. That being said, there are still many researchers who do investigate screen usage and brain development from this angle.
https://jamanetwork.com/journals/jamapediatrics/fullarticle/2754101
With a public health concern as serious as this, and the surgeon general is also “on board”, it should be fairly easy to recruit researchers on the biological side. I suggest that Jon Haidt include these concerns during his media appearances and in his written work. This issue is gathering more and more traction, grant money should be easily obtained and the audience for this issue is broad and includes many healthcare professionals.
You should be cautious about the involvement of more powerful institutions. The surgeon general in particular is someone whose motives I would be skeptical of, he has a history of embellishing data to fit his agenda. I've previously documented his lack of integrity on the issue of smoking cigarettes and loneliness, and I would be extremely wary of him wanting to become a leading figure in the discussion on social media.
https://whoisjohngalt.substack.com/p/loneliness-and-the-apocryphal-fifteen
As I've already shown, there is plenty of biological data, it just exists in a different field than the one Haidt operates. I've already linked John Hutton's excellent work on this topic, but are many others who are already doing the work that you speak of. For sure I would love to see someone with a more concrete focus into brain development be featured as a guest post or join Haidt's team, but I understand why he doesn't make it the focal point of his discussion. Self-evaluations of depression are much easier to interpret than fMRI results for the average person, although if you have the background or the willingness to acquire one, there is a rich and growing body of literature to sink your teeth into.
https://jamanetwork.com/journals/jamapediatrics/article-abstract/2799812
I’ll agree that the loneliness/cigarette metrics are at best clumsy, at worst a desperate attempt to generate headlines. Quantifying loneliness simply cannot be done. At the same time, I believe health professionals have already incorporated loneliness assessments into their care plans. Jonathan Haidt has a platform, an audience and can reach far more people. I’m quite certain that these fields can complement each other and Jonathan Haidt has been more than willing to collaborate with other professionals. I look forward to hearing more from him and the surgeon general. The 82 page document appears to be written with care and aside from the cigarette comparison the work is sound. Healthcare professionals are not data analysts but they do look holistically at each patient and have intimate first-hand experience that other professions do not. It would be wise to heed his advice and also remember that patient care is a very different career than running numbers and analyzing groups. I’m quite sure he is doing the best he can within the constraints of his training and experience.
He is most senior leader in public health and as such he should be held to the highest standards. We're not talking about a graduate student p-fishing to get results for their supervisor. This is the U.S. Surgeon General saying "above 40%" when the data explicitly says 35%, selectively cherry-picking the most favorable results, citing numbers that can't be traced from the underlying data, as if they sourced from thin air. These behaviors are unbecoming of anyone who works in the field, never mind the head of public health. The mentioned errors most likely could not have occurred naturally.
Murthy has done lots of positive work as I do point out, but nine truths do not excuse a single lie. This cavalier attitude towards deceitful behavior from officials is precisely why there has been such a precipitous decline in trust for public health institutions. It's not as I don't want to be able to trust him, but the more I familiarize myself with his work, the lower my expectations become.
That's not to mention all the baggage he carries around with the political statements he's made and endorsements, which are typically quite unbecoming of a surgeon general. Past surgeon generals like Koop and Carmona had the courage to speak out against their administration when they believed the scientific consensus was at odds with national policy (Elders too, although perhaps she was a bit too candid). Murthy's close ties to this administration make it seem highly doubtful that he would run roughshod of their plans.
Sorry to see this comment diverting so far off topic. We are here to comment on Jonathan Haidt’s work, which is extremely timely and important. The Surgeon General has offered some guidance on the topic that should be reviewed by the public. His report may not be perfect but it is very helpful and presented in a way that is accessible to everyone. Very sorry to hear you are unhappy with his report but it is not appropriate to hijack a thread in order to criticize his job performance. I’m assuming you have not gone through rigorous medical training, residency and practice, we should respect his training and experience.