66 Comments
User's avatar
Jud Heugel MD's avatar

As a pediatrician working primarily in health technology startups and also continuing to care for families, I really appreciated this article and agreed with so many of the sentiments, especially the call for AI companions to be created in an evidence-based, clinically-informed manner, rather than a social platform play. However, one fundamental aspect was missing in the intro from this important piece of writing: there are simply not enough professionals to have the human conversations required to tend to the mental health of our children, many in crisis or borderline crisis.

Whenever I'm in clinic and refer a patient for counseling (that is pretty much every single day for the current general pediatrician or PCP who cares for kids full time), I do it with a sense of slight shame, knowing that, realistically, that child will not talk with a professional for many weeks to months, and that's only if they have the financial means. The system for mental health care of our children is broken, and there is space for real growth to help meet the needs of our kids in an age appropriate, evidenced-based, technology-driven and genuinely caring way. Thank you so much for writing this.

Expand full comment
Mandy McLean's avatar

Thank you for this powerful reflection and for the work you’re doing on both fronts, Jud. You’re absolutely right that access is one of the biggest challenges we face in mental health care. We focused this piece on the popular AI companions already in use, which are not designed by mental health professionals and are driven by incentives to maximize engagement rather than well-being.

That said, I completely agree there are meaningful opportunities to use AI intentionally in the mental health space. It can be a force for good and a way to increase access and equity. We just need to move carefully, especially when it comes to anything that touches our kids.

Expand full comment
Jud Heugel MD's avatar

Incredibly thoughtful reply. I subscribed to your substack and look forward to learning more from you and leading together in this nebulous and often scary space, one that holds a ton of promise. Thx again.

Expand full comment
Poul Eriksson's avatar

There may be opportunities to use AI intentionally in the mental health space, but this has become a very large space indeed. When even adults decide the best ever therapist is now AI, we have arrived at the same place. Outsourced empathy. Human contact downgraded. Design and regulation are having to put up a fight against the fact that, in spite of evolution operating on the opposite principle, we appear to want to be fooled. Thanks for your work and that of your hosts.

Expand full comment
Mandy McLean's avatar

Thanks for reading and for this thoughtful reflection, Poul. I agree - outsourcing empathy is one of the most unsettling parts of this shift. It’s not just about what AI can do, but what we’re willing to trade away. If we lose sight of the value of human connection, especially in care and mental health, no amount of regulation or design will be enough. The challenge is finding the right balance of using AI to support, not replace, the relationships that matter most.

Expand full comment
HoldTheCenter's avatar

It pains me to see the resources poured into this tech when the core problem calls for real humans to interact and care for one another. I work with a peer-led mental health nonprofit that connects REAL people to other REAL people who can relate to their stories. https://www.dbsacalifornia.org/

Expand full comment
Paige Gardner's avatar

1) thank you for your very important work! 2) it makes me so sad to hear that there are that many children in crisis or borderline crisis. We have to do better, although I read articles like these and can’t help but feel like we are up against monsters the size of mountains

Expand full comment
Jud Heugel MD's avatar

Totally agree and feel the same. But within my sphere at health tech companies (primarily focused on enabling virtual care for children and families) I’ve found so many devoted and caring professionals across all aspects of clinical-product development (ie, counselors, nurses, PAs, NPs, doctors, product managers, program managers, software engineers, leaders of companies, etc) who are trying to do the right thing and to do it in a better way than depicted here. I am excited for the future here and working to take it toward a healthier direction.

Expand full comment
Paige Gardner's avatar

Thank you for this very much needed boost of optimism!! 🙏 I’m glad there are so many good people in this fight

Expand full comment
Natalia Lavrishina's avatar

I had the same feeling, it’s so sad and depressing I couldn’t finish reading the article.

I did and keep doing more than a lot to protect my teens from all of this, but I feel that it’s a never ending battle and that I m depleted.

Expand full comment
Paige Gardner's avatar

I don't even have children, so reading this can paint a very bleak future for me when I think about raising my own one day! I think, as Jud said, it is important to remember how many GOOD people are in this fight...and I also think its beneficial to take a step back from the online world ourselves. (I think my SubStack algorithm has become a little too doom and gloom lately!)

Expand full comment
Brandi Day's avatar

In response to just such a problem, our local school district is providing students access to the AI-based Alongside app. I can see the benefit, but I fear it is only a gateway to other chatbots with fewer (no?) guardrails. There has to be a better way.

Expand full comment
Jud Heugel MD's avatar

Yeah that is a worthwhile initial approach and I’m excited about others in that sector who will offer even better products in that daily coaching space combined with human counseling.

Expand full comment
Ruth Gaskovski's avatar

Thanks for this excellent, well-researched wake-up call Mandy. We've learned how social media and a phone-based life have created massive mental health problems and an “anxious generation”. Now we are diving headlong into another societal experiment with AI.

I was heartened to read that your recommendations not only rested on need for regulations, but for parents to take the lead in protecting their children from profit-driven mind manipulation.

Expand full comment
Mandy McLean's avatar

Thanks for reading and for the thoughtful note, Ruth. I agree that we can’t afford to repeat past mistakes with AI, especially as it moves at an unprecedented pace. Parents have a real opportunity to lead here, even before policy catches up.

Expand full comment
Bumps in the Road's avatar

Thank you for this thoroughly researched and accessible article. AI is so many things but we are witnessing the negative impact that smartphones have had on our youth from depression and distraction to lack of attention, motivation and emotional disregulation. This tool has brought social media, gaming, entertainment into the tiny hands of our children. I believe technology is a tool for professionals, not children.

Children are perhaps the canary in the coal mine. Their rapidly growing brains are being hard-wired to technology when they should be connecting to humans.

Expand full comment
Mandy McLean's avatar

Thanks for reading and for the thoughtful note. I completely agree. Children’s experiences are showing us in real time what happens when technology moves faster than our care and responsibility. We need to pay attention.

Expand full comment
Brian Villanueva's avatar

I can not reinforce this enough. I am a former IT pro myself and I now give seminars to homeschool parents. These parents are involved, wary of technology, and already deeply countercultural, but most are utterly ignorant of this risk.

No child needs access to ANY AI tool prior to age 16. Beyond that, it should be used only or school related research, and Mom & Dad get access to any account used.

Like most issues of keeping kids safe online, it would be better if the solution were legal, but absent that, parents need to be extremely careful about this.

Expand full comment
Mandy McLean's avatar

100%. I really enjoy the Hard Fork podcast from NYT. The August 1st episode on Age Gating summed this up well, "And parental controls are the thing that everyone talks about, but any parent can tell you these are not perfect. **You practically have to become like a part-time IT person just to be able to like understand and control what your underage kids are doing on the internet.**"

Expand full comment
Barbara Swander Miller's avatar

Terrific article with careful research! Thank you for your thorough examination of this growing and scary development.

This insidious danger has already begun to impact our lives and most of us are unaware. As a former teacher who has bemoaned the impact of addictive video games on young people and our society, I fear AI companions will have an even more far-reaching and devastating effect on kids’ brains, their lives and our world. You’re right: it’s about the money.

Expand full comment
Emily's avatar

My daughter was in first grade when the pandemic hit and so spent the first 3 years of elementary school fluctuating between online and in person school. And even in person learning had limited group work, collective performances, or even team sports. By the time she entered fourth grade I was devastated to see how far behind her socialization skills were from what would be typical for a 9 nine year old. The good news is that after three years of actively encouraging her to play on school teams, join clubs, and sing in the choir, so as to find her "people", she seems to finally be demonstrating a healthy social level as she heads to middle school. Online relationships are a meagre shadow of the real thing. Calling it "social" media, doesn't mean it teaches socialization and AI companions are the most inadequate of playmates.

Expand full comment
Mandy McLean's avatar

It’s great to hear how much progress your daughter has made - rebuilding social skills after a disruption like the pandemic takes deliberate, sustained effort. Kids don’t just “bounce back”; they do when adults create real opportunities for connection and teamwork.

Expand full comment
Emily's avatar

The happy story is that there are pathways to recovery and with TLC kids can bounce back. I worry about other kids whose families aren't in a position like us to be able to recognize the issue and take steps to resolve it.

Expand full comment
Dinah's avatar

This is shocking! I didn't know kids could get this attached to an AI chatbot. This is another conversation to have with your kids!!!

Expand full comment
Mandy McLean's avatar

It really is, and it’s happening much faster than most people realize!

Expand full comment
Brendan B's avatar

Of course their answer to the problems caused by technology is more, better technology. They're tech companies!

Expand full comment
Michael D.B. Duong's avatar

This is an incredibly powerful piece and I agree 100% with the premise. I’m personally building edtech for early education, but with most of the points made in this article in mind.

That is to say, the objective is not competing for attention and building feedback loops for engagement.

Inversely, my edtech is designed to be closer to a “why” answerer. Encourage curiosity and creative learning/thinking.

To be used as a TOOL to enhance real-life experiences. Not as a babysitter or substitute for emotional (human-based) learning. I’d love to chat with you about your thoughts on the intentional design.

This is a very vulnerable (but if done right, the guidance and encouragement given at this stage is the most critical) stage in human development.

I’d like to ensure we’re getting it just right.

Expand full comment
Mandy McLean's avatar

Thanks so much for this, Michael. I’m also working in the education space, though focused on K-12. It’s encouraging to hear from others thinking intentionally about design, especially in early ed where the foundation is being built. I completely agree - this isn’t about grabbing attention, it’s about supporting curiosity, creativity, and real connection. Getting it right at this stage really does matter.

Expand full comment
Avi Chai's avatar

"Nearly a third said chatting with an AI felt at least as satisfying as talking to a person, including 10% who said it felt more satisfying."

That's because we became overly accustomed with texting each other on our phones! So just bring back the habit of meeting in person...

Expand full comment
Mandy McLean's avatar

I totally agree! Thanks for reading and reflecting on the piece.

Expand full comment
DanB1973's avatar

We hate our children. We hate them so much that we are ready to work overtime and in multiple jobs only to get some money to pay a stranger (a babysitter) to remove them away from us.

Children are like side effects. Inevitable, but we do not want to deal with them.

They are dangerous. They are watching us all the time, mimic our worst habits and put them up right before our eyes.

Over time, they learn to demand and fight for what they want. And they will always succeed.

We are so humiliated by our children that we feel the urge to crush them from day 1 of their life. We keep them in prison (aka home) for up to 20 years. We hurt their souls, destroy their relationships, impose our whims on them, plan their future and demand obedience.

Maybe one day we will come to the understanding that we do not know their language, not a single word.

Expand full comment
Mandy McLean's avatar

It’s hard to read, but I think that’s the point. Kids have a way of revealing our deepest contradictions - our love and our limits, our ideals and our exhaustion. Maybe the most important shift is to stop seeing parenting as managing outcomes and start seeing it as a relationship we’re always learning how to do better.

Expand full comment
DanB1973's avatar

I used to think like (the mirror thing) that because it seems logical and rational. Then I came across a bunch of teachers and masters, what we call “spiritual”. And I experienced the same thing with them. Or stronger, mostly stronger :-) So I concluded that neither do anything. Children don’t do anything and masters don’t do anything. Both simply are. Maybe because they “only” are, all those things that you refer to come to the surface and keep punching us in the face until we finally get the point and start being there, too.

I love your turning kid management into a relationship - isn’t it what all life is about?

All this applies equally to adults, I guess. We do our worst to manage them and push them around and command and check boxes in our mental checklists. Instead of sharing our own life story with them and allowing them to share their part with us.

[Yes, there are monster kinds and monster adults. Herding and managing them is the only way to sanity. I haven’t met a lot of these, fortunately.]

A great article and a great read from you, thank you.

Expand full comment
David Zelenka's avatar

A student of mine goes home and chats with ChatGPT until it asks to upgrade. So sad. I just wrote a book on the effects of digital media. https://interactive-earth.com/analog-jesus

Expand full comment
Mandy McLean's avatar

Thank you for sharing, David.

Expand full comment
Lukas Bird's avatar

So, let’s pull this thread at scale.

FACT: Meta investing in personal super intelligence

MODEL: Companies from Hasbro to Sony to Sex Robots will build atop this personal super intelligence to deliver a lifetime of evolving social companions from talking teddy bears and video games and sex bots. Always on. Always ready to play, flatter, sooth, sell.

FACT: men and women are divided politically and that divide will only increase as our society grows more extreme.

WE WILL: Turn to these replacement relationships at scale. Forsake the opposite sex. Forget how to seduce, forget how to negotiate. Stop forming families. Fall further beneath replacement rates. Shrinkage of those to support infrastructure and defend borders.

This “cure” for loneliness is like adding YouPorn and Only Fans apps to our phones and telling us to increase our personal connections.

Expand full comment
Mandy McLean's avatar

This is a powerful (and chilling) thread to pull, Lukas. When tech is designed to soothe, flatter, and remove friction, it’s not hard to imagine people forgetting how to do the real thing.

I don’t think this future is inevitable (I won’t let myself believe that), but it’s a real risk. We need to invest more in the kinds of environments, relationships, and experiences that actually build the capacity for human connection, especially early in life.

Expand full comment
Lukas Bird's avatar

I’m not a natural pessimist. But I am a natural realist.

Realistically, our young women are hardening as progressives and our young men are moving hard to the right. This is the curse of The Personal is Political identity politics. This is a severe headwind to relationships in and of itself.

Add to this Nor’easter of gender based cultural bile (thanks progressives) a hurricane of AI synthetic intimacy - brought to us by A Really Big Corporation so their founder can brag about being earth’s first trillionaire.

These twin social events combine to be The Perfect Storm. We could probably deal with either alone. But not both together.

Easy to play this out: Conservative men (alpha) will continue to mate with conservative women. Who will be more prized than ever. Liberal men (beta) will struggle to mate with liberal women who quit men more than ever. Beta men will turn more than ever to AI intimacy, porn, Only Fans and sex bots )as they become more affordable and fitted with personal AI).

At scale, this trend leads to conservative natalism in the suburbs and rural and plummeting family formation in cities. Which will fail due to office jobs shipped to the cloud and lower marriages. Liberal women will keep each other company, adopt children, and begin to shrink as a cohort as they neglect to procreate. They live Sex in the City consumption lives of pleasure. America’s next generation will trend conservative because they chose to have kids and raised them with similar values.

Expand full comment
Tim Colman's avatar

This story in the AP today agrees with our perspective. What lunatics are running our asylum?

New study sheds light on ChatGPT’s alarming interactions with teens https://apnews.com/article/chatgpt-study-harmful-advice-teens-c569cddf28f1f33b36c692428c2191d4

Expand full comment
Mandy McLean's avatar

Thanks for sharing, Tim.

Expand full comment
William C. Green's avatar

Emotional exploitation of minors by for-profit AI platforms is serious. Yes, regulate--but spare the sanctimony. Panic is not policy. Fear is not foresight. Treating all AI as a single monolithic threat distracts from designing real solutions. It is easy, again tiresome, to blame the tool instead of confronting deeper social absences. If we value human connection for kids, we need to show it ourselves—less outrage, more responsibility. Otherwise, we’re just moralizing while the world changes without us.

Expand full comment
Mandy McLean's avatar

Appreciate this, William. I’m with you that panic isn’t helpful, and it’s easy to flatten the conversation into “all AI is bad” when the real work is more complicated. I wrote the piece to call out specific risks with AI companions designed to maximize engagement, especially when they’re marketed to teens. But I agree - if we want kids to value real connection, we have to show what that looks like.

Expand full comment
Brandi Day's avatar

William, I also agree -- to a point. But, as with social media, AI is designed to draw users in and keep them using. In that case, the problem is the tool and it is something to fear. So we need to have the foresight to stop it. I don't expect that to come from regulation given how ineffective our government has been on that front. In fact, everyone from our president on down to our teachers are pushing it on our kids. The solution, as always, does need to come from individuals -- those who say no to using AI and those who actually take the time to invest in relationships to keep people from feeling the need to turn to AI.

Expand full comment
William C. Green's avatar

In turn, I also agree with you—to a point. Like it or not, AI is pervasive: often in the background, sometimes in the foreground, and always growing stronger by the minute, shaping the news we receive and the things we buy—whether we realize it or not. Turning off the television and buying artisanal food or clothing may be fine as a lifestyle choice, but it’s a poor policy. We’re left with the responsibility to manage what we cannot escape, pressing for regulation and control rather than retreat. Simply saying no can become a selfish exit that leaves the real story to unfold without us. I suspect we’re not far apart. I’m just wary of binaries and simple YES or NOs. - Thanks for your response!

Expand full comment
Brendan Kelly's avatar

When children are ignored because parents are too busy with their own addictions to various Apps what do we expect children to do as they enter adolescence? I can see a whole generation seeking relationships in AI because they never had real relationships with their parents to start with. Parents need to wake up and step up by examining their own behavioural patterns.

Expand full comment