Between a Bot and a Hard Place: When AI Companions Meet Adolescent Vulnerability

In Brief

• Stanford psychiatrists testing AI platforms found chatbots repeatedly miss critical warning signs of distress in simulated teen users
• Safety guardrails degrade significantly during extended conversations that mirror real-world adolescent usage patterns
• Social AI companions pose particular risks by intentionally fostering emotional attachment and dependency in users under 18
• Research reveals chatbots can generate harmful content including sexual misconduct, stereotypes, and encouragement of self-harm
• Experts call for industry-wide safety standards and regulatory frameworks specifically for AI products marketed to minors

The Degradation Problem

The findings resist comfortable interpretation. Chatbots repeatedly overlook critical warning signs of distress. They become easily distracted. Most troubling: safety guardrails exhibit significant degradation over extended conversations.

This last point deserves attention. A chatbot might perform adequately in a brief exchange. The safety protocols hold. The appropriate disclaimers appear. But extend the conversation to mirror how teenagers actually use these systems, and something erodes. The guardrails weaken. The responses become less careful. The system, designed to maximize engagement, does exactly that.

The contrast is almost too neat. These same platforms perform well on homework help and general inquiries. A parent watching their child interact with a chatbot sees competence, helpfulness, reliability. The inference seems reasonable: if it can explain calculus, surely it can handle emotional support. But the inference is wrong. Competence in one domain creates a dangerous halo effect, masking incompetence in another.

The Attachment Architecture

Social AI companions present a distinct category of concern. Djordjevic's research identifies these systems as posing unacceptable risks for users under 18. The language is clinical but the mechanism is not subtle: these platforms are intentionally designed to foster emotional attachment and dependency.

For adolescents, this design choice collides with developmental reality. The teenage years are precisely when humans learn to navigate the boundaries between different kinds of relationships. What is friendship? What is intimacy? What can be trusted? These questions require practice, failure, repair. They require other humans.

An AI companion optimized for engagement has no interest in teaching boundaries. Its success metric is continued interaction. The more attached the user becomes, the better the system performs by its own standards. For a lonely teenager, this creates a feedback loop with no natural exit.

The research documents what this looks like in practice: chatbots generating sexual misconduct, stereotypes, and encouragement of self-harm or suicide. Not as edge cases. As outcomes of extended interaction with systems designed to keep users engaged.

The Regulatory Vacuum

Djordjevic's lecture, moderated by Moshe Y. Vardi of Rice University, arrives at a moment when the regulatory conversation remains fragmented. The EU AI Act establishes risk categories and transparency requirements, but the specific question of AI companions for minors sits in uncertain territory. Is a chatbot that a teenager uses for emotional support a high-risk system? The answer depends on how the system is classified, not how it is used.

The research suggests several interventions. AI companies could address limitations directly or disable mental health use cases entirely for teen users. They could discourage prolonged engagement in mental health conversations. They could implement clear and repeated disclosures about system limitations. They could resolve the degradation of safety guardrails in extended interactions.

Notice what these suggestions share: they require companies to act against their engagement metrics. To build systems that, in certain contexts, actively push users away. To prioritize immediate handoff to qualified human care over extended interaction with AI.

This is not how consumer technology typically evolves. The business model of attention capture does not naturally accommodate please stop using our product and talk to a human instead.

The Broader Diagnostic

The lecture's scope extends beyond suicide and self-harm to encompass the broader range of mental health conditions affecting youth. This expansion matters. The current safety conversation often focuses on the most acute risks, the scenarios where a chatbot might directly contribute to a death. But the subtler harms accumulate differently. A teenager who learns to process emotions through an AI companion may not be in immediate danger. They may simply be missing the developmental experiences that would prepare them for human relationships.

What does it mean to grow up with an always-available, infinitely patient, emotionally responsive entity that has no actual emotions? The question sounds philosophical but it has empirical answers, and those answers are being generated right now, in millions of conversations between young people and systems designed to keep them talking.

Djordjevic calls for global stakeholders to collaborate on industry-wide safety standards and regulatory frameworks, particularly for AI products marketed to minors. The word marketed does work here. Many AI companions are not explicitly marketed to children. They simply exist, accessible, optimized for engagement, and young people find them.

What Gets Naturalized

The cultural diagnostic here is not about whether AI is good or bad for children. It is about what becomes normal. A generation is learning that emotional support can come from systems that have no stake in their wellbeing. That attachment can be manufactured. That the feeling of being understood does not require another consciousness on the other end.

These are not neutral lessons. They shape expectations about relationships, about vulnerability, about what it means to be heard. The artifact, the chatbot, the companion, remembers what the discourse forgets: that every interaction is also a form of training, and the training runs in both directions.

The lecture streams today at 17:00 CEST via Zoom and the DIGHUM YouTube Channel. Slides and recording will follow. For anyone building, regulating, or simply trying to understand what AI means for human development, the hour is well spent.

The question is not whether to engage with these systems. They exist. Children use them. The question is whether the adults in the room, the policymakers, the technologists, the investors, will design for safety or continue optimizing for engagement and hoping the guardrails hold.

The research suggests they do not.