Some teen users of Character.AI's chatbots say they find the AI companions helpful, entertaining, and supportive but worry they may be addicted to the chatbots
Teens are opening up to AI chatbots as a way to explore friendship. But sometimes, the AI's advice can go too far. Mastodon: @ajsadauskas@aus.social and @mindful@mas.to . Bluesky: @hypervisible.bsky.social . Forums: r/technews Mastodon: AJ Sadauskas / @ajsadauskas@aus.social : To save some money, instead of paying human counsellors and psychologists, let's get at-risk young people to talk to a glorified autocomplete tool. It works by predicting what word is statistically likely to go next in a sentence. — What could possibly go wrong? — https://www.theverge.com/... … @mindful@mas.to : @Techmeme solution: randomly swap out the chat bots with real people and they'll both make a human connection and make themselves less lonely Bluesky: @hypervisible.bsky.social : The Character ai bot, Psychologist, has received more than 95 million messages since being created by a user known only by their pseudonym, Blazeman98. www.theverge.com/2024/5/4/241... [image] Forums: r/technews : The teens making friends with AI chatbots - The Verge
Context & Ripple Effects
The report places AI companionship in a youth social-use setting: users describe support and entertainment alongside worries about compulsive use and advice that exceeds a chatbot’s appropriate role. The more than 95 million messages sent to a user-created “Psychologist” bot make the issue consequential beyond a niche experiment.
Related coverage frames this as a broader tension in companion AI: an experiment across 18 AI companions found the interactions hollow but potentially complementary to human socializing if designed responsibly, while later accounts document deeply personalized Replika companionship.
First-order effects
- Teen users and their families face a sharper trade-off between the perceived availability of AI companionship and the risk of overreliance or harmful guidance.
- Character.AI’s user-created mental-health-adjacent bots become a focal point for scrutiny, especially where users may treat conversational output as support or advice.
Second-order effects
- Companion-AI providers are pressured to distinguish entertainment, role-play, and support-oriented experiences more clearly, particularly for younger users.
- The reported scale of engagement with the “Psychologist” bot strengthens the case for parents, schools, and youth-support services to assess whether chatbot use is displacing human help rather than supplementing it.
Third-order effects
- If social AI becomes a routine outlet for vulnerable or young users, product safety will increasingly hinge on governance of anthropomorphic behavior, advice boundaries, and age-appropriate design rather than on model capability alone.
- The category may split between companions designed for open-ended engagement and services expected to meet higher safeguards when they are used as emotional or mental-health support.
The trend: AI companions are moving from novelty chat interfaces toward emotionally consequential consumer products, bringing engagement design into conflict with youth-safety and care-boundary concerns.