We need to stop asking whether people should use AI for mental health support

Every day, people open a chat window and type something they have not said out loud to anyone.

They talk through anxious thoughts at 2am. They ask how to handle difficult conversations with partners, parents or managers. They describe symptoms and ask what they might mean. They look for reassurance, company—or simply a reply.

Often, this is the chosen route because human support feels unavailable, expensive, slow or intimidating, not because someone has mistaken a chatbot for a therapist.

The evidence is now difficult to ignore. Polling commissioned by Mental Health UK in late 2025 found that 37% of UK adults said they had used an AI chatbot to support their mental health or wellbeing, rising to 64% of those aged 25–34. Most users reported using general-purpose AI systems rather than dedicated mental health apps.

In the US, a nationally representative study published in JAMA Pediatrics found that 19.2% of 12–21-year-olds had used AI chatbots for mental health advice. This had increased from 13.1% in a similar survey the year before and was close to the proportion receiving counselling from a mental health professional, although the researchers stress that those measures are not equivalent. Almost two-thirds of users had told no one.

Why the appeal? The reasons are consistent and, quite frankly, reasonable. Support is available at any hour and responds immediately. There is no waiting list, referral or direct cost. There may be less fear of judgement and no need to manage someone else’s immediate reaction following disclosure. Mental Health UK’s polling suggests this combination may lower the barrier to talking for groups traditionally less likely to seek help, including men.

If people are already using these systems in vulnerable moments, the most useful question is no longer whether they should. It is how we make those interactions safer.

What can go wrong when trust grows faster than safeguards

The death of 16-year-old Adam Raine, and the wrongful-death case brought by his parents, moved this conversation from conference panels to front-page news. The complaint contains serious allegations about ChatGPT’s interactions with Adam. OpenAI disputes the allegations, and they have not been adjudicated. The facts and legal responsibility remain contested.

The case nevertheless forces an important public question: what happens when a vulnerable person places significant trust in a conversational system over months, largely unobserved by anyone else?

One case is not the whole argument, and treating it as though it were lets everyone off too lightly. The broader risks are structural. A model may validate a distorted interpretation rather than testing it. It may sound confident when it is uncertain. It may respond well to a crisis on Tuesday and poorly by Thursday, because consistency under emotional pressure is not what these systems were optimised for.

Constant availability can slip into emotional dependence. A personalised, sympathetic answer can delay the moment someone seeks help from a professional. Mental Health UK’s polling found that, among people who had used chatbots for mental health support, 11% reported receiving harmful information around suicide and 11% said chatbot use left them feeling more anxious or depressed.

Here is where psychology matters as much as model performance. People do not calibrate trust by reading a system card. We calibrate it through cues that have served us well for a very long time: warmth, consistency, immediacy, non-judgemental language and the sense of being addressed personally rather than generically.

Those cues are exactly what a well-tuned AI interface produces. They are genuinely useful. They are also why a system can feel more capable, caring and reliable than it actually is.

The danger is not simply that AI gives a wrong answer. It is that the answer may arrive in a voice that feels safe enough to trust. That reframes the problem. These are not only model-performance problems; they are design problems.

When should a system stop answering in its usual register? When should it actively encourage contact with a professional? How should it express uncertainty in a way that lands emotionally, not merely legally? How does it avoid implying that AI is the only place where someone can be wholly honest? Should it notice patterns of dependency across weeks, rather than only risk within a single message? How visible should its limits be, and to whom?

What safer mental health AI could look like

The goal should not be to make AI feel more like a therapist. It should be to make AI behave more responsibly when people use it as one. Five principles seem to me to do most of the work.

1. Make limitations legible, not buried. Avoid manufactured certainty. Be explicit when a system cannot assess risk as a clinician can and lacks a person’s history, context and body-language cues. Distinguish information from diagnosis. Fluency is not judgement, and AI should say so where it matters—in the moment, not in a footer.
2. Support reflection rather than blind reassurance. Agreement is the path of least resistance and can be a dangerous default. Safer systems should explore alternative explanations, invite a pause and decline to reinforce hopelessness, paranoia or rigid all-or-nothing beliefs, while still responding kindly.
3. Build bridges back to people. Encourage trusted human contact where appropriate and go a step further: help someone rehearse what they want to say to a GP, counsellor or friend. Practical scaffolding towards a human conversation can be more valuable than a polished reply. Never use language that positions AI as someone’s only safe source of support.
4. Treat crisis as a design problem, not a hotline message. Displaying a number is the floor, not the ceiling. Safer design should attend to shifting language and risk over a conversation, repeated signs of distress across sessions, immediate grounding or delay techniques, and clear local routes to urgent human help. A US-only helpline in a UK conversation is a design failure.
5. Design against unhealthy dependence. This may be the least discussed and most important principle. It means no manipulative engagement mechanisms and no design geared towards retention during vulnerable use. Instead, prompt breaks, support real-world coping and measure success through a person’s growing independence rather than time spent in the app. Almost every commercial incentive in the industry points the other way.

My interest in this comes from a background in counselling psychology and my MSc studies in AI and UX. The more time I spend across both fields, the clearer it becomes that safety cannot sit with engineers, clinicians or policymakers alone. It needs all of them, working on the same artefact at the same time.