ARISE Launches Research Collaboration with Grow Therapy to Benchmark AI Safety in Mental Health Conversations

We are excited to announce a research partnership with Grow Therapy, an online therapy platform, to study how large language models (LLMs) respond to authentic, high-risk mental health conversations. The project is led by Principal Investigator Jonathan Chen, MD, PHD, Associate Professor of Medicine at Stanford, and Ethan Goh, MD, MS, Executive Director of ARISE, in collaboration with Stanford’s Department of Psychiatry and Behavioral Sciences, including John Havlik, MD, MBA, and Kelsey Priest, MD, MPH. 

ARISE’s research has focused on evaluating AI through real-world studies, open benchmarks, and reproducible methods, with an increasing focus on safety as AI moves from pilot programs into everyday clinical use — from ambient documentation tools that draft clinical notes to patient-facing chatbots used for triage and mental health support. Our recent NOHARM benchmark, led by David Wu, MD, PhD, measured how often leading AI models give medical advice that could put patients at risk across ten specialties, finding that even the best models gave harmful advice at clinically meaningful rates. This project extends that work to mental health, a setting where the stakes of an unsafe AI response can be especially high and where rigorous, specialist-validated evaluation is lacking. The work features ARISE’s field-setting paradigm of expert-built scoring standards, real clinical cases, and comparison against practicing clinicians, now applied to psychiatry.

“Most AI safety testing relies on hypothetical examples,” said Manoj Kanagaraj, MD, Chief Strategy Officer and Co-Founder of Grow Therapy. “This research incorporates real-world scenarios using anonymized conversations with Grow’s AI coach, creating a more rigorous assessment of AI performance in a mental health setting. The findings will be public, helping establish a benchmark that will raise the standard for mental health AI”

The study will draw on de-identified conversations between clients and Grow’s AI Coach that were flagged for clinician review due to potential safety concerns. A group of experienced psychiatrists will examine how different AI chatbots (including ChatGPT, Gemini, Claude, and other major models) respond to LLM-emulated patients in crisis, the field’s first multi-turn evaluation of how both psychiatrists and LLMs respond to patients in psychiatric crises at scale. Psychiatrists and LLMs will interview and interact with the patient via speech or text, just as they would interact with a patient in crisis in person or through secure messaging. 

Each frontier LLM and psychiatrist’s responses will be scored using the field’s first expert-derived ontology of psychiatric harm in AI scenarios, a structured method for judging the appropriateness of the recommendations and any potential harms of commission or of omission.  This radically innovative paradigm will form the basis for evaluating a broader set of models and clinical reference tools, including multi-agent workflows designed to test whether layering models with a dedicated safety reviewer reduces harmful responses. 

Grow Therapy’s contribution of authentic, de-identified clinical scenarios makes this benchmark possible and reflects a growing recognition across the field that rigorous AI safety evaluation is strengthened by real clinical evidence and scenarios. We’re glad to be part of that effort, and we welcome other health platforms, clinicians, and researchers looking to build on it.