ARISE Awarded $250,000 Gordon and Betty Moore Foundation Grant to Evaluate AI-Clinician Collaboration in Complex Medical Decision-Making

The ARISE research network has been awarded a $250,000 grant from the Gordon and Betty Moore Foundation to launch a multi-site randomized controlled trial evaluating how clinicians perform on complex diagnostic and management cases with and without AI assistance.

The one-year project, titled “Evaluating the Performance of LLMs and Clinicians in Complex Diagnostic and Management Cases: A Randomized Controlled Trial,” will be led by Principal Investigator Dr. Jonathan Chen (Stanford University), with Co-Investigators Dr. Adam Rodman (Beth Israel Deaconess Medical Center / Harvard Medical School), Dr. Ethan Goh (Stanford University), Dr. Andrew Parsons (University of Virginia), and Dr. Andrew Olson (University of Minnesota), and a multidisciplinary team of physicians, informaticians, and researchers.

Addressing Diagnostic Excellence with AI Support

Building on recent JAMA and JAMA Internal Medicine publications demonstrating the surprising proficiency of large language models (LLMs) like GPT-4 in diagnostic and management reasoning, this study represents a critical next step: shifting the focus from AI vs. humans to humans with AI vs. humans without AI.

“AI won’t replace doctors, but doctors who use AI will replace doctors who don’t,” said Dr. Curt Langlotz, Professor of Radiology and Medicine at Stanford.

Study Design

The trial will randomize practicing physicians across three study arms:

• Clinicians with AI support (GPT-4)
• Clinicians with standard reference tools (e.g., UpToDate)
• AI alone

Participants will complete challenging diagnostic and management vignettes curated by expert panels and scored with rigorously developed rubrics. Outcomes will assess not only diagnostic accuracy but also the quality of clinical reasoning and appropriateness of management decisions.

The study will also include qualitative interviews to better understand how AI systems influence decision-making and how they may be safely integrated into everyday practice.

Why It Matters

Diagnostic error is a leading cause of preventable patient harm. While AI systems show promise, their safe and effective use depends on careful evaluation of how they interact with—and augment—the reasoning of human clinicians. This study will generate high-quality evidence to inform both medical practice and policy on AI integration in healthcare.

Collaboration and Impact

This multi-institutional effort brings together leading voices in clinical reasoning and AI:

• Stanford University (BMIR, CERC)
• University of Minnesota
• Beth Israel Deaconess Medical Center / Harvard

The findings will advance benchmarks for evaluating medical AI, shape guidelines for safe adoption, and contribute to the Moore Foundation’s mission of advancing diagnostic excellence in patient care.

About the Gordon and Betty Moore Foundation

The Gordon and Betty Moore Foundation fosters pathbreaking scientific discovery, environmental conservation, patient care improvements, and preservation of the special character of the Bay Area.

Full announcement: https://www.moore.org/grants