Hallucination, Harm & Hype: Can LLMs Be Trusted in Healthcare?
Balancing Innovation with Risk in Patient-Centered AI
Description
This panel explores the fast-growing integration of large language models (LLMs) into the healthcare ecosystem. From diagnostics to patient engagement and clinical documentation, generative AI is transforming workflows — but not without risk. Panelists will investigate the potential for hallucinated outputs, the impact of AI errors on patient safety, and the regulatory frameworks guiding safe deployment. How can healthcare leaders navigate innovation while protecting patients?
Background
The integration of large language models (LLMs) into healthcare marks a transformative moment in the evolution of clinical tools and patient interaction. These models, capable of generating human-like text, are already being piloted in areas such as diagnostics, clinical documentation, administrative workflow optimization, and even patient communication. By automating routine tasks and enhancing d…
Date: 2025-06-02
Time (ET): 5:00 PM EDT, Jun 2, 2025
Time (Local): 9:00 PM UTC, Jun 2, 2025
Location: online
Speakers
Duncan Eddy
Research Fellow, Stanford University
Dan Roth
Chief AI Scientist, Oracle, Professor at the Department of Computer and Information Science, University of Pennsylvania
Maneesh Goyal
Chief Operating Officer, Mayo Clinic
Gagan Bansal
Senior Researcher, Microsoft Research
Anil Jain
Chief Innovation Officer, Innovaccer
Soroush Saghafian
Professor, Harvard University
Guided Questions
Maneesh Goyal
With your background in scaling global centers of excellence and healthcare consulting, how can health systems build the operational foundations needed to responsibly implement LLMs, including governance, talent, and cross-functional collaboration?
Duncan Eddy
Drawing from your work at the Center for AI Safety, what frameworks or principles do you believe are most effective for investigating and mitigating failure modes in LLMs used for clinical decision support?
Dan Roth
Given your extensive work in natural language understanding, reasoning, and machine learning, how should we rethink the design or training of large language models to improve their reasoning capabilities and reduce hallucinations—particularly in high-stakes healthcare settings where patient safety is paramount?
Gagan Bansal
With your expertise in human-AI interaction and agentic systems, how should we design LLM-powered agents in healthcare to effectively collaborate with clinicians — supporting their decisions without overwhelming or misleading them?
Anil Jain
Given your experience shaping national health IT policy and contributing to the Federal Health IT Advisory Committee, what regulatory or policy frameworks do you believe are most urgent to ensure that LLMs in healthcare protect patient safety without stifling innovation?
Soroush Saghafian
Given your collaboration with institutions and state health departments, what are the most pressing challenges you’ve observed in translating AI innovations, like LLMs, from academic research into scalable, real-world healthcare applications?