AI Scribes in Medical Education: Impact on Student Note Quality
Publication Title: Impact of an Ambient AI Scribe on Medical Student Observed Structured Clinical Examination Notes: A Nonrandomized Clinical Trial.
Summary
- Question
- This study aimed to assess the impact of an ambient artificial intelligence (AI) scribe on the quality of clinical notes written by first-year medical students during a structured clinical examination. Researchers specifically evaluated whether integrating AI-generated notes with student-written notes (hybrid notes) influenced documentation quality and clinical reasoning.
- Why it Matters
- Documentation is a critical skill for medical students, helping them organize and communicate clinical findings effectively while developing clinical reasoning. AI scribes, which generate notes by analyzing patient-doctor conversations, are increasingly used in clinical practice, but their effect on medical education remains unclear. Understanding how AI scribes influence student learning is vital for integrating this technology responsibly into training while preserving essential skills like critical thinking and accurate documentation.
- Methods
- The study involved 104 first-year medical students at a U.S. medical school. Each student completed a human-only clinical note based on a simulated patient encounter. A subgroup of 47 students also created hybrid notes by revising their original notes using AI-generated notes. Notes were scored by blinded raters using a standardized tool called QNOTE, which evaluates key elements of clinical documentation. Students who created hybrid notes completed a survey about their experience with the AI notes.
- Key Findings
- The quality of human-only notes was high, with most elements receiving scores in the acceptable range. Hybrid notes showed minimal differences compared to human-only notes, except for a slight decline in scores for the 'Chief Complaint' section, which often omitted symptom duration in AI-generated notes. Among students with initially lower scores, hybrid notes showed improvement. Survey results revealed that students found AI notes concise and useful as a starting point but noted that they often omitted important details. A minority of students expressed concern that reliance on AI could hinder their ability to learn effective note-writing skills.
- Implications
- The findings suggest that AI scribes can complement medical student documentation without significantly affecting note quality or clinical reasoning, particularly when students generate independent notes before incorporating AI content. For lower-performing students, AI scribes may enhance note quality. However, educators should implement safeguards to prevent overreliance on AI and ensure students continue developing critical skills like clinical reasoning and detailed documentation.
- Next Steps
- The authors recommend future studies to explore the long-term effects of AI scribes on medical training, including their impact in real-world clinical workflows where AI notes serve as initial drafts. Research should also address strategies to mitigate potential risks, such as automation bias and overdependence on AI tools, while optimizing their educational value.
- Funding Information
This research was supported by the Yale School of Medicine Office of Curriculum.
Full Citation
Talwalkar JS, Wright DS, Schwamm LH, Leydon G, Shabanova V. Impact of an Ambient AI Scribe on Medical Student Observed Structured Clinical Examination Notes: A Nonrandomized Clinical Trial. JMIR Med Educ 2026 PMID: 42125891, DOI: 10.2196/88264.
This AI-assisted summary has been reviewed and approved by at least one of the study's authors to ensure it accurately reflects the research.
Authors
Jaideep S. Talwalkar, MD
First AuthorProfessor of Internal Medicine (General Internal Medicine)
Veronika Shabanova, PhD
Last AuthorAssociate Professor of Pediatrics (General Pediatrics) and of Biostatistics