BIDS Grand Rounds

" The Observer Project: Bringing Biomedical Researchers into the Room Where It Happens" by Kevin B. Johnson, MD, MS - David L. Cohen University Professor of Biomedical Informatics, Computer Science, Pediatrics, and Science Communication at the University of Pennsylvania.

Abstract: Primary care is the front door of the American healthcare system and one of its least studied environments. Despite decades of investment in electronic health records, clinical decision support, and administrative automation, the outpatient visit itself remains largely a black box. What physicians and patients actually say to one another, how attention shifts during documentation, whether an agenda gets addressed or quietly abandoned: these phenomena are rarely captured, and almost never studied at scale.

The Observer Project is an effort to change that. Observer is a multimodal repository of primary care encounters, combining synchronized audio, video, and structured clinical data collected under informed consent from patients and clinicians across diverse practice settings. The repository is designed as an open research resource, enabling investigators across medicine, engineering, linguistics, and the social sciences to study ambulatory care as it is actually practiced.

This presentation describes the methodological and ethical infrastructure required to build Observer, including automated de-identification pipelines for clinical video and speech, multistakeholder review processes, and data governance frameworks designed to support broad research access. It also presents early findings from experiments using Observer data to develop and evaluate tools for clinical care, including pre-visit history collection, ambient documentation, real-time agenda management, and passive screening for cognitive impairment.

The path to trustworthy, equitable clinical AI runs directly through the exam room. Getting there requires looking carefully at what happens inside it.

Bio: Kevin B. Johnson, MD, MS is the David L. Cohen University Professor of Biomedical Informatics, Computer Science, Pediatrics, and Science Communication at the University of Pennsylvania, and Vice President for Applied Clinical Informatics at Penn Medicine. He earned his MD from Johns Hopkins University and an MS in Medical Informatics from Stanford University. Prior to joining Penn, Dr. Johnson served as Chair of Biomedical Informatics and Chief Informatics Officer at Vanderbilt University Medical Center.

An internationally recognized leader in clinical informatics, Dr. Johnson pioneered the use of text messaging for behavior change and developed early electronic documentation and e-prescribing systems that transformed care delivery. At Penn, he directs the Artificial Intelligence for Ambulatory Care Innovation (AI4AI) lab, advancing AI to reimagine the clinical encounter.

Dr. Johnson has authored over 200 publications and is a member of the National Academy of Medicine and a fellow of the American College of Medical Informatics, the International Academy of Health Sciences Informatics, and the American Institute for Medical and Biological Engineering. He also produces documentaries and a podcast to engage public audiences in informatics and STEMM.

CME accredited seminar. Information for claiming credit will be provided at the start of the session.