Postdoctoral Fellowship in Childhood Neuropsychiatric Disorders (T32) Trainee Talks

Name: Postdoctoral Fellowship in Childhood Neuropsychiatric Disorders (T32) Trainee Talks
Uploaded: 2026-05-26T21:31:29.6033333Z
Duration: 1 h 7 min 16 s

May 26, 2026

YCSC Grand Rounds May 26, 2026
Moderated by Michael Crowley, PhD, Assistant Professor, Yale Child Study Center

Lacey Chetcuti, PhD: "Bridging Transdiagnostic Models and Measurement: Dimensional Characterization of Social Functioning in Autism"
Joseph Heffner, PhD: "What Language Reveals About Depression Trajectories and the Impacts of Human-AI Conversations on Well-being"
Max Rolison, MD: "The Data We Don't Have: Building Clinical Research Infrastructure for Neurodevelopmental Disabilities"

Information

ID: 14249
To Cite: DCA Citation Guide

Download Transcript

00:06Good afternoon, everyone.
00:09I wanna thank you all
00:10for attending this year's Grand
00:12Rounds t thirty two trainee
00:13presentation.
00:15We're really proud of our
00:16trainees this year.
00:19Oh, before I, I wanna
00:21mention there's the, the, sign
00:23in for CME credits on
00:24the screen now. So if
00:25you want CME credits, please,
00:26grab that while it's up.
00:33K. I'm going a little
00:33slower because I'm always frustrated
00:35when someone says do something,
00:36and then they they turn
00:37it off quickly. So you
00:38have that.
00:39So,
00:41our two thirty two is
00:42in its forty
00:44starting a forty second year
00:45this fall, renewed for another
00:47five years. We just finished
00:48this first year.
00:51And I'm,
00:53Michael Block and I, have
00:54have been working hard to
00:56to make this program grow.
00:58And, so one plug that
01:00I wanna give before I
01:01tell you about the trainees
01:01and about next week's presentation
01:03is that we also have
01:04many people who are not
01:05on the t thirty two
01:06or participate in the t
01:07thirty two. So if you
01:07have a postdoc who's on
01:08your r one grant or
01:10some other fellowship or an
01:11f thirty two or something
01:12like that,
01:14you know, with the the
01:15the cohorts that we grow,
01:18have a lot of benefit
01:19to all the trainees. So,
01:20you know, please know that
01:21that's there and available to
01:22you.
01:24So,
01:24next week's grand rounds is
01:26the,
01:28Viola Bernard lecture,
01:30and,
01:31Niranjan
01:32Karnik
01:33is gonna be presenting next
01:34week using big data for
01:35child mental health research.
01:37So please, mark your calendars
01:39for that.
01:40And then lastly, without further
01:42ado, I give you our
01:43three trainees who'll be presenting
01:44today. I'm not gonna read
01:44their titles. They can do
01:45that. We have
01:47Lacey
01:48Chakudi,
01:49and, we have
01:51Joseph Heffner, and we have
01:53Max Rollison,
01:54and, three very talented trainees
01:57presenting different,
01:58types of work.
02:00And I hope that you're
02:01all excited to see what
02:02they have to offer.
02:04And I we ask that
02:05you save one question per
02:07trainee. So we want one
02:08question, after their presentation, then
02:10we'll save the end for
02:11the the remaining presentations.
02:13Thank you.
02:25Okay.
02:25Hi, everyone, and thank you
02:27for coming. My name is
02:28Lacey Chakutty, and I'm a
02:30Hillebrand postdoctoral fellow within the
02:31McPartland Lab. And today I'll
02:33be talking about how we
02:34can bridge transdiagnostic
02:36dimensional frameworks and existing clinical
02:38measurement approaches
02:39to better understand clinical phenomena,
02:41and I'll be focusing specifically
02:43on social functioning and autism.
02:48So first up, I'm going
02:49to briefly review traditional and
02:50alternative frameworks for conceptualizing clinical
02:52phenomena and discuss how our
02:54conceptualization
02:55influences the way that clinical
02:56and phenomena are measured, studied,
02:58and ultimately understood.
03:00From there, I'm going to
03:01walk through our factor analytic
03:02study demonstrating how clinical questionnaires
03:05originally developed within a traditional
03:06conceptual framework
03:08can potentially be reorganized to
03:09reflect alternative conceptualizations.
03:13And then finally, I'll demonstrate
03:15how the way we conceptualize
03:16and measure clinical phenomena affects
03:18our ability to identify underlying
03:19biological mechanisms.
03:24So psychiatric diagnosis currently relies
03:26on categorical systems such as
03:28the ICD and most commonly
03:29in the US, the DSM.
03:31So these systems operationalize clinical
03:33phenomena in binary terms, meaning
03:35that diagnoses are considered either
03:37present or absent
03:39based on symptom criteria being
03:40met or not met.
03:44So autism in particular is
03:46defined through two core
03:48symptom domains in the DSM.
03:50Criterion A is social communication
03:52and interaction
03:53deficits, and criterion B is
03:55restricted repetitive behaviors, interests, or
03:57activities.
03:59So this clinical approach to
04:01conceptualizing
04:02clinical phenomena has been valuable
04:03for a number of purposes.
04:05So for example it helps
04:06us standardize our diagnosis, it
04:08helps us communicate with each
04:10other, and it also helps
04:11us organize our research efforts.
04:13However, there's also a number
04:14of limitations.
04:17So firstly,
04:19DSM defined symptom domains such
04:21as these defined for autism
04:22might not reflect how clinical
04:24features naturally cluster together.
04:27So for example, the DSM
04:28groups sensory sensitivities,
04:30repetitive movements, and insistence on
04:32sameness together under restricted and
04:34repetitive behaviors.
04:36But research,
04:37is suggesting that these features
04:39actually represent distinct dimensions and
04:41might have different correlates
04:43and also have different underlying
04:44mechanisms.
04:47So next, increasing evidence is
04:49also suggesting that many clinical
04:50features are dimensional in nature.
04:53So this means that they
04:54vary in degree across the
04:55whole population rather than being
04:56simply present or qualitatively
04:58different in those who have
05:00a diagnosis
05:01and absent or qualitatively typical
05:03in those who don't have
05:04a diagnosis.
05:06And then finally, and importantly
05:07for our study, DSM defined
05:09symptom domains don't always correspond
05:11clearly to underlying mechanisms.
05:14And this might reflect the
05:15fact that these domains are
05:16grouping together mechanistically
05:18distinct behavioral domains into a
05:20single or unified category.
05:25So these issues have important
05:26implications for how we assess
05:28clinical phenomena.
05:29So most of the clinical
05:30measures that we currently are
05:32using were developed within a
05:33DSM based conceptual framework.
05:36In many cases, the items
05:37within these questionnaires were specifically
05:39designed to map onto DSM
05:41defined domains,
05:42and the scoring structures were
05:43built around the assumption that
05:44these domains reflect coherent underlying
05:46constructs.
05:48However, if the DSM defined
05:49domains themselves don't reflect coherent
05:51constructs, as the research is
05:52suggesting, then the questionnaire composites
05:54derived from them might not
05:56either.
05:59So as a result our
06:00clinical questionnaires are grouping together
06:02behaviours that might look similar
06:03clinically or that fall within
06:05the same DSM domain but
06:07that are actually driven by
06:08different underlying biological mechanisms.
06:11So ultimately this might be
06:12limiting our ability to link
06:14those behavioral features to underlying
06:16biological mechanisms.
06:20Given the limitations of categorical
06:22systems, there has been increasing
06:23interest in alternative frameworks.
06:26Broadly speaking, these frameworks propose
06:27that we should move away
06:28from categorical diagnosis
06:30and instead conceptualise clinical phenomena
06:32in terms of functioning within
06:34discrete dimensions and systems that
06:35cut across traditional diagnostic boundaries.
06:39So there are a number
06:40of key differences between categorical
06:42models and transdiagnostic
06:43models.
06:44So firstly,
06:45whereas the DSM might ask
06:46whether someone meets criteria
06:49for a diagnosis based on
06:50the presence or absence of
06:51clinical features,
06:52transdiagnostic models conceptualize the same
06:54clinical features as varying in
06:56degree across individuals.
06:58And rather than reducing this
06:59information to a yes or
07:01no diagnosis,
07:02that dimensional profile itself becomes
07:04the focus of characterization and
07:05the focus of study.
07:09Next, where the DSM groups
07:11symptoms based on how they're
07:12appearing clinically, transdiagnostic
07:14models aim to group symptoms
07:15based on the underlying processes
07:17that might be driving them.
07:19And finally whereas the DSM
07:21focuses on what makes diagnoses
07:22different from one another, transdiagnostic
07:25approaches focus on features that
07:26are shared across conditions.
07:30So a number of transdiagnostic
07:32dimensional models have been put
07:33forward, but one that's been
07:34particularly influential is the research
07:36domain criteria or RDoC.
07:39So this framework was introduced
07:41by the National Institute of
07:42Mental Health in two thousand
07:44and ten and has the
07:45ultimate goal of progressing precision
07:47medicine in psychiatry.
07:50So rather than organizing clinical
07:52phenomena around diagnostic categories,
07:54RDoC conceptualizes psychopathology
07:56in terms of functioning across
07:58core dimensions of human behavior
07:59and neurobiology.
08:02And currently, RDoc defines six
08:03core dimensions. These are the
08:05negative valence system, positive valence
08:08system,
08:09systems for social processes,
08:10cognitive systems,
08:12arousal regulatory systems, and then
08:14sensory motor systems.
08:16Each of these domains comprises
08:17a number of more specific
08:18constructs and sub constructs, and
08:20these can be measured across
08:21units of analysis from the
08:22brain to behaviour.
08:25So for example, one that's
08:26particularly relevant to autism is
08:28systems for social processes,
08:30And these are processes that
08:31underlie responses to interpersonal and
08:33social stimuli
08:34and includes the constructs of
08:36attachment and affiliation, social communication,
08:38and perception and understanding of
08:40the self and others. And
08:41each of these constructs also
08:43comprises sub constructs, and they
08:45can be measured across the
08:46universe- and units of analysis
08:48shown on the slide there.
08:52So although frameworks like RDoC
08:55sound good in theory, operationalizing
08:57them in practice remains a
08:58major challenge,
09:00And this is because to
09:01operationalize our Ad hoc, we
09:02need to have measures that
09:03capture the dimensional constructs that
09:05are defined within their framework.
09:07However, most of the clinical
09:08questionnaires that we currently use
09:10weren't developed to capture the
09:11more fine grained and biologically
09:13informed constructs that Ad hoc
09:14emphasizes.
09:16And there have been no
09:17measures specifically developed to capture
09:19RDoC constructs.
09:22So my research then is
09:24broadly focused on addressing this
09:25gap, by developing measurement approaches
09:27that can support the implementation
09:29of transdiagnostic
09:30dimensional frameworks like RDoC in
09:32research and clinical practice.
09:35And today I'm going to
09:35be sharing a project that
09:36I've been working on during
09:37my time at Yale that
09:38examines whether RDoC systems for
09:40social processes
09:41could be derived from clinical
09:42questionnaires that were originally developed
09:44within a DSM based framework.
09:50In doing so, my broader
09:51goal in this research is
09:52to facilitate a more comprehensive
09:54dimensional characterization of social functioning
09:56and autism,
09:57to further evaluate the utility
09:59of RDoC as an organizing
10:00framework, and also to provide
10:02new ways that we might
10:03be able to leverage our
10:04existing datasets for dimensional and
10:06biologically informed research.
10:11So we used data from
10:12the Autism Biomarkers Consortium for
10:14Clinical Trials or ABCCT,
10:16which is a multi site
10:17cohort that's collected across five
10:19sites in the United States.
10:21This sample was deeply phenotyped,
10:23and the dataset included multiple
10:24caregiver reported
10:26clinical questionnaires alongside EEG and
10:28eye tracking assessments, which made
10:29it really well suited for
10:30our analysis.
10:32And our sample included four
10:34hundred and eighty children who
10:35had a DSM diagnosis of
10:36autism,
10:38they had a mean age
10:39of eight and a half
10:39years, sixty per sixty sorry.
10:42Seventy six percent were male
10:43and sixty six percent identified
10:45as white.
10:48Our first step in our
10:49procedure was to canvas the
10:51existing caregiver report clinical measures
10:53that were included in the
10:54ABCCT protocol
10:56to determine which RDoC constructs
10:58and RDoC, RDoC sub constructs
11:00were represented by their items.
11:03So among the available questionnaires,
11:05we found that five included
11:06items that were relevant to
11:08Rdoc's social processes, and those
11:09are listed on the slide
11:10there.
11:11And across those five,
11:13questionnaires, we found,
11:16fifty eight items that represented
11:18different constructs in Rdoc- and
11:19Rdoc sub constructs.
11:24So our next step was
11:25to analyze the item content
11:27of each of those items
11:29and determine which RDoC domain
11:31they best corresponded to.
11:33And through that process, we
11:34found that we had items
11:36representing
11:37five RDoC constructs and sub
11:39constructs. So we had items
11:40representing attachment and affiliation,
11:43three sub constructs within the
11:44social communication,
11:46broader construct being the production
11:48of facial communication,
11:49production of non facial communication,
11:51and reception of communication combining
11:53facial and non facial, and
11:54then finally we had items,
11:57reflecting the understanding mental states
11:59subconstruct, which is in within
12:01the broader perception and understanding
12:02of others construct.
12:05And next, we applied factor
12:07analysis.
12:08So this is a statistical
12:10technique that's used to identify
12:12latent or unobserved dimensions
12:15underlying observed behaviors or variables.
12:17So in our case, we
12:18are identifying
12:19unobservable dimensions of social functioning
12:22from observable behaviors as indicated
12:24by our questionnaire items.
12:27For our analysis, we used
12:29confirmatory application of exploratory structural
12:31equation modeling or ERCM,
12:33and we tested models that
12:34specified the five hypothesized rdoc
12:37dimensions as well as bifactor
12:38models that included both those
12:40specific
12:41dimensions as well as a
12:42general factor,
12:43and this general factor, it
12:45essentially reflects a broad overall
12:47dimension of social functioning
12:49or might be represented as
12:51a total score in standard
12:53scoring.
12:56So what we found was
12:57that our initial model with
12:58those five specific factors demonstrated
13:01good overall fit, however our
13:03items didn't discriminate well between
13:05the production of facial communication
13:06versus the production of non
13:08facial communication,
13:09So we combined those items
13:11into a singular production of
13:12communication factor and then revised
13:14the model and then reran
13:15it,
13:16and our revised model with
13:18four specific factors demonstrated good
13:20fit, but it was further
13:22improved when we included a
13:23general factor as well.
13:26So this is our final
13:27model on the slide here,
13:28so the yellow boxes represent
13:30those individual questionnaire items and
13:32the ovals represent those latent
13:33dimensions.
13:35I realize this text is
13:36quite small, particularly for those
13:37yellow boxes, and I absolutely
13:39don't expect you to read
13:39them. The key point is
13:41that the final structure included
13:43both shared variance across all
13:44of those items as reflected
13:46in our general factor, as
13:47well as more specific dimensions
13:49being attachment affiliation, production of
13:51communication,
13:52reception of communication, and understanding
13:54mental states.
13:58So taken together, our results
14:00suggest that existing DSM based
14:02clinical questionnaires may provide us
14:03with a practical means of
14:05operationalizing
14:06dimensions of functioning that RDoC
14:07conceptualizes,
14:09and that the dimensions we
14:10have provide a more fine
14:12grained characterization of social functioning
14:14than DSM defined domains and
14:16the broader composites typically used
14:17within our existing questionnaires.
14:20And importantly, because RDoC constructs
14:22are conceptualized as reflecting underlying
14:24biological systems,
14:26these dimensions
14:27may better capture separable underlying
14:29biological
14:30processes.
14:32And this led us to
14:33our next question, which was:
14:34do these dimensions actually relate
14:36to biological measures in meaningful
14:38and distinct ways?
14:40And in doing so, do
14:41they improve our alignment between
14:42behavioral measures and underlying biology?
14:47So one of the central
14:48ideas within the RDoC framework
14:50is that the same constructs
14:51should be measurable across multiple
14:53units of analysis.
14:55So if the dimensions we
14:56identified are meaningful, we would
14:57expect them to relate to
14:59biological measures in systematic and
15:00potentially distinct ways.
15:04So to begin evaluating this,
15:05we examined associations between the
15:07dimensions identified in our factor
15:08models and physiological measures of
15:11social functioning,
15:12being the EEG and eye
15:13tracking biomarkers within the ABCCT.
15:17So these analyses were conducted
15:18using the same ABCCT dataset
15:21in a subsample of two
15:22zero four children with autism
15:24while we were controlling for
15:25differences in age, sex and
15:26IQ.
15:29And what we found was
15:30that each of our RDoC
15:32factors correlated significantly with one
15:34or more biomarker.
15:36However, what was more exciting
15:37is that rather than all
15:38of the social processes relating
15:40to biology in the same
15:42way, we saw a mix
15:43of shared and distinct associations
15:45across the biomarkers,
15:47and each colored tick here
15:48is reflecting a different biomarker.
15:51So this pattern is suggesting
15:52that these dimensions might be
15:54capturing different aspects of social
15:55functioning that are at least
15:57partially biologically
15:58separable.
16:02And when we compared our
16:03correlations to those derived using
16:04standard questionnaire composite scores our
16:06RDoC dimensions appear to provide
16:08clearer, stronger, and more differentiated
16:10relationships with the biomarkers.
16:13And even though these standard
16:14composites are all intended to
16:15broadly capture the same thing,
16:17being some aspect of social
16:18functioning,
16:19they do so by collapsing
16:20across many distinct behavioral domains.
16:22So in doing so, they
16:24might be obscuring these important
16:25differences in how the behaviors
16:27are relating to underlying biology.
16:29So in other words, our
16:31article aligned dimensions might provide
16:33a means of more precisely
16:34pinpointing which behaviors correspond to
16:36which biomarkers.
16:40So I think there are
16:41three broader takeaways from this
16:43work. So firstly that RDoC
16:45provides a more fine grained
16:46and biologically informed framework for
16:48conceptualizing clinical phenomena,
16:51then the DSM defined domains
16:52embedded within many of our
16:54existing clinical measures.
16:57Second, these findings are suggesting
16:58that our existing clinical
17:00questionnaires could potentially be reinterpreted
17:03dimensionally
17:04rather than being limited to
17:05their original scoring structures.
17:07So this means that, we
17:09might not need entirely new
17:10datasets or entirely new measurement
17:12tools to begin moving towards
17:13dimensional and transdiagnostic
17:15approaches.
17:16Instead, we might be able
17:17to leverage the data and
17:18measures that we already have
17:19in new and more informative
17:21ways.
17:23And finally, this work suggests
17:24that more granular and biologically
17:26informed clinical questionnaires
17:28might improve our ability to
17:29identify biological mechanisms, underlying observable
17:32behaviour, and also strengthen links
17:34between behaviours and biological measures.
17:39Okay. So I'd like to
17:40thank the families who participated
17:42in this research, and I'd
17:42also like to thank the
17:43Hillebrand Foundation, ABCCT team, and
17:46the McPartland Lab for all
17:47of their support, collaboration, and
17:48mentorship, and thank you all
17:49for listening. Happy to take
17:51one question.
18:05Very interesting.
18:07One question that I have
18:08is, how did you handle
18:09sex in those initial factor
18:11models? And I guess as
18:12part of that question, did
18:13you test for measuring variance
18:16across males and females?
18:18Yeah. Thank you. I'm not
18:19sure the mic is working,
18:20so I'll repeat it. So
18:21you asked how I handled
18:22sex in the factor models
18:23and whether I tested in
18:24variance,
18:24across the factor models.
18:26Yes. I tested in variance.
18:28So with the factor models,
18:29I derived them in the
18:30whole sample with both of
18:31the sexes,
18:32combined, and then I ran
18:34in variance testing across,
18:35age, sex, and also IQ,
18:38and results supported
18:40strict in variance, across all
18:42of those factors. So, yeah,
18:43we can we can assume
18:44that the factor structure and
18:46also the loadings and intercepts
18:48are all stable across sex.
18:50Yeah.
18:54K.
19:04Awesome. So, nice to meet
19:05everyone. My name is Joey
19:07Hefner. I'm really excited to
19:08present some of the work
19:09that I've been doing here
19:10at Yale, in the psychology
19:11department. So I'm gonna be
19:12talking a little bit about
19:13what language reveals about depression
19:15trajectories
19:15and the impacts of human
19:17AI conversations on well-being.
19:19So clinicians have long known
19:21that language acts as a
19:22window into mental states and
19:23one of the most promising
19:24approaches is to develop prediction
19:26tools for clinical disorders such
19:28as depression,
19:29through the use of naturalistic
19:30language. So prior research has
19:32shown that emotional language expressed,
19:33for example, on social media
19:35platforms such as Facebook or
19:36Twitter can predict depression status
19:38in electronic health records.
19:39We also know from EMA
19:41data, that sentiment expressed in
19:43private text messages between friends
19:45improves the prediction of, depression.
19:48And lastly, it's been shown
19:49that, language and therapy contexts
19:51are associated with treatment outcomes.
19:53So for example, reduction of
19:55use in first person pronouns,
19:57predicted better treatment responses on
19:58the platform Talkspace.
20:00So together, these studies suggest
20:01that it may be possible
20:02to build tools that predict
20:03changes in,
20:05depression from language, but these
20:06studies rely on really large
20:08corpuses of data, so your
20:09entire Facebook feeds, for example,
20:11or they rely on really
20:12sensitive personal information like your
20:13private text message or the
20:14therapy transcripts.
20:15So this led us to
20:16a simple question, which is,
20:18can we predict these changes
20:19in depression using the emotional
20:20tone index, but from a
20:22shorter task?
20:23So we developed about a
20:24ten minute task, to see
20:25if it contained the amount
20:26of variance that we needed
20:27to understand these changes in
20:29depression.
20:30So this approach was led
20:31by a now clinical PhD
20:32student in the psychology department
20:34here at Yale, Ji Hyun
20:35Hur. So we what we
20:36did was we wanted to
20:37adopt the PHQ nine, which
20:39is a validated short form
20:40questionnaire assessing depression severity in
20:42the general population, and we
20:43wanted to do it to
20:44open ended responses.
20:46So just to show you
20:47two items in the PHQ
20:48nine, these are the cardinal
20:49symptoms for depression. So the
20:50first one,
20:51asks you to rate how
20:52often you've been bothered by
20:53the following problems, little interest
20:54or pleasure in doing things
20:56over the past two weeks
20:57or feeling down, depressed, or
20:58hopeless is two separate items.
21:00And our goal was to
21:01translate these into neutrally toned
21:03questions that allow participants to
21:04express positive or negative sentiment
21:06regarding these symptoms. So the
21:08open ended versions of these
21:09are could you describe your
21:10gender mood in the past
21:11two weeks, and, you know,
21:12participants type or write a
21:14bit. And then the second
21:15one is how would you
21:16describe your level of interest,
21:17and things in the past
21:18two weeks.
21:19So the goal is that
21:20maybe this can contextualize participants'
21:22ratings that they're giving you
21:23on the PHQ nine, which
21:24are designed to follow the
21:25DSM five,
21:26symptoms and, you know, contextualize
21:28what those depression severity or
21:30symptoms mean in their daily
21:31life.
21:32So can we predict the
21:34change? For the methodology,
21:35we had online participants do
21:37an initial study where they
21:38completed the same questionnaire, the
21:40PHQ nine, as sort of
21:41our metric of how, their
21:42depression severity is for their
21:44symptoms, as well as the
21:45open ended questions that we
21:46developed based on the PHQ-nine.
21:49And then we followed up
21:50with them three weeks later
21:51where they just completed the
21:52PHQ-nine
21:53again so we could assess
21:54that follow-up depression severity.
21:56We did this over two
21:57samples. So, in study one,
21:59you can see the retention
22:01rate was about ninety percent
22:02over the three week period
22:03which is great. And for
22:04study two, it's a little
22:05lower about seventy two percent.
22:07And one thing to clarify
22:08about the use of study
22:09two is there was a
22:10small difference. We did want
22:12to assess the PHQ nine
22:13and the open ended questions
22:14on different days just to
22:15avoid having people make direct
22:17comparisons to them. So for
22:18the initial portion in study
22:20two, participants did both of
22:21those a day apart. So
22:22we did lose a little
22:23bit of retention, by recruiting
22:24four hundred and only three
22:25hundred and twenty four did
22:26both of them on a
22:27day to day basis.
22:29So this is our, our
22:30studies that we're gonna be
22:31analyzing over. So for the
22:33results on the x axis,
22:34I'm showing you the beta
22:35coefficients from a regression model
22:37where we're predicting the follow-up
22:39PHQ nine score as a
22:40function of the initial PHQ
22:42nine score plus the human
22:43sentiment score. And what that
22:44means is that is the
22:45average emotional tone as rated
22:47by external,
22:48humans for the free text
22:50that you gave. So we
22:52take all nine items, we
22:53average them, and that average
22:54score represents how positive relative
22:56to negative that text is,
22:58for that individual.
23:00So,
23:00unsurprisingly,
23:02initial PHQ nine is highly
23:03correlated with the follow-up so
23:05this is a very,
23:06strong predictor. So the critical
23:08part is does the human
23:08sentiment score add, predictive variance?
23:11And the answer is that
23:12it does for both, studies.
23:14So in other words, if
23:15you had two people who
23:16had the same initial PHQ
23:17nine score, if one of
23:19them talked more positively about
23:20their depression symptoms and one
23:21talked more negatively, it predicts
23:23the person who talked positively
23:24will have a decrease in
23:25depression severity after a three
23:27week period and the other
23:28will have an increase.
23:29Just to show this in
23:30a slightly different way, I
23:31can plot whether the model
23:33is predicting a PHQ nine
23:34increase, no change, or a
23:35decrease and showing you the
23:37actual PHQ nine change on
23:38the x axis. And you
23:40can see that you get
23:40about a one point five,
23:42change in the PHQ nine
23:43score. For reference, the total
23:45is, twenty four for the
23:46most severe depression and zero
23:48for the least.
23:49So in short, I think
23:50this is good evidence that
23:51the negative emotional tone,
23:53in these descriptions of your
23:55depression symptoms, predicted increased depression
23:58severity three weeks later.
24:00Now the way we did
24:00this is we had external
24:02human raters.
24:03They take a long time
24:03to collect. They're kind of
24:04expensive. And so we wondered,
24:06could we automate this process
24:07using the leading models at
24:08the time?
24:10So can we automate this?
24:11For those who are familiar
24:12with text analysis,
24:13the Luke twenty two was
24:15the leading one. Yeah.
24:19The Luke two is a
24:20dictionary based approach. So the
24:21way that it works, just
24:22to give you one real
24:23participant example, this is them
24:25giving their open ended response
24:26to that first question. So
24:27they say my move has
24:28been lower in the past
24:29week but it was a
24:30bit better the week before.
24:31I find that it fluctuates
24:32at the moment. I've been
24:33feeling tired more recently.
24:34What the loop does is
24:36it looks for key emotional
24:37terms based on a predefined
24:39dictionary so we can have
24:40a positive emotion percentage and
24:41a negative emotion percentage.
24:43In this case, only the
24:44word tired triggers a connection.
24:46So the rating that loop
24:47gives you is this is
24:48zero percent positive on the
24:49expression but three percent, negative,
24:52which is that what is
24:53the total proportion of emotion
24:54words to the total number
24:55of words you used. So
24:56that's the metric that Luke
24:57used, which was the leading
24:58one at the time.
24:59And at the time we
25:00were collecting the data, chat
25:01g b t had just
25:02been released, so we wanted
25:03to compare this against that
25:04model. So this was done
25:05in September of twenty twenty
25:07three for reference and using
25:08chat gbt three point five.
25:11And so what we did
25:11was we gave chat gbt
25:13these instructions to sort of
25:14map onto the loop, mappings.
25:16So give us two integer
25:18ratings from zero to ten,
25:20on both scales. So for
25:21example, the chat gbt rated
25:23this, utterance as a two
25:25out of ten on positivity,
25:26so not very positive, and
25:27a seven out of ten
25:28on negativity.
25:29So we're basically just seeing
25:30can the, outputs of these
25:32two programs
25:33recapitulate the pattern of results
25:35that we saw with our
25:35human raters.
25:37So if we first look
25:38at the loop twenty two,
25:39we have the same regression
25:40model. We're just replacing the
25:41sentiment score with the loop
25:42prediction.
25:43And we can see that
25:44the loop model is unable
25:45to explain any variance in
25:47follow ups on the, depression
25:49status. So that means that
25:50just getting the key emotional
25:52words isn't enough in order
25:53to predict changes in depression,
25:55at least using the small
25:56amount of data that we
25:56have. Luke tends to use
25:58larger corpuses for example.
26:00What about chat GBT?
26:01Well chat gbt was able
26:03to recapitulate the same pattern
26:04of results that we saw
26:05so you get additional variance
26:06but this time it was
26:07just done in an automated
26:08way.
26:09So why why would this
26:10be? And the short answer
26:12is that chat gbt sentiment
26:13ratings are highly correlated with
26:15our external human raters. So
26:16in our data set as
26:17well as some others that
26:18we've done, the correlation's around
26:20point nine five, point nine
26:21six which is quite strong,
26:22whereas for reference the Luke
26:24correlation is only about point
26:25six. So in other words,
26:27you miss a lot of
26:27the variance when it comes
26:28to sentiment ratings by not
26:30taking into account the entire
26:31context
26:32of what's being said not
26:33just the emotional words.
26:36One thing to note is
26:37that I think Chattopbt is
26:38really good at this as
26:38well as other models which
26:39I'm happy to talk about.
26:41For this dataset this costs
26:42us less than a dollar,
26:43for doing the sentiment analysis
26:44so I think it's a
26:45really promising approach for clinicians
26:46to begin considering if they
26:48want to do this.
26:50So we've covered one way
26:51that language can be used
26:52as a window in our
26:53mental states and given what
26:55we just talked about one
26:56obvious extension is to consider
26:57how human AI interactions
26:59are gonna be playing a
27:00role. So I'm sure many
27:02of you have had conversations
27:03of both personal and professional
27:04use of how we're gonna
27:05use these large language models
27:06we have in the t32,
27:08meetings.
27:09And, the reason this is
27:10really important to consider is
27:12that a recent review,
27:13relatively recent,
27:15is showing that the use
27:16case of these LLMs, emotional
27:18topics are on the rise.
27:19So if you look at
27:20the top five use cases
27:21in twenty twenty four, therapy
27:23and companionship is there, but
27:24there's a lot of other
27:25things like generating ideas or
27:26using it as a search
27:27engine or exploring topics. Whereas
27:29you can see in twenty
27:30twenty five, the dominant use
27:32cases are for this personal
27:33and professional support. So people
27:35are using these for, you
27:36know, assistance in this way.
27:37So we had a very
27:38simple question, which is how
27:40do human AI interactions impact
27:42people's well-being?
27:44So for our experimental design,
27:46we,
27:47derived a list of topics
27:48that people were gonna talk
27:49about from some self disclosure
27:51research.
27:52The goal here is just
27:53that it varies in positivity.
27:54So some topics like talking
27:56about things you're grateful for
27:57or something you're proud about
27:58in your life are quite
27:59positive.
28:00And you know the more
28:01negative ones are, talking about
28:03a time you felt guilty
28:04or a time someone hurt
28:04your feelings for example.
28:07The the way the methods
28:08work is we had one
28:09group of participants engage in
28:10chatbot conversations. This was with
28:12GPT four in January twenty
28:14twenty four, just for reference.
28:16And the people in the
28:17chatbot conversations are gonna have
28:18three topics randomly picked. This
28:20is just showing you one
28:21example subject where you're gonna
28:22talk for about that topic
28:23for about five minutes in
28:24a back and forth sort
28:25of AOL style exchange.
28:27Critically, after each conversation, people
28:30give us their happiness ratings.
28:31This is what we're gonna
28:32use to understand the impacts
28:33that it's having on, momentary
28:35happiness.
28:35This is a scale that
28:36we often use in the
28:37lab and we can do
28:38computational modeling on this. So
28:39it's it's very simple. You
28:40just rate how happy you
28:41are now on a zero
28:42to a hundred scale ranging
28:44from very unhappy to very
28:45happy.
28:46And although this isn't the
28:47same as depression, I should
28:48note that baseline happiness across
28:50a variety of our tasks
28:51is strongly predictive of PHQ
28:53nine score.
28:54For our comparison group, we
28:56chose to use journaling, in
28:57isolation about the topics. People
28:59don't like to journal as
29:00long, about these topics. They
29:02kind of fall off after
29:03about a minute. So we
29:04have people just do the
29:05entire topics in a randomized
29:07order for about one minute
29:08and again we get that
29:09happiness rating after
29:10each one. So journaling is
29:12thought to be, pretty important
29:13for things like homework compliance
29:15and, CBT therapies as well
29:17as it's a common,
29:18positive well-being intervention. So for
29:20example, having a gratitude journal
29:22is one of the more
29:22popular ones in positive psychology.
29:24So we thought this was
29:25a pretty good comparison just
29:26to see how well our
29:27chatbot conversations are holding up.
29:30So how do chatbots affect
29:31happiness?
29:32In this graph I hope
29:33you generated some expectations about
29:34where you thought the topics
29:36will be ordered, but I'm
29:37gonna be showing the average
29:38happiness for the two different
29:39studies on the x axis
29:40where you have zero very
29:41unhappy, hundred very happy, and
29:43the topics on the y
29:44axis. These are ordered by
29:46the average happiness from the
29:47journaling study. So unsurprisingly
29:49people on average are a
29:50little happier when they're journaling
29:51about gratitude or their perfect
29:53day or something they're proud
29:54about, and they're less happy
29:56when talking about things they're
29:57guilty for they feel guilty
29:58about or times they felt
29:59depressed.
30:00So the key part is
30:01what about the chatbot comparison?
30:03So when I put the
30:04this data up you can
30:05see two patterns emerge. So
30:07the first is that you
30:07do have a pretty big
30:08main effect, so on average
30:09people are just happier, maybe
30:11it's a more engaging task
30:12after all you're having this
30:13interesting back and forth with
30:14an AI agent.
30:16But the more important part
30:17is that there is a
30:18significant interaction here, which is
30:20to say that the group
30:21who did the chatbot conversations
30:23are less affected by the
30:24topic negativity.
30:25Another way to plot this
30:26a little more intuitively is
30:27just to plot the difference
30:29between the average chatbot happiness
30:31and the journal happiness per
30:32topic. And you can see
30:33that the biggest chatbot happiness
30:34boost on the order of
30:35about a fifteen,
30:37fifteen point swing on a
30:38hundred point scale comes from
30:39the most negative topics.
30:41So in short, we see
30:43that chatbots do increase momentary
30:44happiness compared with journaling,
30:46and we see the greatest
30:47benefits occur during discussions about
30:49negative topics in particular.
30:51So we've only been doing
30:52this sort of group level
30:53comparisons but we want to
30:54get some insight into what's
30:55going on in the chatbot
30:56conversations that might be resulting
30:58in this increase in momentary
30:59happiness.
31:00And so the way we're
31:01gonna do that is something
31:02very similar to what we
31:03did before. We're gonna use
31:04an AI assisted sentiment analysis.
31:06So this schematic is showing
31:07you each utterance is sort
31:09of like an AOL style,
31:10thing that you're entering before
31:12pressing enter. So we separate
31:13out the utterances for both
31:14the user and the chatbot.
31:16We feed them into a
31:17separate large language model with
31:18some simple instructions
31:19for giving an integer sentiment
31:21rating and the output of
31:22that is just a single
31:23number for what the utterances
31:24on a zero to ten
31:25scale. So this is the
31:26way we're gonna automate our
31:27sentiment analysis.
31:31Unfortunately,
31:32I should have been disappeared,
31:33but to explain the x
31:34axis, the utterance pairs represent,
31:36each utterance that the user
31:38in the chatbot took in
31:39order going from one all
31:40the way to six. Ninety
31:41five percent of conversations ended
31:43after
31:45about six, utterances, so it's
31:45sort of showing you the
31:46full range. And the sentiment
31:47as rated by an external
31:48LM is shown on the
31:49y axis. And the critical
31:50part here is you can
31:52see, the sentiment is increasing
31:54on average throughout the entire
31:55corpus. So the user is
31:57starting around let's say five
31:58point five but they're ending
31:59at about a little under
32:00seven. So we think this
32:02chat this sentiment boost is
32:03actually a key part to
32:04why individuals in the chatbot
32:06condition are ending up a
32:07little happier than those in
32:08the journaling.
32:10Just to show you this
32:11broken out by topic in
32:12case you're interested, so the
32:14dynamics are for the average
32:15across all the topics, but
32:16you can see for some
32:17of the positive topics, it's
32:18really hard to maintain a
32:20ten out of ten sentiment
32:21the entire time. So there's
32:22a little bit of a
32:23dip for the user when
32:24it comes to their last,
32:25last sentiment utterance versus their
32:27first. But again, you can
32:28see the biggest impacts are
32:29for those most negative topics.
32:31So you get a really
32:31large difference in the sentiment,
32:33for example, discussing depression, and
32:34we think this is key
32:35for why you get the
32:36happiness boost, but I'm happy
32:38to take more questions about
32:39it. So just to summarize
32:40this,
32:41we show some evidence that
32:42the emotional tone becomes more
32:44positive across the conversation. We
32:45also have evidence that we
32:46think for mirroring where the
32:47user and the chatbot are
32:49both mirroring the other person.
32:51And again you see the
32:52strongest improvements in sentiment were
32:53discussions for the negative topics,
32:55which we think is key
32:55for the happiness boost.
32:58So just to summarize,
32:59and wrap up, I just
33:00want to leave you with
33:01this idea that language is
33:02a really powerful window into
33:03our mental states. And I
33:04hope I've shown how AI,
33:06specifically tools like sentiment analysis,
33:08can make sense of language's
33:09relationship to mental health. If
33:11you're interested in exploring this
33:12further, I will be leading
33:13a one hour workshop using
33:15open source models for sensitive
33:17clinical data that will be
33:18held at the computational psychiatry
33:19conference here at Yale, which
33:21is going on from July
33:22fourteenth to sixteenth. So look
33:23online for that.
33:25And finally, I also hope
33:26that this talk complements some
33:27of the ongoing discussions about
33:29the potential impacts both the
33:30benefits and the risks of
33:32human AI interaction. I think
33:33this is especially critical for
33:34vulnerable populations like adolescents where
33:36AI use is growing rapidly
33:38so So we need more
33:38research to understand both the
33:40short and long term impacts.
33:42So I'd just like to
33:43acknowledge the Rutledge lab and
33:44my advisor Rob Rutledge who
33:45let me start this line
33:46of research as well as
33:47thanks to both Michaels and
33:48the child studies center for
33:49t thirty two for supporting
33:50this. And I want to
33:51end by announcing I will
33:53be starting as an assistant
33:54professor in psychology department at
33:55Stanford in the fall of
33:56twenty twenty seven. So if
33:57you know anyone who's interested
33:58in this kind of work,
33:59have them look out for
34:00hiring announcements for lab manager,
34:02graduate student, and postdoc. Thank
34:04you all for your attention,
34:05and this opportunity.
34:13Questions for doctor Apner?
34:21Wonderful talk. Really impressive.
34:24I'm wondering,
34:26you used the, sentiment analysis,
34:28and it it's it seems
34:29to be like a pep
34:30talk.
34:31And I'm wondering,
34:32your thoughts about using this
34:34for other,
34:35walks of life. You know?
34:36Just imagine my my kid
34:38struggling with sports and and,
34:40you know,
34:41having a tough time on
34:42the field. And, you know,
34:44do you think that this
34:44type of approach could be
34:46or or could guide parents,
34:47for instance, to coach them
34:48on, how to improve their
34:50sentiment, that type of thing?
34:51Thanks.
34:54Ashid.
34:55Thanks for the question. Yeah.
34:56I think,
34:57as far as implications for
34:59that kind of work, I
35:01think what I what I
35:02think about is I think
35:03chatbots have a certain way
35:04of increasing our momentary happiness,
35:06and part of that is
35:07through increasing in our sentiment.
35:09I didn't discuss this, but
35:10I also think it's driving
35:11people towards a solution. So,
35:13like, figuring out a solution
35:14to the problem that you're
35:15talking about, and I think
35:16that's part of the way
35:16chatbots have been reinforced. So
35:18I think it's one path
35:19to making someone happier, but
35:20I think there's also lots
35:21of other paths that clinicians
35:21in their own clinicians in
35:22their own can probably think
35:23about in terms of making
35:24people happier. So one of
35:25the goals that we're trying
35:26to do is to compare
35:27human AI conversations in the
35:29same topic list to human
35:30to human. And I think
35:31one of the key things
35:32that people can do that
35:33chatbots are explicitly designed not
35:35to, although in theory, they
35:36could, is, reciprocate.
35:37So I can tell you
35:38about a time I you
35:39know, someone hurt my feelings
35:40and that sort of reciprocal
35:42bonding, even though it's of
35:43a low sentiment, might actually
35:44result in some boosts both
35:46in the short term and
35:46long term. It's just not
35:48something we'll see in the
35:49default chatbot models, but it
35:50is a potential. So I
35:51guess the way I would
35:52think about it is there's
35:53methods the chatbots might be
35:54using that differ or complement
35:56the way humans are doing
35:57it.
35:58So I don't know if
35:58that answers, but that's what
35:59I think about it. Thanks
36:00for the question.
36:22This is gonna be a
36:22little different. I'm talking a
36:23lot about what I'm building,
36:26and I feel like a
36:27little bit of an imposter
36:28standing up here after Lacey
36:29and Joey presented really impressive
36:31work.
36:32For those who don't know
36:33me, I'm Max Rolison. I'm
36:34a chief fellow in the
36:35Sonnet Integrated program.
36:38I've been lucky this year,
36:40as part of the flexibility
36:41of the program to be
36:42able to do kind of
36:44really specific training in neurodevelopmental
36:46disabilities. And after this year,
36:48I'll be joining the faculty
36:49at Boston Children's Hospital this
36:51summer.
36:54Today, I wanna talk to
36:55you about a problem that
36:56sits at the intersection of
36:57clinical care and research
36:59infrastructure for the patients. I'm
37:01gonna describe people with autism,
37:02intellectual disability, and related genetic
37:05conditions,
37:06where we're really missing fundamental
37:08data that the rest of
37:09medicine takes for granted. And
37:11the talk is gonna be
37:12about why this gap exists,
37:13what it costs us clinically,
37:15and what a concrete solution
37:17looks like and kinda what
37:19I'm building.
37:20So
37:21I wanna
37:22start with acknowledging all of
37:24the people that have gotten
37:25me to this point. I
37:26have spent the past fifteen
37:28years at Yale, in the
37:30child study center. I came
37:31at seventeen
37:32after my junior year in
37:34high school. So it's a
37:35very bittersweet transition right now
37:37as I feel like, I
37:38don't know, like, I'm graduating
37:39high school and growing up
37:40finally.
37:42But,
37:43you know, there's so many
37:44people in the room in
37:45the child study center who
37:46have intellectually shaped this work
37:47and how I got here,
37:50who are pictured here,
37:52as well as kind of
37:53funding opportunities through the Solnit
37:55program, the r twenty five,
37:57the t thirty two, and
37:58a child studies center pilot
37:59award, as well as mentors
38:01at Boston Children's Hospital.
38:05So in terms of learning
38:06objectives for the talk,
38:08I'm gonna talk about
38:10structural barriers, including fragmented measurement,
38:13systematic research exclusion, and poor
38:15generalizability
38:16that limit our
38:18evidence base in this population.
38:20We're gonna talk about measurement
38:22based care embedded in the,
38:23like, the electronic health record
38:24and how it functions not
38:26just as a clinical tool,
38:27but as a research infrastructure.
38:29And then I'm gonna highlight
38:30and talk about catatonia in
38:32neurodevelopmental
38:33disabilities
38:34as a model syndrome that
38:35illustrates precisely kind of what
38:37happens when we're missing this
38:38longitudinal baseline data.
38:41So to start with the
38:42problem, I wanna define the
38:43population that we're talking about,
38:46because it's
38:47matters for everything that kind
38:49of is going to follow.
38:51A lot of this could
38:51be true about pediatric mental
38:53health in general, but this
38:54is the population I'm really
38:55kind of highlighting.
38:56So we have three overlapping
38:58groups. We have autism spectrum
38:59disorder,
39:01where about a third to
39:03half of individuals with co
39:05have co occurring intellectual disability.
39:08And that's not generally the
39:09picture from research samples, but
39:10it's the reality of clinically
39:12when we're seeing these patients
39:13who's showing up to see
39:14a psychiatrist.
39:15We have patients with intellectual
39:17and developmental disability broadly, where
39:19psychiatric comorbidity is very high,
39:21and patients often can't self
39:23report their symptoms in the
39:24ways
39:25the stand our standardized assessments
39:27assume.
39:29I think we need to
39:30kind of reframe how we
39:31think about this population rather
39:32than thinking about psychiatric comorbidity
39:34and really think about this
39:36as kind of a holistic
39:38psychiatric symptoms or core to
39:39the underlying pathology.
39:41And then specific genetic conditions
39:42associated with neurodevelopmental
39:44disorders, syndromes like Phelan McDermid,
39:46Syngap, Rett Syndrome, CDK, L
39:49five, and twenty two q
39:50eleven point two, where we
39:51increasingly understand the biology,
39:54but we don't have a
39:55good treatment base for it.
39:57I also wanna highlight
39:59where our evidence base stands
40:00in this population.
40:02So right now, there are
40:03two FDA approved medications for
40:05autism,
40:06both addressing a single behavioral
40:08domain broadly.
40:10And it's the entire regulatory
40:12evidence base for a condition
40:13affecting roughly one in thirty
40:14one kids, separate from the
40:16genetic syndromes and the intellectual
40:18disability.
40:19And it's not a failure
40:20of scientific interest. It's really
40:22a structural failure, and that's
40:24what I'm gonna talk about.
40:26So we
40:27you know, why is the
40:28evidence base so thin? And
40:29argue it comes down to
40:31three failures that compound each
40:33other in a reinforcing cycle.
40:34Cycle. So the first is
40:36fragmented measurement. So there's no
40:37standard approach to data collection
40:39for these patients.
40:40Clinicians use different tools, different
40:42rating scales, or no structured
40:44measures at all. The results
40:45is that there's no longitudinal
40:47baseline from which to detect
40:49change. If a patient with
40:50severe or profound autism presents
40:52in crisis,
40:53you have no objective ground
40:55truth to compare against.
40:57Was this behavior present six
40:58months ago? Is this baseline?
41:00Is this a chronic poor
41:01behavior, or is this an
41:02acute change?
41:03We genuinely
41:04don't know with data to
41:06support us, not because no
41:07one was paying attention, but
41:08because the infrastructure
41:10in our health system wasn't
41:11built to capture it.
41:14The second failure is really
41:15systematic exclusion.
41:17So clinical trials almost universally
41:19exclude individuals with severe intellectual
41:21disability,
41:22nonverbal communication, or significant behavioral
41:25presentations,
41:26which are exactly the patients
41:28who most need empirical evidence.
41:31And the evidence base was
41:32built on people who don't
41:33represent the clinical population of
41:35who's coming to see child
41:36psychiatrists the most.
41:38And then the third failure
41:39is poor translation.
41:40So even the evidence that
41:42we have doesn't generalize.
41:44We have controlled narrow samples
41:45and outcomes that don't match
41:47clinical reality. So clinicians default
41:49to idiosyncratic
41:50practice,
41:51which generates no data, which
41:53means the evidence never improves
41:54and the cycle continues.
41:56So
41:57what does it look like
41:58when this problem has actually
41:59been solved? And I wanna
42:01use pediatric oncology as a
42:02model. And I'm gonna walk
42:04through this comparison deliberately
42:06and start with two columns.
42:07On the left, we have
42:08pediatric oncology, which is the
42:10ACHIEVE standard, and on the
42:11right,
42:12NDD or neurodevelopmental
42:14disorders psychiatry today and the
42:16kind of the clinical reality,
42:17and I'll take you for
42:18through four dimensions one at
42:20a time.
42:21So the first dimension is
42:23measurement. So in pediatric oncology,
42:24we have standardized outcome measures
42:27embedded across essentially all treatment
42:29centers, every patient, every visit
42:31using the same tools.
42:32And then when we think
42:33about NDD psychiatry, we have
42:35non overlapping measures across clinicians,
42:37heavy reliance on free text.
42:39There's no standard. Two clinicians
42:41in the same department may
42:42use entirely different approaches to
42:44capture the same same information
42:46if they capture it at
42:47all.
42:49The second dimension is how
42:50evidence gets built. So in
42:52oncology, the field learns from
42:53nearly every patient, not just
42:55the subset enrolled in trials.
42:57Data,
42:58we can aggregate and becomes
43:00cumulative across time and across
43:02sites.
43:03In our population, most patients
43:05contribute almost nothing to the
43:07evidence base in terms of
43:09learning from each individual case.
43:11The data that does exist
43:12cannot be aggregated or compared,
43:14and we have no way
43:15to accumulate knowledge across encounters.
43:19The third dimension is generalizability.
43:21So oncology generates findings that
43:23reflect the full clinical population,
43:24including complex cases that would
43:26traditionally be excluded from trials.
43:29In psychiatry, findings from narrow,
43:31controlled samples don't reflect the
43:33patients that actually that we
43:35actually treat and then actually
43:36show up to clinic or
43:37the hospital.
43:38And a clinician is trying
43:39to apply trial results to
43:40a patient with severe intellectual
43:42disability and multiple comorbidities,
43:45extrapolating from a population that
43:46didn't include anyone like them.
43:49And then the fourth dimension
43:51and the one that ties
43:52everything together is really the
43:53infrastructure.
43:54So pediatric oncology,
43:56you know, like, around two
43:58thousand
43:59built
44:00a unified infrastructure for the
44:01purpose of this. Because kids
44:03were dying, and they were
44:04like, we need to do
44:05a better job of this.
44:07And it's a unified infrastructure
44:09no matter where you go
44:10that simultaneously supports clinical care
44:12trials and natural history in
44:13terms of what actually happens.
44:15So we have one system
44:16with multiple purposes. And in
44:18NDD psychiatry,
44:20clinical care and research
44:22are completely siloed.
44:23They don't feed each other.
44:24The data generated in clinic
44:26doesn't reach research, and the
44:28evidence generated research doesn't reflect
44:30what clinicians actually see.
44:32And the contrast isn't because
44:34oncology is more important. It's
44:35because they made a deliberate
44:37infrastructure investment a really long
44:38time ago, and that's what
44:40I'm proposing that we do,
44:42which brings me to the
44:43next slide.
44:44So
44:45at Boston Children's, I'm building
44:47a measurement based care system
44:49embedded directly in Epic EHR,
44:51which is
44:52a tool used at, like,
44:54more than half of, like,
44:55all of the hospitals
44:56across the country,
44:58which is
44:59important because it talks speaks
45:01to generalizability.
45:02And,
45:04you know, people like Jamie
45:05and Adam taught me fancy
45:06words like harmonization
45:07of data. And that's been
45:09really important in terms of
45:10how we think about this
45:11from the start.
45:12So
45:13we're building this design from
45:14the ground up for patients
45:15who can't self report. And
45:16I'm gonna walk you through
45:17how it works.
45:20And it starts
45:21with a patient and their
45:22caregiver arriving to a routine
45:24clinic visit.
45:25Nothing special. Nothing extra. Just
45:28a standard appointment.
45:29The whole purpose of this
45:30is that, like, these are
45:31families who
45:33go to the doctor more
45:34than, like, anyone else. Most
45:36of the caregivers I see,
45:38they can't even have jobs
45:39because they're spending all of
45:40their time taking their kids
45:41to doctor's appointments.
45:43But yet we're not collecting
45:44data that can actually help
45:46their kid
45:47in in a larger scheme
45:48for research.
45:50So our goal is really
45:51to reduce burden and do
45:52this in standard care.
45:55So when they check-in,
45:57we're kinda pushing out,
45:59and prompting the caregiver to
46:00complete a structured battery of
46:02validated proxy measures that are
46:04caregiver report instruments that span
46:06the key domains relevant to
46:07our NDD patients in terms
46:09of behavior, function,
46:10motor adaptive skills.
46:12It's automated. It doesn't require
46:14the clinician using their brain
46:15and, like, remembering to do
46:16this among the nine thousand
46:17other things they need to
46:18do. It also
46:20feeds directly back into the
46:22clinical record. So by the
46:24time they make it in
46:24the room, it is built
46:26into your note so that
46:28the clinician is able to
46:29use this, and it's not
46:30like, let me type in
46:32this thing and ask you
46:33this question. It's, like, to
46:34make people's lives easier.
46:36It happens at home when
46:37they have time in the
46:38waiting room on a tablet.
46:40There's no extra visit, no
46:41re separate research consent, and
46:43no additional burden on the
46:44family. It's part of collecting
46:46the data that we're otherwise
46:47collecting in a way that's
46:48organized.
46:53So we kind of are
46:54able to map trajectories. It's
46:56all getting incorporated. There are
46:57really cool ways that we're
46:58starting to do this across
47:00other specialties incorporating genetics, developmental
47:03medicine,
47:04neurology,
47:06and even going down to
47:08really address the question of
47:09developmental milestones,
47:11which when I'm seeing a
47:12kid for the first time
47:12at, like, nine or ten,
47:14they don't remember, like, when
47:16they first had a social
47:17smile or when
47:19they first took steps or
47:21when they lost that if
47:22they did, which, you know,
47:23does come up. So we're
47:25working to build a structured
47:26in the pediatric primary care
47:28centers as well as our
47:29NICU grad clinics
47:31to think about
47:32building a structure that, like,
47:35these people like, we shouldn't
47:36be asking them every single
47:37time they come to a
47:38doctor, when did you take
47:39your first steps? That's not
47:40changing. And if it is
47:41changing, it's because it's inaccurate,
47:43not because it actually changed
47:45historically.
47:46And I think that's a
47:47way that, you know, I'm
47:48pretty excited because I think
47:49that's a really important research
47:50data as we think about
47:51how do we phenotype these
47:52kids.
47:55So here's what the single
47:57infrastructure enables. So on the
47:58left, we have the clinician
47:59opening the chart, seeing a
48:00longitudinal
48:01trend, the patient scores across
48:03twelve visits,
48:05over a year flagging a
48:06meaningful change from their individual
48:08baseline.
48:09That decision support that doesn't
48:10currently exist for most of
48:12these patients, are they getting
48:13better or worse? Most of
48:14the time, I'm kinda like,
48:16Like, maybe.
48:18I don't know if it's
48:19because, like, they're less constipated
48:20or, like, they had a
48:22better day at school.
48:24I'm doing things like putting
48:25them on medication. I can't
48:26really reliably answer. Are they
48:28better or worse? Which I
48:29think data can be really
48:31helpful for. And on the
48:32right, we have a researcher
48:33querying the database and finding
48:35a phenotype longitudinally
48:36characterized population
48:38built from real clinical encounters
48:39without eligibility screens, without excluding
48:42the patients who need it
48:43most,
48:44better care for this patient,
48:45evidence for all patients,
48:47the same infrastructure, no additional
48:49burden.
48:50And that's the oncology model
48:52in a lot of ways
48:53applied to our field.
48:55So because we're capturing
48:58pharmacologic and behavioral interventions at
49:00every visit in Epic,
49:02the same infrastructure allows you
49:04to track treatment and inflection
49:06points
49:06in terms of what actually
49:08happens. Does this treatment work,
49:10or did it not? Is
49:11this the natural history of
49:13this condition and they're just
49:14getting better? Or did we
49:15really do something with this
49:16intervention?
49:17And that's a really important
49:18question.
49:19When we're kind of offering
49:21all of these things,
49:22we need to answer, did
49:24this medicine make a difference
49:25or did it not? If
49:26it didn't make any difference,
49:27I don't wanna keep giving
49:28it to them with all
49:29sorts of side effects.
49:31And this is kind of
49:32how we fuel all these
49:34things together.
49:35So I'm gonna jump to
49:37talk about catatonia,
49:39because I think it's
49:41a model syndrome that really
49:44drives this point home, and
49:45it's something I see a
49:46lot of.
49:48I suspect some of you
49:50in this room have seen
49:51it without recognizing it, but
49:52it's a neuropsychiatric
49:54syndrome of motor behavioral and
49:55autonomic dysregulation.
49:58Typically, we're treating with benzodiazepines
50:00and ECT.
50:01And in NDD, inpatient populations,
50:03in in general, the prevalence
50:05estimates range from roughly ten
50:06to seventeen percent.
50:09And just to walk through
50:10what it looks like,
50:11the motor features are what
50:13most people think of when
50:14they hear the word catatonia
50:15catatonia in terms of stupor,
50:17mutism, rigidity,
50:19but also posturing, waxy flexibility,
50:21stereotypies,
50:23echopraxia or echolalia,
50:25refusal to eat or drink,
50:26and gait disturbance.
50:27They're dramatic findings when they're
50:29present. But in a patient
50:30who already has ASD or
50:31IDD,
50:32they can be subtle or
50:33attributed to the underlying condition,
50:35and there's a lot of
50:36diagnostic overshadowing.
50:38The behavioral features are where
50:40it gets diagnostically
50:42treacherous to say the least
50:43because we see withdrawal, loss
50:44of engagement,
50:46agitation, self injury,
50:47loss of previously
50:49acquired skills, and regression in
50:50activities of daily living and
50:52social withdrawal.
50:54Every one of those features
50:55could be and often is
50:56attributed to behavioral deterioration in
50:58autism.
51:00They're not specific to catatonia.
51:01And in a patient who
51:02can't tell you what's wrong,
51:03the overlap with the underlying
51:05condition makes the diagnosis
51:06genuinely hard.
51:08I hope no one ever
51:09looks at me and says,
51:10like,
51:11Max suddenly, like, stopped using
51:13a toilet after he did
51:14for, like, his entire life
51:15and says, like, oh, that
51:16that's just Max. Because that's,
51:18like, really what happens to
51:19a lot of these kids,
51:20and we're seeing them in
51:21clinic. We're like, that is
51:22not normal. You don't lose
51:23that skill even if you
51:24have autism or intellectual disability.
51:27But that's what happens in
51:28clinical practices. People don't recognize
51:30this.
51:31And why it gets missed
51:32so that we have, you
51:33know, no self report that's
51:34kind of inherent to catatonia
51:36with the mutism.
51:38They overlap.
51:39We have no baseline to
51:41compare against. There's clinician
51:43unfamiliarity
51:44and kind of perhaps most
51:46commonly,
51:47it gets labeled as regression
51:48and it stops there.
51:50And I wanna draw a
51:51specific connection to this audience
51:53because I think many of
51:54you have seen this under
51:55a different name, childhood disintegrative
51:57disorder.
51:58So c d d, you
52:00know, part of DSM four,
52:02has been described for a
52:03long, long time at the
52:05Child Study Center,
52:07is a pattern where we
52:08see a child developing normally
52:09then losing
52:10language, social skills, and adaptive
52:12function,
52:13often dramatically over weeks to
52:15months.
52:16It kind of ended, and
52:17we stopped really following those
52:18kids back in two thousand
52:20thirteen with a switch to
52:21DSM five.
52:22But with work, you know,
52:23it's funny moving institutions and
52:25being like, oh, we're seeing
52:26the exact same thing, but
52:27we're calling it something really
52:28different.
52:29And we're treating it really
52:31differently.
52:32I'm working with, you know,
52:33Abba, Fred, and Lexi
52:36to really validate that many
52:38of these cases were likely
52:39catatonia,
52:41unrecognized and untreated catatonia in
52:43a child who happened to
52:44have kind of an underlying
52:45neurodevelopmental
52:46condition
52:47that made the presentation somewhat
52:49atypical and the diagnosis easy
52:51to miss.
52:52And we're kind of going
52:53back forty years of videos
52:55and coding and validating that
52:57hypothesis.
52:58But we're seeing some of
52:59these kids, and they're coming
53:00to me. And we're able
53:01to get them quite a
53:03bit better,
53:04which is really incredible.
53:10And I think, you know,
53:11this is a prime example
53:12of where
53:14not having structured data is
53:16a barrier to care.
53:18So in practice,
53:20when we don't have structured
53:21baselines,
53:23a patient
53:24presents with increasing rigidity, withdrawal,
53:27and refusal to eat. The
53:29question you need to answer
53:30is, is this new? Is
53:31this a change? Did something
53:32shift? But you have no
53:33data.
53:34Catatonia is diagnosed by detecting
53:36change against a prior baseline.
53:37And if there's no baseline,
53:38there's no anger.
53:40You can't distinguish an acute
53:41catatonic process from a gradual
53:43functional decline. And the true
53:45trajectory is, in a very
53:46real sense, unknown to you.
53:48With repeated structures, assessments, that
53:50changes entirely. You have a
53:52longitudinal record. You can see
53:53where the line bent. You
53:54can see the inflection point,
53:56the visit where scores dropped,
53:57where function changed, where something
53:58clearly shifted from this patient's
54:01own prior state,
54:02motor behavior, activity level, functional
54:04engagement, the measure that captures
54:06what catty catatonia produces are
54:08in the record at every
54:10prior visit.
54:11We see when they're losing
54:12these adaptive skills,
54:15because we
54:16have the infrastructure able to
54:17capture it.
54:22We often see huge delays
54:24in treatment because of this,
54:25often averaging well over two
54:27hundred days.
54:29And, ultimately, timely treatment leads
54:31to better prognosis for all
54:33these kids.
54:36And I wanna talk about
54:37where this brings us. Because
54:39some of you are thinking
54:40about research.
54:41Some of you are thinking
54:42about clinical encounters.
54:44And, you know,
54:45the implications extend well beyond
54:47catatonia.
54:47So the core asset is
54:49this longitudinal structured data from
54:50every patient
54:52in a neurodevelopmental
54:53disabilities clinic captured at every
54:54visit over time.
54:56So what does that make
54:57possible? So
54:58two things immediately. First, natural
55:00history studies and rare diseases.
55:02So for conditions like Phelan
55:04McDermid or SYNGAP1, there are
55:05no registries that capture the
55:07full clinical phenotype
55:08and certainly not longitudinally.
55:11Patient advocacy
55:13groups have done a really
55:14good job of doing that,
55:15but it's burdensome. And these
55:16are kids who are ending
55:17up in doctor's offices anyway.
55:19So measurement based care in
55:20the EHR does without requiring
55:22separate enrollment, without excluding the
55:24most severely affected patients, and
55:26without
55:26ascertainment bias that plagues every
55:29opt in research registry.
55:31Second, we have the possibility
55:32for n of one and
55:33single subject study designs. So
55:35when every patient has their
55:36own longitudinal
55:37control,
55:38you can study individual trajectories
55:40in ways that population level
55:41RCTs cannot. You can detect
55:43progression, recovery, and medication responses,
55:46the level of the individual,
55:47which is exactly what clinicians
55:48need and what families are
55:50asking for.
55:51And
55:52then two more downstream is
55:54really thinking about trial readiness.
55:55When the clinic population is
55:57already phenotyped and your outcome
55:59measures are already validated in
56:01the context of routine care,
56:02you can run trials faster
56:04with populations that actually represent
56:06the patients
56:07the intervention is meant to
56:08help. The infrastructure does the
56:10legwork
56:11that enrollment and characterization normally
56:13require.
56:14And biomarker development, so longitudinal
56:16functional data at scale creates
56:18the opportunity to identify functional
56:20correlates of genetic diagnosis, candidate
56:22behavioral endpoints
56:24for early intervention trials before
56:26we have the biological measures
56:27in place.
56:30And every clinical encounter becomes
56:31a research data point without
56:33excluding the patients who need
56:34it most. The same infrastructure
56:36that catches a catatonic episode
56:38in clinic is the infrastructure
56:39that makes the science possible.
56:42That's what we're building. The
56:43data we don't have isn't
56:44missing by accident. It's missing
56:45because the infrastructure to capture
56:47it has never been made
56:48a priority.
56:50Thank you. Happy to answer
56:51any questions.
57:00Maybe just while people are
57:01collecting their thoughts, I know
57:02you have an excellent colleague
57:03at Boston Children's to think
57:04about this with Mark Mercurio,
57:05but can you talk us
57:06through your thoughts about the
57:08consent process
57:10for families entering into this?
57:11Will it be opt in?
57:12Will it be opt out?
57:13And potentially how you will
57:15link that into biological sampling.
57:17So I think it's really
57:18interesting, and it's something that
57:19we're giving a lot of
57:20thought to, and I spend
57:22a lot of time talking
57:22to Mark and the IRB
57:23about.
57:25So I think it's important,
57:27and I think
57:29one thing to acknowledge is
57:30that, generally,
57:32all of our health records
57:34are
57:36opt out for research purposes.
57:38So everything that is collected
57:40by default
57:41is eligible for research. I
57:42think the question becomes, what
57:44are we doing and what
57:45is standard of care?
57:46Because every doctor's appointment, someone
57:48can go in and, you
57:49know, pull it and say,
57:51I wanna, you know, see
57:54a bunch of thirty year
57:54olds and, like, what happens
57:56to them when we tell
57:57them to exercise.
57:59That is already happening whether
58:00or not people know it.
58:02What we've been really intentional
58:03about is wanting to think
58:05about
58:07what is the additional burden
58:08to these families and what
58:10is useful.
58:11I think
58:12the idea is that this
58:13should be less burdensome. And
58:15when we've worked with community
58:16advisory boards to really kind
58:18of ask them, you know,
58:19what
58:21what do you think about
58:22this? What do you think
58:22about this process?
58:24They've overwhelmingly been enthusiastic
58:26because they're like, I really,
58:29you know, wanna partner in
58:31this. And I think it's
58:32important to acknowledge the population
58:34that we're talking about is
58:35mostly those with pretty profound
58:36disabilities and intellectual disability
58:39and genetic syndromes.
58:40It's less
58:42kind of broader psychiatry, which
58:43I think slightly sways it.
58:45But I think the other
58:46thing that we've thought a
58:47lot about is
58:50kind of data availability,
58:52data sharing, because the broad
58:53goal is that, like, I'm
58:55working with collaborators at different
58:56sites to think about harmonization
58:58that, like, because we're in
58:59Epic, we can use the
59:00same flow sheets that someone
59:01at Texas Children's or at
59:03CHOP is using.
59:04And ultimately, for these super
59:05rare samples, harmonize our data
59:07to think about getting meaningful
59:09sample sizes.
59:10But I think that raises
59:12other ethical questions that we're
59:14kind of sorting through. And
59:15one thing that I'm working
59:16with some colleagues on is
59:18developing a
59:19questionnaire for these families. I
59:21think we've had really interesting
59:22infrastructure discussions about
59:25data sharing more broadly with
59:26phenotypic data.
59:29Some people
59:30are very opposed to it,
59:32in terms of kind of
59:33researchers and PIs.
59:36My sense from working with
59:37families is that's not how
59:38they feel.
59:39People are not contributing to
59:41individual sciences because they care
59:42about
59:44a professor's promotion.
59:45They don't care about tenure
59:46track. They care about answers
59:48for their kids. And I
59:50think most families,
59:52anecdotally,
59:54feel similarly about that. So
59:56we're trying to validate that
59:57through some questionnaires.
59:59But I think our goal
01:00:00is
01:00:01really to have this be
01:00:02kind of an opt out
01:00:03process and include everyone.
01:00:15Hey, Max. This is great.
01:00:17And I think that what
01:00:17you have shown us is
01:00:19the X-ray of your next
01:00:20forty years of work
01:00:22because
01:00:23that's how long I think
01:00:24it's gonna take, but it's
01:00:25gonna be worth it. And
01:00:26if someone can do it,
01:00:26you can do it. And
01:00:28I think that you can
01:00:28If someone's stupid enough to
01:00:29do it, it's me. You're
01:00:30you're stupid enough. You're stupid
01:00:31enough. And, you know, to
01:00:34start it, there couldn't be
01:00:35a better place, in support
01:00:37than Boston,
01:00:38Children's. But then I can
01:00:40see this. This has been
01:00:41a dream floating around in
01:00:42the American Academy, the John
01:00:43March, before your time, already
01:00:45talked about. But I think
01:00:47that you can do it.
01:00:47You're young enough, and I
01:00:48think that that's great.
01:00:50My question or my concern
01:00:52is that I think that
01:00:53Epic can do all these
01:00:54beautiful things. I I I
01:00:55can see all this, techno
01:00:57wizardry happening, but
01:00:59our measures are really bad.
01:01:01So are you gonna develop
01:01:03the next forty years of
01:01:04your career to develop good
01:01:05measures? Or how how are
01:01:06you gonna kidding aside, how
01:01:08are you gonna reconcile the
01:01:09badness of our measures with
01:01:10the task at hand? So
01:01:11I think that's where population
01:01:13becomes really important. So I
01:01:14think you're right. Our measures
01:01:16are really bad.
01:01:18I anticipated this question as
01:01:19I was driving down, and
01:01:21I think the
01:01:23you know, we have bad
01:01:25measures. They're not perfect. I
01:01:27think some of it, you
01:01:28know, as Lacy talked about,
01:01:29is really thinking about how
01:01:31do we get to, like,
01:01:31what the crux of what
01:01:32we're measuring is. And I
01:01:33think some of that is
01:01:34using all sorts of measurements
01:01:36and seeing what exists.
01:01:37I also think, you know,
01:01:38if I give someone a
01:01:40ruler that's, like, kinda messed
01:01:41up and, like, you know,
01:01:42someone slips when they were
01:01:44printing it,
01:01:45and everyone uses the same
01:01:46ruler that's kinda messed up
01:01:47and, like, the inches aren't
01:01:49lined up, and I tell
01:01:50them to measure the chair,
01:01:51we come up with the
01:01:52same thing. But if I
01:01:53ask everyone to eyeball how
01:01:55big that chair is,
01:01:56we get things that are
01:01:58very different that I think
01:01:59even with the imprecision of
01:02:00our measurements,
01:02:02I think having
01:02:03unified measurements gets us somewhere
01:02:05that we're not right now
01:02:06in psychiatry.
01:02:07I think the other piece,
01:02:09though, is being really specific
01:02:10about our population. So really
01:02:11narrowing down on intellectual disability
01:02:14and thinking about what measures
01:02:15are validated just in this
01:02:17population
01:02:18and just focusing there, and
01:02:20that's kinda why I'm starting
01:02:21there. I think we can
01:02:22think about, you know, PHQs
01:02:23and all sorts of stuff.
01:02:25I am not gonna ask
01:02:26parents
01:02:27of kids who have no
01:02:29functional language
01:02:31PHQ questions. It's insulting.
01:02:34And I think, you know,
01:02:35that's I think where we
01:02:36start is really breaking down
01:02:38in terms of
01:02:40who is our population and
01:02:41what are we trying to
01:02:41measure.
01:02:46Thank you, everyone. We are
01:02:47at time, but,
01:02:49I'm sure you have other
01:02:50questions. So why don't we
01:02:51take just five minutes five
01:02:53to ten minutes to allow
01:02:54open questions for Lacey, Joey,
01:02:57and Max. So why don't
01:02:58you keep those come up?
01:02:59And,
01:03:01what are the questions we
01:03:02have out there? I'm sure
01:03:03you have some.
01:03:15I have,
01:03:17more of a a comment
01:03:18for this presentation than a
01:03:20question, but perhaps,
01:03:22we can think about it.
01:03:26For the folks
01:03:28that have a
01:03:29profound disability,
01:03:31and as you're saying, is
01:03:32the population that you're really
01:03:34aiming at,
01:03:37probably
01:03:38their families have encountered behavior
01:03:41analysts a lot,
01:03:42and they've
01:03:44encountered, you know, single sample
01:03:46design, single person designs, and
01:03:49that kind of work dominates
01:03:51the behavior analytic field. Right?
01:03:53So I think that,
01:03:55having
01:03:57that experience and knowing
01:03:59everything is scored and then
01:04:01everything has phase lines if
01:04:02they start a new medication,
01:04:04everything. So seeing it myself
01:04:07for a long time,
01:04:08I can vouch for how
01:04:10incredibly
01:04:10powerful it is because
01:04:12you can see something
01:04:14happening and then
01:04:16a drug change happens or,
01:04:17you know, specific event happens
01:04:19or an illness, and then
01:04:20the data completely changes, and
01:04:22it's extremely informative.
01:04:25And I guess the question
01:04:26is, you know, is there
01:04:28a collaboration
01:04:30with
01:04:30the behavior analytic field on
01:04:32developing
01:04:33these measurements? Because I have
01:04:35the same concern is that
01:04:37there's,
01:04:38you know, a lack of
01:04:39validity and reliability in the
01:04:41measures that we use. But
01:04:42I think that that
01:04:44field can be very informative
01:04:45in this population.
01:04:47So one of my
01:04:50one of my favorite things
01:04:51is that
01:04:52I
01:04:53at Children's, we have a
01:04:54bunch of behavior analysts that,
01:04:56like, are in our outpatient
01:04:57clinic that, like, when I
01:04:58have someone having a hard
01:04:59time, we have, like, behavior
01:05:00analysts who are, like, ready
01:05:01to run-in an endpoint, which
01:05:03is, like, amazing.
01:05:05But they're really good thought
01:05:06thought partners and part of
01:05:07these research teams to think
01:05:09about what
01:05:10how do we design this
01:05:11and what is meaningful.
01:05:13So I think,
01:05:15you know, some of it
01:05:16is actually incorporating that data,
01:05:17and it's something I kind
01:05:18of routinely do that, like,
01:05:20every single week for all
01:05:21of my patients when we
01:05:22have an appointment.
01:05:23I'm getting those, like, graphs
01:05:25uploaded from their BCBAs at
01:05:26home. And I think it's
01:05:28super useful for me. I
01:05:29think the question is how
01:05:30do we store that? And
01:05:31I think that's, like, really
01:05:32what the data question is.
01:05:33I think, like, it's being
01:05:34collected.
01:05:35I think
01:05:38on a lot of ways,
01:05:38it's on us because, like,
01:05:40you know, if you put
01:05:41something in a filing cabinet,
01:05:42it's easier to find. If
01:05:43you put it in a
01:05:44stack of paper and then
01:05:45just, like, toss it on
01:05:46a desk, it gets really
01:05:48messy.
01:05:49Can I speculate for one
01:05:51thing? So I come from
01:05:52a different area, but I
01:05:53also think, like, it echoes
01:05:54what Max said about considering
01:05:55the population,
01:05:56which is, for example, in,
01:05:58like, individuals with chronic pain
01:05:59that we have a lot
01:06:00of EMA data for, almost
01:06:02seventy percent of the variance
01:06:03in pain ratings comes within
01:06:04individual, not between.
01:06:06So I think getting that
01:06:07clear cut, like, intervention drug
01:06:10effect,
01:06:11in our opinion, would require,
01:06:13like, really nice EMA style
01:06:15data
01:06:16over time to be able
01:06:17to even make any confident
01:06:18claim that, like, drug a
01:06:19worked or something like that.
01:06:20And so considering the amount
01:06:21of like, I don't know,
01:06:23I don't know how variance,
01:06:24like, catatonia
01:06:25is across time, but, like,
01:06:26I think that's a key
01:06:27element for this. Yeah. And
01:06:28I think it was some
01:06:29one of the questions we
01:06:30post to our community advisory
01:06:31board of, like, how frequent
01:06:32would you wanna hear from
01:06:34us? Yeah. Yeah. Like, when
01:06:35we're making a medication change,
01:06:37would you fill something out
01:06:38every single day
01:06:40for two weeks? And they
01:06:41were like, yeah. Hundred percent.
01:06:43As long as someone's looking
01:06:44at it on the other
01:06:44end
01:06:45yeah.
01:06:47They don't want data that's
01:06:48going nowhere. They want their
01:06:49doctor to look at it.
01:06:50And I think that's a
01:06:51different question, but
01:06:53families are, like, sitting out
01:06:55here in the void, and
01:06:55they're like, I don't know
01:06:57what the heck I'm doing.
01:06:58And they're really looking for
01:06:59a partner in this.
01:07:05Any other questions?
01:07:07Comments?
01:07:08Okay. I'd like you all
01:07:09to give,
01:07:11these three wonderful, talented speakers
01:07:13a round of applause.