Show simple item record

dc.contributor.authorCohen, Alex S.
dc.contributor.authorRodriguez, Zachary
dc.contributor.authorWarren, Kiara K.
dc.contributor.authorCowan, Tovah
dc.contributor.authorMasucci, Michael D.
dc.contributor.authorEdvard Granrud, Ole
dc.contributor.authorHolmlund, Terje Bektesevic
dc.contributor.authorChandler, Chelsea
dc.contributor.authorFoltz, Peter W.
dc.contributor.authorStrauss, Gregory P.
dc.description.abstract<p><i>Background and Hypothesis:</i> Despite decades of “proof of concept” findings supporting the use of Natural Language Processing (NLP) in psychosis research, clinical implementation has been slow. One obstacle reflects the lack of comprehensive psychometric evaluation of these measures. There is overwhelming evidence that criterion and content validity can be achieved for many purposes, particularly using machine learning procedures. However, there has been very little evaluation of test-retest reliability, divergent validity (sufficient to address concerns of a “generalized deficit”), and potential biases from demographics and other individual differences. <p><i>Study Design:</i> This article highlights these concerns in development of an NLP measure for tracking clinically rated paranoia from video “selfies” recorded from smartphone devices. Patients with schizophrenia or bipolar disorder were recruited and tracked over a week-long epoch. A small NLP-based feature set from 499 language samples were modeled on clinically rated paranoia using regularized regression. <p><i>Study Results:</i> While test–retest reliability was high, criterion, and convergent/divergent validity were only achieved when considering moderating variables, notably whether a patient was away from home, around strangers, or alone at the time of the recording. Moreover, there were systematic racial and sex biases in the model, in part, reflecting whether patients submitted videos when they were away from home, around strangers, or alone. <p><i>Conclusions:</i> Advancing NLP measures for psychosis will require deliberate consideration of test-retest reliability, divergent validity, systematic biases and the potential role of moderators. In our example, a comprehensive psychometric evaluation revealed clear strengths and weaknesses that can be systematically addressed in future research.en_US
dc.identifier.citationCohen, Rodriguez, Warren, Cowan, Masucci, Edvard Granrud, Holmlund, Chandler, Foltz, Strauss. Natural Language Processing and Psychosis: On the Need for Comprehensive Psychometric Evaluation. Schizophrenia Bulletin. 2022;48(5):939-948en_US
dc.identifier.cristinIDFRIDAID 2070391
dc.publisherOxford University Pressen_US
dc.relation.journalSchizophrenia Bulletin
dc.rights.holderCopyright 2022 The Author(s)en_US
dc.rightsAttribution 4.0 International (CC BY 4.0)en_US
dc.titleNatural Language Processing and Psychosis: On the Need for Comprehensive Psychometric Evaluationen_US
dc.typeJournal articleen_US

File(s) in this item


This item appears in the following collection(s)

Show simple item record

Attribution 4.0 International (CC BY 4.0)
Except where otherwise noted, this item's license is described as Attribution 4.0 International (CC BY 4.0)