Modeling remotely collected speech data: Applications for psychiatry

Abstract

Detecting signs of disorder from listening to spoken words is a core method in psychiatry. Traditionally the interpretation of speech depends on inherently subjective processes. By contrast, digital technology can be leveraged to detect and analyze what words are spoken, timestamp when they are uttered and quantify the manner in which they are expressed. With the use of mobile communication technology, digital speech processing tools are possible to use outside of traditional laboratory settings. This thesis argues that the necessary infrastructure to move speech processing into clinical practice is currently available. To examine this claim, a mobile application for remote mental state assessments was developed that implemented speech-based neuropsychological testing in 353 participants in two countries. It was possible to collect speech data in ecologically valid settings, but future larger scale implementations must solve technical, legal and cultural challenges by interdisciplinary teamwork. The findings of spoken responses on the classic Stroop color-word test from 57 patients with substance use disorders and 86 healthy participants showed that the production of single-word speech utterances could be measured with a high level of temporal precision. The classic Stroop task response latency interference was replicated and the scope of measurements was extended with novel speech characteristics. The audio files from 59 participants naming words in a category fluency task could be analyzed for both the temporal dynamics of response-word sequences and the semantic relatedness between words. Finally, the story recall ability in 25 patients with serious mental illness and 79 healthy participants was examined, and automated measurements of their ability to retell a story was computed using both simple word-count procedures and more advanced estimates of distances in a semantic vector space. In conclusion, it is technologically feasible to develop instruments for measuring multiple aspects of how patients with psychiatric disorders speak, and traditional speech-based neuropsychological tests can be employed outside of a laboratory setting provided the digital infrastructure is able to ensure the privacy of the users.