Multilingual markers of depression in remotely collected speech samples: A preliminary analysis

Nicholas Cummins; Judith Dineley; Pauline Conde; Faith Matcham; Sara Siddi; Femke Lamers; Ewan Carr; Grace Lavelle; Daniel Leightley; Katie White; Carolin Oetzmann; Edward Campbell; Sara Simblett; Stuart Bruce; Josep Maria Haro; Brenda WJH Pennix; Yatharth Ranjan; Zulqarnain Rashid; Callum Stewart; Amos Folarin; Raquel Bailón; Björn Schuller; Til Wykes; Srinivasan  Vairavan; Richard Dobson; Vaibhav Narayan; Matthew Hotopf; on behalf of the RADAR CNS Consortium

doi:10.1016/j.jad.2023.08.097

Multilingual markers of depression in remotely collected speech samples: A preliminary analysis

Nicholas Cummins, Judith Dineley, Pauline Conde, Faith Matcham, Sara Siddi, Femke Lamers, Ewan Carr, Grace Lavelle, Daniel Leightley, Katie White, Carolin Oetzmann, Edward Campbell, Sara Simblett, Stuart Bruce, Josep Maria Haro, Brenda WJH Pennix, Yatharth Ranjan, Zulqarnain Rashid, Callum Stewart, Amos FolarinRaquel Bailón, Björn Schuller, Til Wykes, Srinivasan Vairavan, Richard Dobson, Vaibhav Narayan, Matthew Hotopf, on behalf of the RADAR CNS Consortium

Research output: Contribution to journal › Article › peer-review

11 Citations (Scopus)

Abstract

Background: Speech contains neuromuscular, physiological and cognitive components, and so is a potential biomarker of mental disorders. Previous studies indicate that speaking rate and pausing are associated with major depressive disorder (MDD). However, results are inconclusive as many studies are small and underpowered and do not include clinical samples. These studies have also been unilingual and use speech collected in controlled settings. If speech markers are to help understand the onset and progress of MDD, we need to uncover markers that are robust to language and establish the strength of associations in real-world data.

Methods: We collected speech data in 585 participants with a history of MDD in the United Kingdom, Spain, and Netherlands as part of the RADAR-MDD study. Participants recorded their speech via smartphones every two weeks for 18 months. Linear mixed models were used to estimate the strength of specific markers of depression from a set of 28 speech features.

Results: Increased depressive symptoms were associated with speech rate, articulation rate and intensity of speech elicited from a scripted task. These features had consistently stronger effect sizes than pauses.

Limitations: Our findings are derived at the cohort level so may have limited impact on identifying intra-individual speech changes associated with changes in symptom severity. The analysis of features averaged over the entire recording may have underestimated the importance of some features.

Conclusions
Participants with more severe depressive symptoms spoke more slowly and quietly. Our findings are from a real-world, multilingual, clinical dataset so represent a step-change in the usefulness of speech as a digital phenotype of MDD.

Original language	English
Pages (from-to)	128-136
Number of pages	9
Journal	Journal of Affective Disorders
Volume	341
Early online date	18 Aug 2023
DOIs	https://doi.org/10.1016/j.jad.2023.08.097
Publication status	Published - 15 Nov 2023

Keywords

Major depressive disorder
Digital phenotypes
Speech
In-the-wild
Speaking rate

Access to Document

10.1016/j.jad.2023.08.097Licence: CC BY

Cite this

Cummins, N., Dineley, J., Conde, P., Matcham, F., Siddi, S., Lamers, F., Carr, E., Lavelle, G., Leightley, D., White, K., Oetzmann, C., Campbell, E., Simblett, S., Bruce, S., Maria Haro, J., Pennix, B. WJH., Ranjan, Y., Rashid, Z., Stewart, C., ... Consortium, O. B. O. T. RADAR. CNS. (2023). Multilingual markers of depression in remotely collected speech samples: A preliminary analysis. Journal of Affective Disorders, 341, 128-136. https://doi.org/10.1016/j.jad.2023.08.097

@article{a74c7b94f65040ff8735335b183a0919,

title = "Multilingual markers of depression in remotely collected speech samples: A preliminary analysis",

abstract = "Background: Speech contains neuromuscular, physiological and cognitive components, and so is a potential biomarker of mental disorders. Previous studies indicate that speaking rate and pausing are associated with major depressive disorder (MDD). However, results are inconclusive as many studies are small and underpowered and do not include clinical samples. These studies have also been unilingual and use speech collected in controlled settings. If speech markers are to help understand the onset and progress of MDD, we need to uncover markers that are robust to language and establish the strength of associations in real-world data.Methods: We collected speech data in 585 participants with a history of MDD in the United Kingdom, Spain, and Netherlands as part of the RADAR-MDD study. Participants recorded their speech via smartphones every two weeks for 18 months. Linear mixed models were used to estimate the strength of specific markers of depression from a set of 28 speech features.Results: Increased depressive symptoms were associated with speech rate, articulation rate and intensity of speech elicited from a scripted task. These features had consistently stronger effect sizes than pauses.Limitations: Our findings are derived at the cohort level so may have limited impact on identifying intra-individual speech changes associated with changes in symptom severity. The analysis of features averaged over the entire recording may have underestimated the importance of some features.ConclusionsParticipants with more severe depressive symptoms spoke more slowly and quietly. Our findings are from a real-world, multilingual, clinical dataset so represent a step-change in the usefulness of speech as a digital phenotype of MDD.",

keywords = "Major depressive disorder, Digital phenotypes, Speech, In-the-wild, Speaking rate",

author = "Nicholas Cummins and Judith Dineley and Pauline Conde and Faith Matcham and Sara Siddi and Femke Lamers and Ewan Carr and Grace Lavelle and Daniel Leightley and Katie White and Carolin Oetzmann and Edward Campbell and Sara Simblett and Stuart Bruce and {Maria Haro}, Josep and Pennix, {Brenda WJH} and Yatharth Ranjan and Zulqarnain Rashid and Callum Stewart and Amos Folarin and Raquel Bail{\'o}n and Bj{\"o}rn Schuller and Til Wykes and Srinivasan Vairavan and Richard Dobson and Vaibhav Narayan and Matthew Hotopf and Consortium, {on behalf of the RADAR CNS}",

note = "Funding Information: The RADAR-CNS project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 115902. This Joint Undertaking receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA (www.imi.europa.eu). This communication reflects the views of the RADAR-CNS consortium and neither IMI nor the European Union and EFPIA are liable for any use that may be made of the information contained herein. The funding body have not been involved in the design of the study, the collection or analysis of data, or the interpretation of data. Participant recruitment in Amsterdam was partially accomplished through Hersenonderzoek.nl, a Dutch online registry that facilitates participant recruitment for neuroscience studies (https://hersenonderzoek.nl/). Hersenonderzoek.nl is funded by ZonMw-Memorabel (project no 73305095003), a project in the context of the Dutch Deltaplan Dementie, Gieskes-Strijbis Foundation, the Alzheimer's Society in the Netherlands and Brain Foundation Netherlands Participants in Spain were recruited through the following institutions: Parc Sanitari Sant Joan de D{\'e}u network of mental health services (Barcelona); Institut Catal{\`a} de la Salut primary care services (Barcelona); Institut Pere Mata-Mental Health Care (Tarrassa); Hospital Cl{\'i}nico San Carlos (Madrid). This paper represents independent research part funded by the National Institute for Health Research (NIHR) Maudsley Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. We thank all the members of the RADAR-CNS patient advisory board for their contribution to the device selection procedures, and their invaluable advice throughout the study protocol design. This research was reviewed by a team with experience of mental health problems and their carers who have been specially trained to advise on research proposals and documentation through the Feasibility and Acceptability Support Team for Researchers (FAST-R): a free, confidential service in England provided by the National Institute for Health Research Maudsley Biomedical Research Centre via King's College London and South London and Maudsley NHS Foundation Trust. We thank all GLAD Study volunteers for their participation, and gratefully acknowledge the NIHR BioResource, NIHR BioResource centres, NHS Trusts and staff for their contribution. We also acknowledge NIHR BRC, King's College London, South London and Maudsley NHS Trust and King's Health Partners. We thank the National Institute for Health Research, NHS Blood and Transplant, and Health Data Research UK as part of the Digital Innovation Hub Programme. We thank our colleagues both within the RADAR-CNS consortium and across all involved institutions for their contribution to the development of this protocol. We thank all the members of the RADAR-CNS patient advisory board for their contribution to the device selection procedures, and their invaluable advice throughout the study protocol design. Funding Information: Participant recruitment in Amsterdam was partially accomplished through Hersenonderzoek.nl, a Dutch online registry that facilitates participant recruitment for neuroscience studies ( https://hersenonderzoek.nl/ ). Hersenonderzoek.nl is funded by ZonMw-Memorabel (project no 73305095003 ), a project in the context of the Dutch Deltaplan Dementie, Gieskes-Strijbis Foundation, the Alzheimer's Society in the Netherlands and Brain Foundation Netherlands Participants in Spain were recruited through the following institutions: Parc Sanitari Sant Joan de D{\'e}u network of mental health services (Barcelona); Institut Catal{\`a} de la Salut primary care services (Barcelona); Institut Pere Mata-Mental Health Care (Tarrassa); Hospital Cl{\'i}nico San Carlos (Madrid). This paper represents independent research part funded by the National Institute for Health Research (NIHR) Maudsley Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London . The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. Funding Information: The RADAR-CNS project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 115902 . This Joint Undertaking receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA ( www.imi.europa.eu ). This communication reflects the views of the RADAR-CNS consortium and neither IMI nor the European Union and EFPIA are liable for any use that may be made of the information contained herein. The funding body have not been involved in the design of the study, the collection or analysis of data, or the interpretation of data. Publisher Copyright: {\textcopyright} 2023 The Authors",

year = "2023",

month = nov,

day = "15",

doi = "10.1016/j.jad.2023.08.097",

language = "English",

volume = "341",

pages = "128--136",

journal = "Journal of Affective Disorders",

issn = "0165-0327",

publisher = "Elsevier",

}

Cummins, N , Dineley, J , Conde, P, Matcham, F, Siddi, S, Lamers, F, Carr, E , Lavelle, G , Leightley, D , White, K , Oetzmann, C, Campbell, E, Simblett, S, Bruce, S, Maria Haro, J, Pennix, BWJH, Ranjan, Y , Rashid, Z , Stewart, C , Folarin, A, Bailón, R, Schuller, B, Wykes, T, Vairavan, S, Dobson, R, Narayan, V, Hotopf, M & Consortium, OBOTRADARCNS 2023, 'Multilingual markers of depression in remotely collected speech samples: A preliminary analysis', Journal of Affective Disorders, vol. 341, pp. 128-136. https://doi.org/10.1016/j.jad.2023.08.097

TY - JOUR

T1 - Multilingual markers of depression in remotely collected speech samples: A preliminary analysis

AU - Cummins, Nicholas

AU - Dineley, Judith

AU - Conde, Pauline

AU - Matcham, Faith

AU - Siddi, Sara

AU - Lamers, Femke

AU - Carr, Ewan

AU - Lavelle, Grace

AU - Leightley, Daniel

AU - White, Katie

AU - Oetzmann, Carolin

AU - Campbell, Edward

AU - Simblett, Sara

AU - Bruce, Stuart

AU - Maria Haro, Josep

AU - Pennix, Brenda WJH

AU - Ranjan, Yatharth

AU - Rashid, Zulqarnain

AU - Stewart, Callum

AU - Folarin, Amos

AU - Bailón, Raquel

AU - Schuller, Björn

AU - Wykes, Til

AU - Vairavan, Srinivasan

AU - Dobson, Richard

AU - Narayan, Vaibhav

AU - Hotopf, Matthew

AU - Consortium, on behalf of the RADAR CNS

N1 - Funding Information: The RADAR-CNS project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 115902. This Joint Undertaking receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA (www.imi.europa.eu). This communication reflects the views of the RADAR-CNS consortium and neither IMI nor the European Union and EFPIA are liable for any use that may be made of the information contained herein. The funding body have not been involved in the design of the study, the collection or analysis of data, or the interpretation of data. Participant recruitment in Amsterdam was partially accomplished through Hersenonderzoek.nl, a Dutch online registry that facilitates participant recruitment for neuroscience studies (https://hersenonderzoek.nl/). Hersenonderzoek.nl is funded by ZonMw-Memorabel (project no 73305095003), a project in the context of the Dutch Deltaplan Dementie, Gieskes-Strijbis Foundation, the Alzheimer's Society in the Netherlands and Brain Foundation Netherlands Participants in Spain were recruited through the following institutions: Parc Sanitari Sant Joan de Déu network of mental health services (Barcelona); Institut Català de la Salut primary care services (Barcelona); Institut Pere Mata-Mental Health Care (Tarrassa); Hospital Clínico San Carlos (Madrid). This paper represents independent research part funded by the National Institute for Health Research (NIHR) Maudsley Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. We thank all the members of the RADAR-CNS patient advisory board for their contribution to the device selection procedures, and their invaluable advice throughout the study protocol design. This research was reviewed by a team with experience of mental health problems and their carers who have been specially trained to advise on research proposals and documentation through the Feasibility and Acceptability Support Team for Researchers (FAST-R): a free, confidential service in England provided by the National Institute for Health Research Maudsley Biomedical Research Centre via King's College London and South London and Maudsley NHS Foundation Trust. We thank all GLAD Study volunteers for their participation, and gratefully acknowledge the NIHR BioResource, NIHR BioResource centres, NHS Trusts and staff for their contribution. We also acknowledge NIHR BRC, King's College London, South London and Maudsley NHS Trust and King's Health Partners. We thank the National Institute for Health Research, NHS Blood and Transplant, and Health Data Research UK as part of the Digital Innovation Hub Programme. We thank our colleagues both within the RADAR-CNS consortium and across all involved institutions for their contribution to the development of this protocol. We thank all the members of the RADAR-CNS patient advisory board for their contribution to the device selection procedures, and their invaluable advice throughout the study protocol design. Funding Information: Participant recruitment in Amsterdam was partially accomplished through Hersenonderzoek.nl, a Dutch online registry that facilitates participant recruitment for neuroscience studies ( https://hersenonderzoek.nl/ ). Hersenonderzoek.nl is funded by ZonMw-Memorabel (project no 73305095003 ), a project in the context of the Dutch Deltaplan Dementie, Gieskes-Strijbis Foundation, the Alzheimer's Society in the Netherlands and Brain Foundation Netherlands Participants in Spain were recruited through the following institutions: Parc Sanitari Sant Joan de Déu network of mental health services (Barcelona); Institut Català de la Salut primary care services (Barcelona); Institut Pere Mata-Mental Health Care (Tarrassa); Hospital Clínico San Carlos (Madrid). This paper represents independent research part funded by the National Institute for Health Research (NIHR) Maudsley Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London . The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. Funding Information: The RADAR-CNS project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 115902 . This Joint Undertaking receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA ( www.imi.europa.eu ). This communication reflects the views of the RADAR-CNS consortium and neither IMI nor the European Union and EFPIA are liable for any use that may be made of the information contained herein. The funding body have not been involved in the design of the study, the collection or analysis of data, or the interpretation of data. Publisher Copyright: © 2023 The Authors

PY - 2023/11/15

Y1 - 2023/11/15

N2 - Background: Speech contains neuromuscular, physiological and cognitive components, and so is a potential biomarker of mental disorders. Previous studies indicate that speaking rate and pausing are associated with major depressive disorder (MDD). However, results are inconclusive as many studies are small and underpowered and do not include clinical samples. These studies have also been unilingual and use speech collected in controlled settings. If speech markers are to help understand the onset and progress of MDD, we need to uncover markers that are robust to language and establish the strength of associations in real-world data.Methods: We collected speech data in 585 participants with a history of MDD in the United Kingdom, Spain, and Netherlands as part of the RADAR-MDD study. Participants recorded their speech via smartphones every two weeks for 18 months. Linear mixed models were used to estimate the strength of specific markers of depression from a set of 28 speech features.Results: Increased depressive symptoms were associated with speech rate, articulation rate and intensity of speech elicited from a scripted task. These features had consistently stronger effect sizes than pauses.Limitations: Our findings are derived at the cohort level so may have limited impact on identifying intra-individual speech changes associated with changes in symptom severity. The analysis of features averaged over the entire recording may have underestimated the importance of some features.ConclusionsParticipants with more severe depressive symptoms spoke more slowly and quietly. Our findings are from a real-world, multilingual, clinical dataset so represent a step-change in the usefulness of speech as a digital phenotype of MDD.

AB - Background: Speech contains neuromuscular, physiological and cognitive components, and so is a potential biomarker of mental disorders. Previous studies indicate that speaking rate and pausing are associated with major depressive disorder (MDD). However, results are inconclusive as many studies are small and underpowered and do not include clinical samples. These studies have also been unilingual and use speech collected in controlled settings. If speech markers are to help understand the onset and progress of MDD, we need to uncover markers that are robust to language and establish the strength of associations in real-world data.Methods: We collected speech data in 585 participants with a history of MDD in the United Kingdom, Spain, and Netherlands as part of the RADAR-MDD study. Participants recorded their speech via smartphones every two weeks for 18 months. Linear mixed models were used to estimate the strength of specific markers of depression from a set of 28 speech features.Results: Increased depressive symptoms were associated with speech rate, articulation rate and intensity of speech elicited from a scripted task. These features had consistently stronger effect sizes than pauses.Limitations: Our findings are derived at the cohort level so may have limited impact on identifying intra-individual speech changes associated with changes in symptom severity. The analysis of features averaged over the entire recording may have underestimated the importance of some features.ConclusionsParticipants with more severe depressive symptoms spoke more slowly and quietly. Our findings are from a real-world, multilingual, clinical dataset so represent a step-change in the usefulness of speech as a digital phenotype of MDD.

KW - Major depressive disorder

KW - Digital phenotypes

KW - Speech

KW - In-the-wild

KW - Speaking rate

UR - http://www.scopus.com/inward/record.url?scp=85171901104&partnerID=8YFLogxK

U2 - 10.1016/j.jad.2023.08.097

DO - 10.1016/j.jad.2023.08.097

M3 - Article

SN - 0165-0327

VL - 341

SP - 128

EP - 136

JO - Journal of Affective Disorders

JF - Journal of Affective Disorders

ER -

Multilingual markers of depression in remotely collected speech samples: A preliminary analysis

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this