Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities

Bahman Mirheidari, Andre Bittar, Nicholas Cummins, Johnny Downs, Helen L. Fisher, Heidi Christensen

Research output: Contribution to journalArticlepeer-review

1 Downloads (Pure)


Research into clinical applications of speech-based emotion recognition (SER)
technologies has been steadily increasing over the past few years. One such potential application is the automatic recognition of expressed emotion (EE) components within family environments. The identification of EE is highly important as they have been linked with a range of adverse life events. Manual coding of these events requires time-consuming specialist training, amplifying the need for automated approaches. Herein we describe an automated machine learning approach for determining the degree of warmth, a key component of EE, from acoustic and text natural language features. Our dataset of 52 recorded interviews is taken from recordings, collected over 20 years ago, from a nationally representative birth cohort of British twin children, and was manually coded for EE by two researchers (inter-rater reliability 0.84–0.90). We demonstrate that the degree of warmth can be predicted with an F1-score of 64.7% despite working with audio recordings of highly variable quality. Our highly promising results suggest that machine learning may be able to assist in the coding of EE in the near future.
Original languageEnglish
Article numbere0300518
JournalPLoS One
Issue number3 March
Early online date21 Mar 2024
Publication statusPublished - Mar 2024


Dive into the research topics of 'Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities'. Together they form a unique fingerprint.

Cite this