Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning

Sagar Jilka; Clarissa Mary Odoi; Janet van Bilsen; Daniel Morris; Sinan Erturk; Nicholas Cummins; Matteo Cella; Til Wykes

doi:10.1038/s41537-021-00197-6

Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning

Sagar Jilka^*, Clarissa Mary Odoi, Janet van Bilsen, Daniel Morris, Sinan Erturk, Nicholas Cummins, Matteo Cella, Til Wykes

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

4 Citations (Scopus)

Abstract

Stigma has negative effects on people with mental health problems by making them less likely to seek help. We develop a proof of principle service user supervised machine learning pipeline to identify stigmatising tweets reliably and understand the prevalence of public schizophrenia stigma on Twitter. A service user group advised on the machine learning model evaluation metric (fewest false negatives) and features for machine learning. We collected 13,313 public tweets on schizophrenia between January and May 2018. Two service user researchers manually identified stigma in 746 English tweets; 80% were used to train eight models, and 20% for testing. The two models with fewest false negatives were compared in two service user validation exercises, and the best model used to classify all extracted public English tweets. Tweets classed as stigmatising by service users were more negative in sentiment (t (744) = 12.02, p < 0.001 [95% CI: 0.196–0.273]). Our linear Support Vector Machine was the best performing model with fewest false negatives and higher service user validation. This model identified public stigma in 47% of English tweets (n5,676) which were more negative in sentiment (t (12,143) = 64.38, p < 0.001 [95% CI: 0.29–0.31]). Machine learning can identify stigmatising tweets at large scale, with service user involvement. Given the prevalence of stigma, there is an urgent need for education and online campaigns to reduce it. Machine learning can provide a real time metric on their success.

Original language	English
Article number	1
Journal	NPJ SCHIZOPHRENIA
Volume	8
Issue number	1
DOIs	https://doi.org/10.1038/s41537-021-00197-6
Publication status	Published - 7 Feb 2022

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1038/s41537-021-00197-6

Cite this

@article{5eac7d36859a41349a63c69d23883c95,

title = "Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning",

abstract = "Stigma has negative effects on people with mental health problems by making them less likely to seek help. We develop a proof of principle service user supervised machine learning pipeline to identify stigmatising tweets reliably and understand the prevalence of public schizophrenia stigma on Twitter. A service user group advised on the machine learning model evaluation metric (fewest false negatives) and features for machine learning. We collected 13,313 public tweets on schizophrenia between January and May 2018. Two service user researchers manually identified stigma in 746 English tweets; 80% were used to train eight models, and 20% for testing. The two models with fewest false negatives were compared in two service user validation exercises, and the best model used to classify all extracted public English tweets. Tweets classed as stigmatising by service users were more negative in sentiment (t (744) = 12.02, p < 0.001 [95% CI: 0.196–0.273]). Our linear Support Vector Machine was the best performing model with fewest false negatives and higher service user validation. This model identified public stigma in 47% of English tweets (n5,676) which were more negative in sentiment (t (12,143) = 64.38, p < 0.001 [95% CI: 0.29–0.31]). Machine learning can identify stigmatising tweets at large scale, with service user involvement. Given the prevalence of stigma, there is an urgent need for education and online campaigns to reduce it. Machine learning can provide a real time metric on their success.",

author = "Sagar Jilka and Odoi, {Clarissa Mary} and {van Bilsen}, Janet and Daniel Morris and Sinan Erturk and Nicholas Cummins and Matteo Cella and Til Wykes",

note = "Funding Information: We thank all user-researchers and advisors who supported this work; Chinelo Daniels-Ifekwe, Helena Griffiths, Catherine Kilkenny, Sazan Meran, Caecilia Pawitra, Joel Vasama, Emma Wilson, Khaizer Rizvi, Magano Mutepua, Gregory Verghese, Sumithra Velupillai, and Angus Roberts. We thank the Young Person{\textquoteright}s Mental Health Advisory Group for their constant input and advice on this important issue, and the King{\textquoteright}s College London NLP reading group. This work was supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King{\textquoteright}s College London (IS-BRC-1215-20018). Publisher Copyright: {\textcopyright} 2022, The Author(s).",

year = "2022",

month = feb,

day = "7",

doi = "10.1038/s41537-021-00197-6",

language = "English",

volume = "8",

journal = "NPJ SCHIZOPHRENIA",

issn = "2334-265X",

publisher = "Nature Publishing Group",

number = "1",

}

TY - JOUR

T1 - Identifying schizophrenia stigma on Twitter

T2 - a proof of principle model using service user supervised machine learning

AU - Jilka, Sagar

AU - Odoi, Clarissa Mary

AU - van Bilsen, Janet

AU - Morris, Daniel

AU - Erturk, Sinan

AU - Cummins, Nicholas

AU - Cella, Matteo

AU - Wykes, Til

N1 - Funding Information: We thank all user-researchers and advisors who supported this work; Chinelo Daniels-Ifekwe, Helena Griffiths, Catherine Kilkenny, Sazan Meran, Caecilia Pawitra, Joel Vasama, Emma Wilson, Khaizer Rizvi, Magano Mutepua, Gregory Verghese, Sumithra Velupillai, and Angus Roberts. We thank the Young Person’s Mental Health Advisory Group for their constant input and advice on this important issue, and the King’s College London NLP reading group. This work was supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London (IS-BRC-1215-20018). Publisher Copyright: © 2022, The Author(s).

PY - 2022/2/7

Y1 - 2022/2/7

N2 - Stigma has negative effects on people with mental health problems by making them less likely to seek help. We develop a proof of principle service user supervised machine learning pipeline to identify stigmatising tweets reliably and understand the prevalence of public schizophrenia stigma on Twitter. A service user group advised on the machine learning model evaluation metric (fewest false negatives) and features for machine learning. We collected 13,313 public tweets on schizophrenia between January and May 2018. Two service user researchers manually identified stigma in 746 English tweets; 80% were used to train eight models, and 20% for testing. The two models with fewest false negatives were compared in two service user validation exercises, and the best model used to classify all extracted public English tweets. Tweets classed as stigmatising by service users were more negative in sentiment (t (744) = 12.02, p < 0.001 [95% CI: 0.196–0.273]). Our linear Support Vector Machine was the best performing model with fewest false negatives and higher service user validation. This model identified public stigma in 47% of English tweets (n5,676) which were more negative in sentiment (t (12,143) = 64.38, p < 0.001 [95% CI: 0.29–0.31]). Machine learning can identify stigmatising tweets at large scale, with service user involvement. Given the prevalence of stigma, there is an urgent need for education and online campaigns to reduce it. Machine learning can provide a real time metric on their success.

AB - Stigma has negative effects on people with mental health problems by making them less likely to seek help. We develop a proof of principle service user supervised machine learning pipeline to identify stigmatising tweets reliably and understand the prevalence of public schizophrenia stigma on Twitter. A service user group advised on the machine learning model evaluation metric (fewest false negatives) and features for machine learning. We collected 13,313 public tweets on schizophrenia between January and May 2018. Two service user researchers manually identified stigma in 746 English tweets; 80% were used to train eight models, and 20% for testing. The two models with fewest false negatives were compared in two service user validation exercises, and the best model used to classify all extracted public English tweets. Tweets classed as stigmatising by service users were more negative in sentiment (t (744) = 12.02, p < 0.001 [95% CI: 0.196–0.273]). Our linear Support Vector Machine was the best performing model with fewest false negatives and higher service user validation. This model identified public stigma in 47% of English tweets (n5,676) which were more negative in sentiment (t (12,143) = 64.38, p < 0.001 [95% CI: 0.29–0.31]). Machine learning can identify stigmatising tweets at large scale, with service user involvement. Given the prevalence of stigma, there is an urgent need for education and online campaigns to reduce it. Machine learning can provide a real time metric on their success.

UR - http://www.scopus.com/inward/record.url?scp=85124453167&partnerID=8YFLogxK

U2 - 10.1038/s41537-021-00197-6

DO - 10.1038/s41537-021-00197-6

M3 - Article

AN - SCOPUS:85124453167

SN - 2334-265X

VL - 8

JO - NPJ SCHIZOPHRENIA

JF - NPJ SCHIZOPHRENIA

IS - 1

M1 - 1

ER -

Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning

Abstract

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this