Don’t Let Notes Be Misunderstood:  A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records

George Gkotsis; Sumithra Velupillai; Anika Oellrich; Harry Dean; Maria Liakata; Rina Dutta

Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records

George Gkotsis, Sumithra Velupillai, Anika Oellrich, Harry Dean, Maria Liakata, Rina Dutta

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

32 Citations (Scopus)

179 Downloads (Pure)

Abstract

Mental Health Records (MHRs) contain free- text documentation about patients’ suicide and suicidality. In this paper, we address the prob- lem of determining whether grammatic vari- ants (inflections) of the word “suicide” are af- firmed or negated. To achieve this, we pop- ulate and annotate a dataset with over 6,000 sentences originating from a large repository of MHRs. The resulting dataset has high Inter- Annotator Agreement ( κ 0.93). Furthermore, we develop and propose a negation detection method that leverages syntactic features of text 1 . Using parse trees, we build a set of ba- sic rules that rely on minimum domain knowl- edge and render the problem as binary clas- sification (affirmed vs. negated). Since the overall goal is to identify patients who are ex- pected to be at high risk of suicide, we focus on the evaluation of positive (affirmed) cases as determined by our classifier. Our negation detection approach yields a recall (sensitivity) value of 94.6% for the positive cases and an overall accuracy value of 91.9%. We believe that our approach can be integrated with other clinical Natural Language Processing tools in order to further advance information extrac- tion capabilities

Original language	English
Title of host publication	Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology
Subtitle of host publication	From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016
Editors	Kristy Hollingshead, Lyle Ungar
Publisher	Association for Computational Linguistics (ACL)
Pages	95-105
Number of pages	11
ISBN (Electronic)	9781941643549
Publication status	Published - 16 Jun 2016
Event	3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016 - San Diego, United States Duration: 16 Jun 2016 → …

Publication series

Name	Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016

Conference

Conference	3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016
Country/Territory	United States
City	San Diego
Period	16/06/2016 → …

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

Gkotsis2016_CLPsych_negation-supplementOther version, 73.7 KB

Cite this

Gkotsis, G., Velupillai, S., Oellrich, A., Dean, H., Liakata, M., & Dutta, R. (2016). Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records. In K. Hollingshead, & L. Ungar (Eds.), Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016 (pp. 95-105). (Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016). Association for Computational Linguistics (ACL). http://hollingk.github.io/CLPsych/pdf/CLPsych10.pdf

Gkotsis, George ; Velupillai, Sumithra ; Oellrich, Anika et al. / Don’t Let Notes Be Misunderstood : A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records. Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016. editor / Kristy Hollingshead ; Lyle Ungar. Association for Computational Linguistics (ACL), 2016. pp. 95-105 (Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016).

@inbook{df0d8b5759164cfc9d56df302f39fa2d,

title = "Don{\textquoteright}t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records",

abstract = "Mental Health Records (MHRs) contain free- text documentation about patients{\textquoteright} suicide and suicidality. In this paper, we address the prob- lem of determining whether grammatic vari- ants (inflections) of the word “suicide” are af- firmed or negated. To achieve this, we pop- ulate and annotate a dataset with over 6,000 sentences originating from a large repository of MHRs. The resulting dataset has high Inter- Annotator Agreement ( κ 0.93). Furthermore, we develop and propose a negation detection method that leverages syntactic features of text 1 . Using parse trees, we build a set of ba- sic rules that rely on minimum domain knowl- edge and render the problem as binary clas- sification (affirmed vs. negated). Since the overall goal is to identify patients who are ex- pected to be at high risk of suicide, we focus on the evaluation of positive (affirmed) cases as determined by our classifier. Our negation detection approach yields a recall (sensitivity) value of 94.6% for the positive cases and an overall accuracy value of 91.9%. We believe that our approach can be integrated with other clinical Natural Language Processing tools in order to further advance information extrac- tion capabilities",

author = "George Gkotsis and Sumithra Velupillai and Anika Oellrich and Harry Dean and Maria Liakata and Rina Dutta",

note = "Funding Information: RD is supported by a Clinician Scientist Fellowship from the Health Foundation in partnership with the Academy of Medical Sciences. RD, GG and HD are funded by the e-HOST-IT research programme. SV is supported by the Swedish Research Council (2015-00359) and the Marie Sk?odowska Curie Actions, Cofund, Project INCA 600398. AO would like to acknowledge NIHR Biomedical Research Centre for Mental Health, the Biomedical Research Unit for Dementia at the South London, the Maudsley NHS Foundation Trust and Kings College London. ML would like to acknowledge the PHEME FP7 project (grant No. 611233). The data resource is funded by the National Institute for Health Research (NIHR) Biomedical Research Centre and Dementia Biomedical Research Unit at South London and Maudsley NHS Foundation Trust and King's College London. Publisher Copyright: {\textcopyright} 2016 Association for Computational Linguistics; 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016 ; Conference date: 16-06-2016",

year = "2016",

month = jun,

day = "16",

language = "English",

series = "Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016",

publisher = "Association for Computational Linguistics (ACL)",

pages = "95--105",

editor = "Kristy Hollingshead and Lyle Ungar",

booktitle = "Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology",

}

Gkotsis, G , Velupillai, S , Oellrich, A, Dean, H, Liakata, M & Dutta, R 2016, Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records. in K Hollingshead & L Ungar (eds), Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016. Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016, Association for Computational Linguistics (ACL), pp. 95-105, 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016, San Diego, United States, 16/06/2016. <http://hollingk.github.io/CLPsych/pdf/CLPsych10.pdf>

Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records. / Gkotsis, George ; Velupillai, Sumithra ; Oellrich, Anika et al.
Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016. ed. / Kristy Hollingshead; Lyle Ungar. Association for Computational Linguistics (ACL), 2016. p. 95-105 (Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

TY - CHAP

T1 - Don’t Let Notes Be Misunderstood

T2 - 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016

AU - Gkotsis, George

AU - Velupillai, Sumithra

AU - Oellrich, Anika

AU - Dean, Harry

AU - Liakata, Maria

AU - Dutta, Rina

N1 - Funding Information: RD is supported by a Clinician Scientist Fellowship from the Health Foundation in partnership with the Academy of Medical Sciences. RD, GG and HD are funded by the e-HOST-IT research programme. SV is supported by the Swedish Research Council (2015-00359) and the Marie Sk?odowska Curie Actions, Cofund, Project INCA 600398. AO would like to acknowledge NIHR Biomedical Research Centre for Mental Health, the Biomedical Research Unit for Dementia at the South London, the Maudsley NHS Foundation Trust and Kings College London. ML would like to acknowledge the PHEME FP7 project (grant No. 611233). The data resource is funded by the National Institute for Health Research (NIHR) Biomedical Research Centre and Dementia Biomedical Research Unit at South London and Maudsley NHS Foundation Trust and King's College London. Publisher Copyright: © 2016 Association for Computational Linguistics

PY - 2016/6/16

Y1 - 2016/6/16

N2 - Mental Health Records (MHRs) contain free- text documentation about patients’ suicide and suicidality. In this paper, we address the prob- lem of determining whether grammatic vari- ants (inflections) of the word “suicide” are af- firmed or negated. To achieve this, we pop- ulate and annotate a dataset with over 6,000 sentences originating from a large repository of MHRs. The resulting dataset has high Inter- Annotator Agreement ( κ 0.93). Furthermore, we develop and propose a negation detection method that leverages syntactic features of text 1 . Using parse trees, we build a set of ba- sic rules that rely on minimum domain knowl- edge and render the problem as binary clas- sification (affirmed vs. negated). Since the overall goal is to identify patients who are ex- pected to be at high risk of suicide, we focus on the evaluation of positive (affirmed) cases as determined by our classifier. Our negation detection approach yields a recall (sensitivity) value of 94.6% for the positive cases and an overall accuracy value of 91.9%. We believe that our approach can be integrated with other clinical Natural Language Processing tools in order to further advance information extrac- tion capabilities

AB - Mental Health Records (MHRs) contain free- text documentation about patients’ suicide and suicidality. In this paper, we address the prob- lem of determining whether grammatic vari- ants (inflections) of the word “suicide” are af- firmed or negated. To achieve this, we pop- ulate and annotate a dataset with over 6,000 sentences originating from a large repository of MHRs. The resulting dataset has high Inter- Annotator Agreement ( κ 0.93). Furthermore, we develop and propose a negation detection method that leverages syntactic features of text 1 . Using parse trees, we build a set of ba- sic rules that rely on minimum domain knowl- edge and render the problem as binary clas- sification (affirmed vs. negated). Since the overall goal is to identify patients who are ex- pected to be at high risk of suicide, we focus on the evaluation of positive (affirmed) cases as determined by our classifier. Our negation detection approach yields a recall (sensitivity) value of 94.6% for the positive cases and an overall accuracy value of 91.9%. We believe that our approach can be integrated with other clinical Natural Language Processing tools in order to further advance information extrac- tion capabilities

UR - http://www.scopus.com/inward/record.url?scp=85120006875&partnerID=8YFLogxK

M3 - Conference paper

T3 - Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016

SP - 95

EP - 105

BT - Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology

A2 - Hollingshead, Kristy

A2 - Ungar, Lyle

PB - Association for Computational Linguistics (ACL)

Y2 - 16 June 2016

ER -

Gkotsis G , Velupillai S , Oellrich A, Dean H, Liakata M, Dutta R. Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records. In Hollingshead K, Ungar L, editors, Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016. Association for Computational Linguistics (ACL). 2016. p. 95-105. (Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych 2016 at the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2016).