Validating Transformers for Redaction of Text from Electronic Health Records in Real-World Healthcare

Zeljko Kraljevic*, Anthony Shek, Joshua Au Yeung, Ewart Jonathan Sheldon, Haris Shuaib, Mohammad Al-Agil, Xi Bai, Kawsar Noor, Anoop D. Shah, Richard Dobson, James Teo

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Protecting patient privacy in healthcare records is a top priority, and redaction is a commonly used method for obscuring directly identifiable information in text. Rule-based methods have been widely used, but their precision is often low causing over-redaction of text and frequently not being adaptable enough for non-standardised or unconventional structures of personal health information. Deep learning techniques have emerged as a promising solution, but implementing them in real-world environments poses challenges due to the differences in patient record structure and language across different departments, hospitals, and countries.In this study, we present AnonCAT, a transformer-based model and a blueprint on how deidentification models can be deployed in real-world healthcare. AnonCAT was trained through a process involving manually annotated redactions of real-world documents from three UK hospitals with different electronic health record systems and 3116 documents. The model achieved high performance in all three hospitals with a Recall of 0.99, 0.99 and 0.96.Our findings demonstrate the potential of deep learning techniques for improving the efficiency and accuracy of redaction in global healthcare data and highlight the importance of building workflows which not just use these models but are also able to continually fine-tune and audit the performance of these algorithms to ensure continuing effectiveness in real-world settings. This approach provides a blueprint for the real-world use of de-identifying algorithms through fine-tuning and localisation, the code together with tutorials is available on GitHub (https://github.com/CogStack/MedCAT).

Original languageEnglish
Pages (from-to)544-549
Number of pages6
JournalProceedings - 2023 IEEE 11th International Conference on Healthcare Informatics, ICHI 2023
DOIs
Publication statusPublished - 2023
Event11th IEEE International Conference on Healthcare Informatics, ICHI 2023 - Houston, United States
Duration: 26 Jun 202329 Jun 2023

Keywords

  • electronic health records
  • text deidentification
  • transformers

Cite this