A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets

I Bojic; J Halim; V Suharman; S Tar; QC Ong; D Phung; M Ravaut; S Joty; J Car

A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets

I Bojic, J Halim, V Suharman, S Tar, QC Ong, D Phung, M Ravaut, S Joty, J Car

Research output: Other contribution

Abstract

Low-quality data can cause downstream problems in high-stakes applications. Data-centric approach emphasizes on improving dataset quality to enhance model performance. High-quality datasets are needed for general-purpose Large Language Models (LLMs) training, as well as for domain-specific models, which are usually small in size as it is costly to engage a large number of domain experts for their creation. Thus, it is vital to ensure high-quality domain-specific training data. In this paper, we propose a framework for enhancing the data quality of original datasets1. We applied the proposed framework to four biomedical datasets and showed relative improvement of up to 3340tuning of retrieval/reader models on the BioASQ dataset when using back translation to enhance the original dataset quality.

Original language	Undefined/Unknown
Number of pages	14
Publication status	Published - 1 May 2023

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Cite this

@misc{074a92d3236c4736a00d4f7907f85d34,

title = "A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets",

abstract = "Low-quality data can cause downstream problems in high-stakes applications. Data-centric approach emphasizes on improving dataset quality to enhance model performance. High-quality datasets are needed for general-purpose Large Language Models (LLMs) training, as well as for domain-specific models, which are usually small in size as it is costly to engage a large number of domain experts for their creation. Thus, it is vital to ensure high-quality domain-specific training data. In this paper, we propose a framework for enhancing the data quality of original datasets1. We applied the proposed framework to four biomedical datasets and showed relative improvement of up to 3340tuning of retrieval/reader models on the BioASQ dataset when using back translation to enhance the original dataset quality.",

author = "I Bojic and J Halim and V Suharman and S Tar and QC Ong and D Phung and M Ravaut and S Joty and J Car",

year = "2023",

month = may,

day = "1",

language = "Undefined/Unknown",

type = "Other",

}

TY - GEN

T1 - A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets

AU - Bojic, I

AU - Halim, J

AU - Suharman, V

AU - Tar, S

AU - Ong, QC

AU - Phung, D

AU - Ravaut, M

AU - Joty, S

AU - Car, J

PY - 2023/5/1

Y1 - 2023/5/1

N2 - Low-quality data can cause downstream problems in high-stakes applications. Data-centric approach emphasizes on improving dataset quality to enhance model performance. High-quality datasets are needed for general-purpose Large Language Models (LLMs) training, as well as for domain-specific models, which are usually small in size as it is costly to engage a large number of domain experts for their creation. Thus, it is vital to ensure high-quality domain-specific training data. In this paper, we propose a framework for enhancing the data quality of original datasets1. We applied the proposed framework to four biomedical datasets and showed relative improvement of up to 3340tuning of retrieval/reader models on the BioASQ dataset when using back translation to enhance the original dataset quality.

AB - Low-quality data can cause downstream problems in high-stakes applications. Data-centric approach emphasizes on improving dataset quality to enhance model performance. High-quality datasets are needed for general-purpose Large Language Models (LLMs) training, as well as for domain-specific models, which are usually small in size as it is costly to engage a large number of domain experts for their creation. Thus, it is vital to ensure high-quality domain-specific training data. In this paper, we propose a framework for enhancing the data quality of original datasets1. We applied the proposed framework to four biomedical datasets and showed relative improvement of up to 3340tuning of retrieval/reader models on the BioASQ dataset when using back translation to enhance the original dataset quality.

M3 - Other contribution

ER -