Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

32 Downloads (Pure)

Abstract

Evidence data for automated fact-checking (AFC) can be in multiple modalities such as text, tables, images, audio, or video. While there is increasing interest in using images for AFC, previous works mostly focus on detecting manipulated or fake images. We propose a novel task, chart-based fact-checking, and introduce ChartBERT as the first model for AFC against chart evidence. ChartBERT leverages textual, structural and visual information of charts to determine the veracity of textual claims. For evaluation, we create ChartFC, a new dataset of 15, 886 charts. We systematically evaluate 75 different vision-language (VL) baselines and show that ChartBERT outperforms VL models, achieving 63.8% accuracy. Our results suggest that the task is complex yet feasible, with many challenges ahead.
Original languageEnglish
Title of host publicationFindings of the Association for Computational Linguistics
Subtitle of host publicationEACL 2023 - Findings
PublisherAssociation for Computational Linguistics (ACL)
Publication statusPublished - 2023
Event17th Conference of the European Chapter of the Association for Computational Linguistics: EACL 2023 - Dubrovnik, Croatia
Duration: 2 May 20234 May 2023

Publication series

NameFindings of the Association for Computational Linguistics: EACL 2023 - Findings

Conference

Conference17th Conference of the European Chapter of the Association for Computational Linguistics
Country/TerritoryCroatia
CityDubrovnik
Period2/05/20234/05/2023

Keywords

  • fact checking
  • misinformation
  • chart misinformation
  • automated fact checking
  • fact verification
  • natural language processing
  • nlp

Fingerprint

Dive into the research topics of 'Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking'. Together they form a unique fingerprint.

Cite this