The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) Shared Task

Rami Aly, Zhijiang Guo, Michael Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

25 Citations (Scopus)

Abstract

The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) shared task, asks participating systems to determine whether human-authored claims are SUPPORTED or REFUTED based on evidence retrieved from Wikipedia (or NOTENOUGHINFO if the claim cannot be verified). Compared to the FEVER 2018 shared task, the main challenge is the addition of structured data (tables and lists) as a source of evidence. The claims in the FEVEROUS dataset can be verified using only structured evidence, only unstructured evidence, or a mixture of both. Submissions are evaluated using the FEVEROUS score that combines label accuracy and evidence retrieval. Unlike FEVER 2018 (Thorne et al., 2018a), FEVEROUS requires partial evidence to be returned for NOTENOUGHINFO claims, and the claims are longer and thus more complex. The shared task received 13 entries, six of which were able to beat the baseline system. The winning team was “Bust a move!”, achieving a FEVEROUS score of 27% (+9% compared to the baseline). In this paper we describe the shared task, present the full results and highlight commonalities and innovations among the participating systems.

Original languageEnglish
Title of host publicationFEVER 2021 - Fact Extraction and VERification, Proceedings of the 4th Workshop
EditorsRami Aly, Christos Christodoulopoulos, Oana Cocarascu, Zhijiang Guo, Arpit Mittal, Michael Schlichtkrull, James Thorne, Andreas Vlachos
PublisherAssociation for Computational Linguistics (ACL)
Pages1-13
Number of pages13
ISBN (Electronic)9781955917032
Publication statusPublished - 2021
Event4th Workshop on Fact Extraction and VERification, FEVER 2021 - Punta Cana, Dominican Republic
Duration: 10 Nov 2021 → …

Publication series

NameFEVER 2021 - Fact Extraction and VERification, Proceedings of the 4th Workshop

Conference

Conference4th Workshop on Fact Extraction and VERification, FEVER 2021
Country/TerritoryDominican Republic
CityPunta Cana
Period10/11/2021 → …

Fingerprint

Dive into the research topics of 'The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) Shared Task'. Together they form a unique fingerprint.

Cite this