Graduate School and Research Center in Digital Sciences

Scrutinizer: A mixed-initiative approach to large-scale, data-driven claim verification

Karagiannis, Georgios; Saeed, Mohammed; Papotti, Paolo; Trummer, Immanuel

VLDB 2020, 46th International Conference on Very Large Data Bases, 31 August-4 September 2020, Tokyo, Japan (Virtual Conference) / To be published in PVLDB 2020, Proceedings of the VLDB Endowment, Vol.13, N°11, August 2020

Organizations spend signi cant amounts of time and money to manually fact check text documents summarizing data. The goal of the Scrutinizer system is to reduce veri cation overheads by supporting human fact checkers in translating text claims into SQL queries on an database. Scrutinizer coordinates teams of human fact checkers. It reduces veri fication time by proposing queries or query fragments to the users. Those proposals are based on claim text classi ers, that gradually improve during the veri cation of a large document. In addition, Scrutinizer uses tentative execution of query candidates to narrow down the set of alternatives. The veri cation process is controlled by a cost-based optimizer. It optimizes the interaction with users and prioritizes claim veri cations. For the latter, it considers expected verifi cation overheads as well as the expected claim utility as training samples for the classi ers. We evaluate the Scrutinizer system using simulations and a user study with professional fact checkers, based on actual claims and data. Our experiments consistently demonstrate signi cant savings in veri cation time, without reducing result accuracy.

Document Arxiv Bibtex

Title:Scrutinizer: A mixed-initiative approach to large-scale, data-driven claim verification
Department:Data Science
Eurecom ref:6216
Copyright: VLDB
Bibtex: @inproceedings{EURECOM+6216, year = {2020}, title = {{S}crutinizer: {A} mixed-initiative approach to large-scale, data-driven claim verification}, author = {{K}aragiannis, {G}eorgios and {S}aeed, {M}ohammed and {P}apotti, {P}aolo and {T}rummer, {I}mmanuel}, booktitle = {{VLDB} 2020, 46th {I}nternational {C}onference on {V}ery {L}arge {D}ata {B}ases, 31 {A}ugust-4 {S}eptember 2020, {T}okyo, {J}apan ({V}irtual {C}onference) / {T}o be published in {PVLDB} 2020, {P}roceedings of the {VLDB} {E}ndowment, {V}ol.13, {N}°11, {A}ugust 2020 }, address = {{T}okyo, {JAPAN}}, month = {08}, url = {} }
See also: