CIKM 2019, 28th ACM International Conference on Information and Knowledge Management, November 3rd-7th, 2019, Beijing, China
Fact checking is the task of determining if a given claim holds. Several
algorithms have been developed to check claims with reference
information in the form of facts in a knowledge base. While individual
algorithms have been experimentally evaluated in the past,
we provide the first comprehensive and publicly available benchmark
infrastructure for evaluating methods across a wide range of
assumptions about the claims and the reference information. We
show how, by changing the popularity, transparency, homogeneity,
and functionality properties of the facts in an experiment, it is possible
to influence significantly the performance of the fact checking
algorithms. We introduce a benchmark framework to systematically
enforce such properties in training and testing datasets with
fine tune control over their properties. We then use our benchmark
to compare fact checking algorithms with one another, as well as
with methods that can solve the link prediction task in knowledge
bases. Our evaluation shows the impact of the four data properties
on the qualitative performance of the fact checking solutions and
reveals a number of new insights concerning their applicability and
performance.
Type:
Conférence
City:
Beijing
Date:
2019-11-03
Department:
Data Science
Eurecom Ref:
5996
Copyright:
© ACM, 2019. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in CIKM 2019, 28th ACM International Conference on Information and Knowledge Management, November 3rd-7th, 2019, Beijing, China http://dx.doi.org/10.1145/3357384.3358036
See also:
