Graduate School and Research Center in Digital Sciences

NERD: A framework for evaluating named entity recognition tools in the Web of data

Rizzo, Giuseppe; Troncy, Raphaël

ISWC 2011, 10th International Semantic Web Conference, October 23-27, 2011, Bonn, Germany

In this paper, we present NERD, an evaluation framework we ave developed that records and analyzes ratings of Named Entity (NE) extraction and disambiguation tools working on English plain text articles performed by human beings. NERD enables the comparison of different popular Linked Data entity extractors which expose APIs such as AlchemyAPI, DBPedia Spotlight, Extractiv, OpenCalais and Zemanta. Given an article and a particular tool, a user can assess the precision of the named entities extracted, their typing and linked data URI provided for disambiguation and their subjective relevance for the text. All user interactions are stored in a database. We propose the NERD ontology that defines mappings between the types detected by the different NE extractors. The NERD framework enables then to visualize the comparative performance of these tools with respect to human assessment.

Document Bibtex

Title:NERD: A framework for evaluating named entity recognition tools in the Web of data
Keywords:Entity extraction, Linked Data, Natural Language Processing, Evaluation of Linked Data entity extraction tools
Type:Conference
Language:English
City:Bonn
Country:GERMANY
Date:
Department:Data Science
Eurecom ref:3515
Copyright: © Springer. Personal use of this material is permitted. The definitive version of this paper was published in ISWC 2011, 10th International Semantic Web Conference, October 23-27, 2011, Bonn, Germany and is available at :
Bibtex: @inproceedings{EURECOM+3515, year = {2011}, title = {{NERD}: {A} framework for evaluating named entity recognition tools in the {W}eb of data}, author = {{R}izzo, {G}iuseppe and {T}roncy, {R}apha{\"e}l }, booktitle = {{ISWC} 2011, 10th {I}nternational {S}emantic {W}eb {C}onference, {O}ctober 23-27, 2011, {B}onn, {G}ermany}, address = {{B}onn, {GERMANY}}, month = {10}, url = {http://www.eurecom.fr/publication/3515} }
See also: