Graduate School and Research Center in Digital Sciences

An ontology-based evidential framework for video indexing using high-level multimodal fusion

Benmokhtar, Rachid; Huet, Benoit

"Multimedia Tools Application", Springer, December 2011

This paper deals with information retrieval and semantic indexing of multimedia documents. We propose a generic scheme combining an ontologybased evidential framework and high-level multimodal fusion, aimed at recognising semantic concepts in videos. This work is represented on two stages: First, the adaptation of evidence theory to neural network, thus giving Neural Network based on Evidence Theory (NNET). This theory presents two important information for decision-making compared to the probabilistic methods: belief degree and system ignorance. The NNET is then improved further by incorporating the relationship between descriptors and concepts, modeled by a weight vector based on entropy and perplexity. The combination of this vector with the classifiers outputs, gives us a new model called Perplexity-based Evidential Neural Network (PENN). Secondly, an ontology-based concept is introduced via the influence representation of the relations between concepts and the ontological readjustment of the confidence values. To represent this relationship, three types of information are computed: low-level visual descriptors, concept co-occurrence and semantic similarities. The final system is called Ontological-PENN. A comparison between the main similarity construction methodologies are proposed. Experimental results using the TRECVid dataset are presented to support the effectiveness of our scheme.

Doi Bibtex

Title:An ontology-based evidential framework for video indexing using high-level multimodal fusion
Keywords:Video shots indexing · Semantic gap · Classification · Classifier fusion · Inter-concepts similarity · Ontology · LSCOM-lite ·TRECVid
Type:Journal
Language:English
City:
Date:
Department:Data Science
Eurecom ref:3597
Copyright: © Springer. Personal use of this material is permitted. The definitive version of this paper was published in "Multimedia Tools Application", Springer, December 2011 and is available at : http://dx.doi.org/10.1007/s11042-011-0936-5
Bibtex: @article{EURECOM+3597, doi = {http://dx.doi.org/10.1007/s11042-011-0936-5}, year = {2011}, month = {12}, title = {{A}n ontology-based evidential framework for video indexing using high-level multimodal fusion}, author = {{B}enmokhtar, {R}achid and {H}uet, {B}enoit}, journal = {"{M}ultimedia {T}ools {A}pplication", {S}pringer, {D}ecember 2011}, url = {http://www.eurecom.fr/publication/3597} }
See also: