Graduate School and Research Center in Digital Sciences

When textual and visual information join forces for multimedia retrieval

Safadi, Bahjat; Sahuguet, Mathilde; Huet, Benoit

ICMR 2014, ACM International Conference on Multimedia Retrieval, April 1-4, 2014, Glasgow, Scotland

Currently, popular search engines retrieve documents on the basis of text information. However, integrating the visual information with the text-based search for video and image retrieval is still a hot research topic. In this paper, we propose and evaluate a video search framework based on using visual information to enrich the classic text-based search for video retrieval. The framework extends conventional text-based search by fusing together text and visual scores, obtained from video subtitles (or automatic speech recognition) and visual concept detectors respectively. We attempt to overcome the so called problem of semantic gap by automatically mapping query text to semantic concepts. With the proposed framework, we endeavor to show experimentally, on a set of real world scenarios, that visual cues can effectively contribute to the quality improvement of video retrieval. Experimental results show that mapping text-based queries to visual concepts improves the performance of the search system. Moreover, when appropriately selecting the relevant visual concepts for a query, a very significant improvement of the system's performance is achieved.

Document Doi Bibtex

Title:When textual and visual information join forces for multimedia retrieval
Keywords:Multimedia retrieval, video search and visual cues
Type:Conference
Language:English
City:Glasgow
Country:UNITED KINGDOM
Date:
Department:Data Science
Eurecom ref:4257
Copyright: © ACM, 2014. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ICMR 2014, ACM International Conference on Multimedia Retrieval, April 1-4, 2014, Glasgow, Scotland http://dx.doi.org/10.1145/2578726.2578760
Bibtex: @inproceedings{EURECOM+4257, doi = {http://dx.doi.org/10.1145/2578726.2578760}, year = {2014}, title = {{W}hen textual and visual information join forces for multimedia retrieval}, author = {{S}afadi, {B}ahjat and {S}ahuguet, {M}athilde and {H}uet, {B}enoit}, booktitle = {{ICMR} 2014, {ACM} {I}nternational {C}onference on {M}ultimedia {R}etrieval, {A}pril 1-4, 2014, {G}lasgow, {S}cotland}, address = {{G}lasgow, {UNITED} {KINGDOM}}, month = {04}, url = {http://www.eurecom.fr/publication/4257} }
See also: