Graduate School and Research Center in Digital Sciences

VERT : automatic evaluation of video summaries

Li, Yingbo; Mérialdo, Bernard

ACMMM 2010, ACM Multimedia 2010, October 25-29, 2010, Firenze, Italy

Video Summarization has become an important tool for multimedia information processing, but the automatic evaluation of a video summarization system remains a challenge. A major issue is that an ideal "best" summary does not exist, although people can easily distinguish "good" from "bad" summaries. A similar situation arise in machine translation and text summarization, where specific automatic procedures, respectively BLEU and ROUGE, evaluate the quality of a candidate by comparing its local similarities with several human-generated references. These procedures are now routinely used in various benchmarks. In this paper, we extend this idea to the video domain and propose the VERT (Video Evaluation by Relevant Threshold) algorithm to automatically evaluate the quality of video summaries. VERT mimics the theories of BLEU and ROUGE, and counts the weighted number of overlapping selected units between the computer-generated video summary and several human-made references. Several variants of VERT are suggested and compared.

Document Doi Bibtex

Title:VERT : automatic evaluation of video summaries
Department:Data Science
Eurecom ref:3188
Copyright: © ACM, 2010. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACMMM 2010, ACM Multimedia 2010, October 25-29, 2010, Firenze, Italy
Bibtex: @inproceedings{EURECOM+3188, doi = { }, year = {2010}, title = {{VERT} : automatic evaluation of video summaries}, author = {{L}i, {Y}ingbo and {M}{\'e}rialdo, {B}ernard }, booktitle = {{ACMMM} 2010, {ACM} {M}ultimedia 2010, {O}ctober 25-29, 2010, {F}irenze, {I}taly}, address = {{F}irenze, {ITALY}}, month = {10}, url = {} }
See also: